Monitoring with containerized applications
Now, with modern applications, and modern application monitoring, all enterprises are taking advantage of the portability aspect of containers. Containers are developed locally and run locally on a laptop or server. As you develop containers in the Cloud, you can deploy them in an on-premise environment, enabling you to leverage the environment that suits your needs.
The problem emerges when you have to monitor these environments by using a single monitoring tool. The best approach to adopt is to ensure that your monitoring tools mirror the application environment. Itâ€™s important to take a proactive approach to monitoring and to focus on ways to improve performance and this is the approach that IBM has taken with its Cloud monitoring tools.
IBM Cloud monitoring tools â€“ a different approach
Nowadays, there is an increased focus on monitoring the containers and the workloads within the containers. To help improve the end-to-end application life-cycle, IBM provides many light-weight tools that help you to monitor your workloads easily when moving from development to production. The new IBM cloud monitoring tools take a different view. The tools get you started quickly. There is no server or additional infrastructure requirement that helps a developer to get started in a few minutes. When the application is deployed into production, the monitoring architecture mirrors your application. This enables you to handle complex deployments requiring no additional change to your containerized applications.
Microclimate and IBM runtime tools integrate monitoring and an application is instrumented automatically with the corresponding data collector. This integration enables monitoring to mirror the application, despite the method that you use to develop and deploy your application. Therefore, monitoring is proactive and you can use the monitoring data as part of your development to prepare for the deployment of the application into production. Currently, the IBM runtime tools support Node, Swift, Java runtimes and they will be extended to support more runtimes.
Steps on the monitoring journey
To guide you through the monitoring journey, we recommend the following steps:
Note: It is assumed that your application is a Node application. However similar steps apply to other runtimes, such as Swift, Java, and Python.
1. Configuring the data collector into your application
- Use the IBM Cloud Developer tools (ibmcloud dev create) to generate a default application project by selecting the appropriate project type for deploying your application into the IBM public or private cloud.
â€˘ This step bundles the data collector into your application automatically. For more information, see IBM Cloud Developer Tools (CLI and Dev Tools) commands.
- â€śWant to use IBM cloud private?â€ť â€śGreat, the same data collector works with the Microclimate too!â€ť When you create, a project using Microclimate or import an existing project from GIT using Microclimate, the data collector is automatically bundled into your application. For more information, see Helping developers build services for IBM Cloud Private.
- If you already have an application project and are not using IBM Cloud Developer tools, follow the Application Metrics for Node.js steps to configure the data collector into your existing Node application. You can use IBM Cloud Developer tools to automate these steps.
- In development mode, you donâ€™t need anything beyond the configuration of the data collector. The data collector collects the performance data from the application runtime stack and you can visualize the performance data by connecting a web browser to the data collector requiring no additional infrastructure. â€śItâ€™s that simple!â€ť
â€˘ The default URL to see the monitoring data for your running Node application is: http://localhost:3001/appmetrics-dash.
- When your application scales, the data collector also scales with your application because it is part of your application. This step produces the high-level architecture that is shown here:
Development Phase architecture
â€˘ However, this architecture has some limitations. The developer can only see the performance of one application without any cross-application transaction correlation. This may not be a major limitation as typically developers focus on getting one application or a small set of applications configured.
2. Deploying your application into production
When your application goes into production, you donâ€™t need to re-install the data collector or change your containerized image that you tested already. Your application is already instrumented with the data collector for monitoring purposes.
3. You have IBM Application Performance Management v8.x?
If you have IBM Application Performance Management v8.x, then this isn’t a problem. You use the same data collector to connect to your IBM Application Performance Management v8.x server. It requires you to configure a few environment variables. To specifically configure a Node.js data collector, see the Node js. data collector and Node.js monitoring. Then, to configure all other data collectors, see Getting started with standalone data collectors.
4. How does it all work?
- Data collector
The data collector is specific to each runtime although all of them share the same architecture. The data collector is just a small package like any other application package. It can be installed using npm for Node applications, pip for Python applications, or installUtility for Liberty applications. Once the data collector is installed and configured, it runs in the same application process space as your application. It has access to the application runtime stack API and your application classes. It collects Garbage collection data, memory, and CPU utilization by calling the runtime stack API.
- Application response times
Your application response times, throughput, stack traces, and method traces are typically collected by various class instrumentation techniques, such as, byte code modification. Your application classes are slightly modified to insert call backs. The call backs are triggered as part of your application request processing. When the call backs are triggered, data is collected and put in a queue to be processed asynchronously. Care is taken to minimize data processing on the application threads. This limits the impact to your application response times. All the performance data is collected as your request is processed but itâ€™s uploaded in a less frequent basis to the server. By default, the performance data is uploaded by the data collector once a minute but the upload interval can be configured to suit your needs.
- Data collector configuration
The Data collector is configuration driven. It has multiple output adapters to send the performance data to different upstream servers. It can send data to the IBM Application Performance Management Server v8, and the IBM Cloud Application Management Server. It scales with your application because it is part of your application containerized image requiring no additional configuration or special scalability considerations.
- IBM Cloud Application Performance Management Server
The IBM Cloud Application Performance Management Server is where the data from the different data collectors from various applications is correlated together. The server provides various end-to-end monitoring and diagnostic workflows to detect, isolate, and diagnose your business application problems across multiple services and applications.
You can dedicate some of your business application to an on -premises environment and some of it to public or private clouds. The IBM Cloud Application Management Server is a cloud native application comprised of many micro services. It is scalable, secure, and efficient. You can integrate data from the server into other management systems easily because you can access most of the data via a REST API. You can also create custom data collectors and post data to it.
- Distribution of the data collectors
Data collectors are available through the runtime specific public package manager sites.
- Download the node data collector for node applications (â€śappmetricsâ€ť package) from NPM: (https://www.npmjs.com/package/appmetrics).
- Download the Liberty data collector for Liberty applications (â€śjavametricsâ€ť package) from the Liberty repository: (https://developer.ibm.com/wasdev/downloads/).
The Liberty package will be available soon (end of June).
- The Swift data collector is available here: https://github.com/RuntimeTools/SwiftMetrics.
- The Python data collector will be available here: pypi.org.
- The Ruby data collector will be available here: rubygems.org.
The dates for the Python and Ruby data collector availability on public package manager sites are still to be determined.
- Data collectors are also available for download in Passport Advantage, for more information see: Part numbers.
Author: Ravi Gadekarla
Technical Contributor: Randy George
Technical Editor/Blog Creator: Carmel Burgess