Monitoring message flows has been of interest to many customers since the origin of IBM Integration Bus. Considerable progress in this area has been made with the addition of monitoring events and the Business Transaction Monitoring functionality. To learn more about these capabilities, see: Business Transaction Monitoring in IIB, Advanced usages of Business Transition Monitoring and Business Transaction Monitoring versus Record and Replay.
However, when additional capabilities are required, such as monitoring several IBM Integration Bus nodes together, obtaining detailed information about the flows; when advanced sorting and filtering is necessary or when statistics about the transaction throughputs or the processing times are expected; the Advanced Monitoring Asset for IBM Integration Bus is a powerful resource which has been designed to extend the capabilities of the Business Transaction Monitoring native functionality and to fulfill such advanced requirements.
In this article, we describe the Advanced Monitoring Asset for IBM Integration Bus in detail and also provide an evaluation version of this asset as side material.
Native monitoring capabilities offered by IBM Integration Bus
When defining message flows with the IBM Integration Toolkit, the developers can request monitoring events to be emitted when the flows execute. These events can later be received and used by monitoring applications such as the Business Transaction Monitoring functionality of IBM Integration Bus, IBM Business Monitor or custom applications to monitor the message flows or the business transactions executed by IBM Integration Bus.
Setting message flows to emit monitoring events
When defining a message flow using the IBM Integration Toolkit, a developer can configure it to emit events. These settings are located in the “Monitoring” page of the properties of the nodes:
From the “Monitoring” page, monitoring events can be added to the selected node or edited:
Business Transaction Monitoring
A “business transaction” is described by the IBM Integration Bus Knowledge Center as “A self-contained business function; for example, the booking of an airline ticket. A business transaction might be implemented as multiple user transactions or activities”. A business transaction may correspond to an IBM Integration Bus message flow or it may encompass several IBM Integration Bus message flows participating in the processing of the same data in sequence.
IBM Integration Bus includes a functionality for monitoring business transactions, named “Business Transaction Monitoring”. The global architecture used by this functionality is the following:
The upstream part of this architecture is the mechanism described above for generating monitoring events as MQ messages and addressing them to MQ topics. The downstream part includes:
- An inner component of IBM Integration Bus which subscribes to the topics in order to receive the monitoring events and record them into tables of a relational database.
- A GUI (Web application), included in the Web User Interface of IBM Integration Bus, enabling administrators to:
IBM Business Monitor
It is also possible to use IBM Business Monitor to monitor message flows or business transactions executed by IBM Integration Bus. IBM Business Monitor is an independent product which must be purchased in addition to IBM Integration Bus.
This approach is fully documented in IBM Integration Bus v10 Knowledge Center
Custom monitoring applications
It is also possible to implement custom applications for monitoring the message flows and the business transactions. Such applications must subscribe to the MQ topics used by IBM Integration Bus in order to receive the monitoring events, then extract the information about the flows from the events and store them somewhere, then provide a GUI to enable the administrators to get access to the information about the flows and perform analysis. The Advanced Monitoring Asset for IBM Integration Bus is a custom monitoring application implemented according to these principles.
Advanced Monitoring Asset for IBM Integration Bus
What is named “business flows” in this part of the article is the same as “business transactions” in the preceding part and in the IBM Integration Bus Knowledge Center. To be more explicit, a business flow means a series of actions which belong to the common processing of a set of data in IBM Integration Bus. It may correspond to an IBM Integration Bus message flow or it may encompass several IBM Integration Bus message flows participating in the processing of the same data in sequence. It might even include some actions performed outside IBM Integration Bus, if they also participate in the same processing and if they are able to generate monitoring events as MQ messages.
The Advanced Monitoring Asset for IBM Integration Bus was designed to monitor IBM Integration Bus message flows and to extend the native Business Transaction Monitoring capabilities of IBM Integration Bus.
- Enable the monitoring of flows executed on several integration nodes from a single GUI.
- Provide detailed information about the flows as well as sorting and filtering capabilities.
- Provide statistics about the throughputs, the processing times…
- Remove the need of a “monitoring model” (use a implicit one instead), so that new business flows can immediately be monitored as soon as their settings enable them to publish events and they are deployed.
The Advanced Monitoring Asset is an asset owned by the IBM Cloud Services organization which can be purchased by customers eager to get advanced monitoring capabilities for IBM Integration Bus. An evaluation version of the asset is included as side material of this article and can be downloaded freely (it is limited to retrieving and displaying not more than 30 matching instances at a time when querying the business flow instance data from the database). Full details about how to setup and use the asset are not included in this article (for that purpose, please read the technical documentation also included with the evaluation version as side material). The main intention of this article is to provide a high level view of the philosophy and the functionalities offered by the asset.
The main functionalities provided by the Advanced Monitoring Asset for IBM Integration Bus are:
- Extract data about the business flow instances executed by IBM Integration Bus from the monitoring event messages generated by the message flows and store them into a database.
- Provide a GUI enabling administrators to query, display and analyze these data.
The GUI part of the asset offers the following functionalities:
- Query the business flow instance data from the database with sorting and filtering them using various criteria (period of time, integration node, integration server, type of business flow, business data) and display the data retrieved as a table.
Display statistics as bar charts about the business flow instances retrieved by a query. Various
statistics are offered:
- About the throughputs by periods of time.
- About the throughputs by nodes/servers and by types of business flow.
- About the processing times by periods of time.
The statistics are displayed as bar charts built using HighCharts, a high quality framework for building business charts developed by the HighSoft company .
The architecture is designed as another monitoring application which subscribes to the monitoring events emitted by the IBM Integration Bus message flows and consumes them. As such, it shares the same base and some common features with the “Business Transaction Monitoring” functionality of IBM Integration Bus (introduced in the previous part). However, the Advanced Monitoring Asset subscribes to a different (more generic) topic, stores data into a database with a different structure and the GUI is also different.
Enabling the message flows to emit monitoring events
Enabling monitoring is done in almost the same way as described in the first part (“Setting message flows to emit monitoring events”). To access these settings, with the IBM Integration Bus Toolkit, open the message flow with the editor, select a node and retrieve the “Monitor” tab in the “Properties” view.
The only specifics of the Advanced Monitor Asset are that:
- For a common flow instance, it is possible to correlate together events coming from several different integration nodes and servers.
- It is possible to have several start events and several end or failed events for a business flow instance.
- The event name is used to hold the business flow type and the status (in addition to the event name).
Using the GUI
The GUI is a web application which can be accessed using a browser (Firefox or Internet Explorer). It first displays the main menu:
Querying/listing business flow instancesUsing the first action of the main menu (“See all business flows”), the business flow instances matching selection criteria (period of time when the business flow instance executed and/or business data) can be retrieved from the database and displayed as a scrollable table.
The business data used for filtering are those defined as the “Event payload” property of the message flow nodes (when defining the monitoring events to be emitted). Up to three data filters can be used.
Both simple and complex data can be used for filtering:
- For simple data, only the name and value must be specified.
- For complex data, the name must include a path to the single data (for example: “Details/Geography/Item/City”).
In the list of results, completed business flow instances are displayed in green, while failed business flow instances are displayed in red and in-process (or in-doubt) business flow instances in orange. The columns of the list are: the ID of the business flow instance (defined as the “Global transaction correlator” property in the message flow nodes), the current status (either “Started”, “In process”, “Completed” or “Failed”), the time when the business flow was started, the time when the business flow was ended, the business flow type (defined as the “Event name” property of the message flow nodes), the integration nodes and servers which executed the business flow instance (or part of it), the business data (defined as the “Event payload” property of the message flow nodes). Using the second action of the main menu (“See all failed business flows”), you can limit the query to failed business flow instances only, matching selection criteria, and display them as a scrollable table.
Displaying statistics about the business flow instances
From the list of business flow instances retrieved by a query, it is possible to display several kinds of statistics.
The statistics are displayed as bar charts built using HighCharts, a high quality framework for building business charts.
Statistics about throughputs by periods of time
The first kind of statistics displays information about the numbers of business flow instances processed by periods of time: the lines represent different periods of time and the lengths of the bars represent the number of instances for the periods of time.
Several actions above the bar chart enable you to change the view:
- With “More details (time)” and “Less details (time)”, it is possible to change the period of time used for splitting the data (by seconds, minutes, hours, days, weeks, months or years).
IBM Integration Bus message flows (or groups of message flows processing the same data) can be monitored in several ways. First, IBM Integration Bus provides an infrastructure for generating monitoring events and addressing them to MQ topics. Next, several applications can make use of these events in order to provide administrators with monitoring information. The first one to be considered is the Business Transaction Monitoring functionality of IBM Integration Bus. When the customer’s expectations go beyond the capabilities of this functionality, other options such as the Advanced Monitoring Asset for IBM Integration Bus should be considered.
The Advanced Monitoring Asset for IBM Integration Bus is an asset implemented by the IBM Cloud Services organization which can be purchased from this organization. It is built on top of the monitoring event infrastructure provided by IBM Integration Bus with an architecture similar the Business Transaction Monitoring functionality. It provides IBM Integration Bus administrators with advanced functionalities such as monitoring business flows executed by different integration nodes or different integration servers from a common GUI, detailed information about the flows, powerful sort and filter capabilities and statistics about the throughputs and the processing times displayed as charts. An evaluation version of the asset is available as side material.
An evaluation version of the asset is provided as side material. Its limitation is that not more than 30 lines can be retrieved when querying the business flow instances from the database.
- A technical documentation in PDF format.
- “IIBAdvMonMDBWeb.war” and “IIBAdvMonGUIWeb.war”: The files to be used to install the two parts of the asset (the Message Driven Bean part and the GUI part). For installing the applications, process as described in the chapter “Installing the asset modules on the application server” of the technical documentation.
- “CreateAdvMonDB_DB2.sql”: A sample DB2 script for creating the DB2 tables needed by the asset. Before running it to create the tables, review its contents and customize it for your environment.
- “CreateAdvMonDB_Oracle.sql”: A sample Oracle script for creating the Oracle tables needed by the asset. Before running it to create the tables, review the contents of this file and customize it for your environment.
- Three example flows ready for monitoring with the asset: