IBM® recently released the IBM Common Data Provider for IBM Z Sample Insight Dashboards for Elastic Stack that demonstrates how to use mainframe operational data, that has been streamed by IBM Common Data Provider for z Systems V1.1.0 from a z/OS-based IT operations environment, to an Elastic Stack environment. It enables customers to identify, isolate, and resolve problems across their enterprise from a single interface. This set of sample dashboards can easily be downloaded and installed from IBM DeveloperWorks.

To access the IBM Common Data Provider for IBM Z Sample Insight Dashboards for Elastic Stack, see the Files section of the System z Management on-line community: ibm.biz/syszmgmt
Look for a file similar to CDPz-Sample-Dashboards-ELK-V1.5.0-20180306.zip

IBM Common Data Provider for IBM Z Sample Insight Dashboards run on Elastic Stack, formerly known as, and still commonly referred to as, the “ELK” stack. Elastic Stack has become a very common open source platform for analytics on machine data. The Kibana visualization component of Elastic Stack hosts dashboards that show near real-time data from the z/OS system console log (SYSLOG) and SMF 30. The dashboards show subsystem information on performance and message indicators based on IBM CICS Transaction Server, IBM Db2 and IBM MQ subsystems. These dashboards can be used out of the box for immediate value from your operational data or they can be used as a starting place to create your own specific Kibana dashboards. Sample data is included in the sample download for validation before IBM Common Data Provider data is available.

To install the sample dashboards and sample data, use the instructions, also posted to the System z Management community, in the Files section. Look for a PDF file with a name similar to CDPz-Sample-Dashboards-ELK-20180307.pdf. This PDF file has pre-requisite information, detailed installation and validation instructions, and great information on how to explore and modify the Kibana sample dashboards. Beyond the sample data provided, to use real IBM Z operational data, you will want to have the IBM Common Data Provider for z Systems installed on at least one of your z/OS Systems and capable of streaming data to your Elastic Stack environment. The IBM Common Data Provider for z Systems ELK Data Ingestion Kit must be installed in your Elastic Stack environment. This kit is included with the IBM Common Data Provider for z Systems mainframe installation and must be installed on the Logstash server. See the IBM Common Data Provider for z Systems knowledge center for information on downloading and installing the ELK Data Ingestion Kit.

The IBM Common Data Provider for z Systems must be configured to send the desired data to Elastic Stack. You will need to include SMF 30 and SYSLOG or Operlog data. The IBM Common Data Provider for z Systems is configured using the z/OSMF web-based configuration tool. Below, is a sample configuration that can be used to feed the Sample Insight Dashboards.

Notice there are two data streams defined, those being the “z/OS SYSLOG” and the “SMF_030”. Each of the streams include a transform to convert the data to UTF-8. The z/OS SYSLOG data is configured with an extra transform to split the data flow such that Elastic is passed individual messages. Finally, the data is sent to a single Elastic server.

Use the pen icon to edit the z/OS SYSLOG Data Stream in the above example, to display the details for that data stream, as shown below.

Similarly, use the pen icon to edit the z/OS SMF_030 Data Stream in the above example, to display the details for that data stream, as shown below.

This panel lets you customize some of the parameters for the data stream, though for the z/OS System Console, the defaults are fine. The entry for the “Data Source Type” should be noted as it will be used later in Elastic Stack to restrict searches to specific data source types.

The Common Data Provider for z Systems is capable of very complex configurations where different data is being collecting, transformed in a variety of ways and sent to different servers, below is the subscriber configuration, where you will config the server to receive data.

On Logstash side server, there is a corresponding config file, the port number should match.
eg: /opt/ibm/cdp_config/B_CDPz_Input.lsh
input { tcp {
port => 8081
codec => “json”}
}

Once you have the IBM Common Data Provider for z Systems running with the above configuration, or the provided sample data loaded, you can begin using the sample dashboards on Elastic Stack. You can start by selecting the Dashboard tab in the Kibana left navigator

This will bring you to a list of sample dashboards including a Welcome Dashboard. The Welcome Dashboard provides an overview of the IBM Common Data Provider for IBM Z Sample Insight Dashboards for Elastic Stack.
From the list of dashboards, select the CICS Dashboard.

The initial set of graphs display performance metrics, such as CPU Time and SRB Time, from the SMF_030 data source type. The performance metrics graphs above are using the sample data provided with the sample dashboard.

Scroll down to see histograms of related CICS messages from SYSLOG.

A Systems Dashboard is provided, showing summaries of major subsystems, such as CICS, Db2, and MQ.

A z/OS Dashboard is also provided

For those that are very familiar with Elastic Stack’s query language, it’s possible to customize the sample dashboards, and use the provided sample dashboards as templates for other dashboards.

If you want to update an existing insight, or create new insight, you can open the Management->Index Patterns to update the script field or add new field with a formula, but be careful that the script field has the potential to damage the insight dashboard.

For searches and visualizations, you can open Management->Saved Objects->Visualizations, choose one visualization, click the button “View Visualization” to update the detailed search statement, add a filter, or update the detail of the data metrics.

.
A lot can be told of the status of your mainframe by simply using the SMF 30 and SYSLOG records. However, this is just the tip of the iceberg of the great variety of data that can be streamed from your mainframe using the IBM Common Data Provider for z Systems and incorporated into your Elastic Stack environment. Hopefully this has given you quick start to understanding the data and how it can be used to manage the operations of your mainframe systems.

Again, to access the IBM Common Data Provider for IBM Z Sample Insight Dashboards for Elastic Stack, see the Files section of the System z Management on-line community: ibm.biz/syszmgmt

3/7/2018

Join The Discussion

Your email address will not be published. Required fields are marked *