Monitoring Streams with Prometheus and Grafana

As a Streams developer or admin, you might need a way to efficiently store and visualize current and historical Streams metrics.  For example, you could compare historical metrics to current values to determine the impact of enhancements or changes to application performance.  You could use current metrics to create application domain specific dashboards (e.g. Translate the nTuplesSubmitted metric into a meaningful application context for operations floor personnel).

This tutorial will introduce an approach to monitoring IBM Streams metrics using the Prometheus time-series database and Grafana dashboard solution and the Streams Metric Exporter.

The Streams Metric Exporter publishes built-in and custom metrics from the IBM Streams JMX Service to the Prometheus database. 
This provides a dynamic, interactive, and customizable capability to create dashboards that can focus on general IBM Streams objects (Jobs, Operators, etc.) as well as be tailored to specific application domains.

Below is a sample dashboard in Grafana using Streams metrics saved in Prometheus:

Streams Version 4.x:

• Tutorial
• Docker Container: docker pull bmwilli1/streams-metric-exporter:4
• Github Link: https://github.com/IBMStreams/streamsx.jmxclients/tree/v4/streams-metric-exporter

Streams Version 5.x:

• Tutorial Coming Soon...
• Docker Container: docker pull bmwilli1/streams-metric-exporter:5
• Github Link: https://github.com/IBMStreams/streamsx.jmxclients/tree/v5/streams-metric-exporter

#CloudPakforDataGroup