The Streams Console is a web application that allows you to easily monitor and manage your Streams instance and jobs. You can quickly gain insights on the health, metrics, issues and performance of your applications and the systems they are running on.
In Streams on Cloud Pak for Data, the Streams Console is a more advanced alternative to the Job Graph.
This article describes the most common features and how to use them.
Streams Console
Opening the Streams Console
Opening the Streams Console will depend on your environment.
IBM Cloud Pak for Data deployment
These instructions assume that you already have an IBM Streams service instance in IBM Cloud Pak for Data. If you need to provision a service instance, start here and follow the instructions.
- In the IBM Cloud Pak for Data web client, open the navigation menu and click on My instances.
- Click on the Provisioned instances tab.
- Find your Streams instance in the list. If your IBM Cloud Pak for Data version is 3.0+, click on the instance. Otherwise, open the context menu for the instance and click on View details. You will be brought to the instance details page.
- In the Endpoints section, find the property called External Console endpoint and click on the corresponding link.
IBM Streams standalone deployment
These instructions assume that you already have an IBM Streams instance. If you need to install an instance, start here and follow the instructions.
To find the Console URL, see the
documentation. Choose
Finding the internal URLs for IBM Streams services or
Finding the external URLs for IBM Streams services depending on whether or not you will be accessing the Streams Console from within the Kubernetes cluster.
IBM Streaming Analytics on IBM Cloud
These instructions assume that you already have a running IBM Streaming Analytics service. If you need to create a service, start here and follow the instructions.
- Navigate to the Resource list in IBM Cloud.
- In the Services section, find your Streaming Analytics service in the list and click on it. You will be brought to the service's details page.
- Click on the Launch button.
IBM Streams Quick Start Edition
These instructions assume that you already have the IBM Streams Quick Start Edition installed. If not, start here and follow the instructions.
There two ways to open the Streams Console:
- Using the bookmark:
- From the Applications menu, go to Favorites > Streams Console.
- Using the command line:
- From the Applications menu, go to Favorites > Terminal.
- Run the following commands:
$ source $STREAMS_INSTALL/bin/streamsprofile.sh
$ streamtool geturl
https://streamsqse.localdomain:8443/streams/domain/console
Open this URL in the Mozilla Firefox browser.
Use the following credentials to log in: streamsadmin/passw0rd.
IBM Streams extension for Visual Studio Code
These instructions assume that you already have the IBM Streams extension for Visual Studio Code installed. If not, start here and follow the instructions.
- Click on the IBM Streams icon on the left side to bring up the Streams Explorer view.
- Hover over an instance node and then click on the Open IBM Streams Console icon that appears on the right. This will open the Streams Console in your browser.
Submitting a job
After opening the Streams Console, you can follow these steps to submit a job to run on your Streams instance. Streams applications are always compiled into a file with a
.sab extension. SAB stands for Streams application bundle. This is a single, relocatable file that contains all the artifacts that are needed to run your application. Learn more about application bundle files
here.
So, when we say "submit a .sab file", we mean, run a compiled Streams application.
- Click on the play icon in the header bar at the top.
- Choose one of the options and point to your
.sab file.
- To configure the job submission, click on the Configure button. To submit the job with the default configuration, click on the Submit button. Learn more about job configuration here.
View a sample of streaming data
You can look at the data on a given stream to observe the data that is being processed or produced by an operator.
To observe the data, create a dashboard view:
- In the Application Dashboard, find the Streams Graph card and expand it.
- Hover over the stream that comes out of an operator and right-click.
- Select Create Dashboard View.
- Accept the default options and click on the OK button.
- Minimize the Streams Graph card.
- A new card is created in the dashboard that displays the tuples in the stream produced by the operator. Notice that the view is updated as new data arrives.
These steps are demonstrated in the following animation.
Creating a view on a stream
Managing and Monitoring Jobs
You can hover over an operator, a job, or a Stream to see available actions. You can perform actions like:
- Set application trace
- Restart PEs,
- Submit job
- View PE restart recommendations
- Download operator and PE logs
- Modify parallel regions
Adding Metric/Health charts and grids:
Monitor metrics and health through various configurable charts like pie charts, bar charts, scatter charts, etc and grids.
How to do it: Click
Add card in the console
In
Visualization Settings tab, choose a type of metric, e.g PE memory consumption, and in the
Card Properties, select the type of graph, e.g. bar chart.
Download and View Logs
Analyze application trace, product and console logs through a reader friendly viewer.
How to use it:
1. Open the log viewer from the far right pane of the Streams Console.
2. Select the job of interest. Expanding the tree and selecting an operator or PE will open the logs pane. Click any of the tabs: Application trace, Console log or Product log to show trace data, messages printed to standard out, and product logs, respectively
Streams Graph
This is a graphical view of your Streams Application which shows flow rates, health of connections & operators and allows you to apply customized metric color schemes.
Learn more
#CloudPakforDataGroup