Introduction to Streams Flows

Streams Flows is a web based IDE for quickly creating streaming analytics applications in IBM Watson Studio. The applications are created in the browser and run on the IBM Cloud. This post is collection of short videos and articles to introduce you to the the canvas and show you how to create and monitor an application.

Table of Contents

Overview

Why should you use Streams Flows? This video provides an overview as well as an introduction to the Streams Flows canvas.

 

Set up the IBM Cloud services

Streams Flows is part of IBM Watson Studio, so you’ll first need to log in/create an account. Applications created with Streams Flows run on the Streaming Analytics service in the IBM Cloud, so after creating an account, follow along in this video to set up the needed services.

Now that your setup is complete, a great way to try out Streams Flows is by running a example application.

Running an example flow

Applications created in Streams Flows are called flows.

In this video, you will learn how you can deploy the Data Historian example flow. This flow ingests data from simulated weather stations and uses the Aggregation operator in Streams Flows to compute statistics like average temperature and humidity.

 

Learn more about this example

Monitor the running flow

The next video in the series demonstrates how you can monitor a running application using the metrics page. You can observe the application’s performance, see the data as it moves between operators and download application logs.

Create your own application


After running an example flow and learning how to interact with a running flow, you’re now ready to create your own applications.

Create a Streams flow with the canvas

Extend the Data Historian example to use Message Hub as a data source

You’ve successfully run an example flow. Now, you probably want to start creating your own applications. Logically, the first step in creating your own flow is connecting to a data source. Right now, supported data sources are the Watson IoT Platform or Message Hub. So now you need to learn how to send data to one of those services.

Follow along in this notebook to see how to modify the Data Historian flow to use data from Message Hub, IBM’s Apache Kafka offering. You will learn how to 1) Send data to Message Hub using Python, 2) Ingest and analyze that data in a Streams flow, and 3) Send results from the flow back to Message Hub.

Open the notebook on the Data Science Experience, and after logging in, click copy to import the notebook into a project for use.

Use data from IoT devices with the WatsonIoT operator

Another common data source is data from Internet of Things (IoT) devices. These are ingested in Streams Flows using the WatsonIoT operator.
Watch this video to learn how to use it:

Download the complete application from Github

Computing moving averages and running totals with the Aggregation operator

You may have noticed in the example flow that the Aggregation operator was used to compute general statistics like averages, max/min, totals, and so on. Learn more about the Aggregation operator and how to use it in this post.

Add custom code using the Python operator

Your application might require customized logic for tuple processing, or you might want to connect to a different database that isn’t currently supported as a source or target operator, such as Cassandra.
You could do so using the Python code operator, and this video shows how to do that:

Add predictive modeling to your Flows

Extract more insights from your streaming data by adding machine learning using the SPSS Model operator. You can train a model on existing data and then apply that model to your streaming data by deploying it in your flow. Watch this video for a walk-through on training and deploying a new model in Watson Studio.

Download the complete application from Github

Useful Links

We are still working on more content, so stay tuned!

Join The Discussion