Build an IoT hub for streaming, storing, and analyzing sensor data in the cloud

In this tutorial, we present the high-level steps that are involved in connecting an Android device to the cloud and developing analytics models to analyze sensor data. By the end of this tutorial you should be able to set up your own IoT hub for streaming, storing and processing device data.

The following figure shows the architecture of our sample app.

cloud architecture

Connecting your Android device to the cloud and developing the analytic models for sensor data involves these five main steps:

  1. Create IBM Cloud account and install IoTool apps
  2. Create a Watson IoT Platform instance
  3. Create a simple Node-RED app
  4. Create a live dashboard using Node-RED
  5. Build an AI Classifier using IBM Watson Studio

Prerequisites

This tutorial requires an Android device (smartphone), an internet connection, and an IBM Cloud account.

Estimated time

1 – 2 hours.

Steps

Step 1. Create IBM Cloud account and install IoTool apps

In Step 1 you will create an account on IBM Cloud and install an application on your Android phone.

This figure below shows the Android phone display of a live graph representing recorded phone movement after the completion, installation, and configuration of the necessary apps.

IoTool dashboard

Complete the steps in the Create IBM Cloud account and install IoTool apps section of the workshop in my GitHub repository.

Step 2. Create a Watson IoT Platform instance

In Step 2 you will create an instance of the IBM Watson IoT Platform service and register your Android device for secure communication over MQTT and configure your Android phone to connect to your IoT Platform instance over MQTT using the IoTool app. The MQTT communication protocol works under the publish-subscribe model. Device(s) can publish (send) data onto a topic that other device(s) can subscribe (listen) to. You will also stream sensor data from your Android device to the cloud and monitor its contents in the IBM Watson IoT Platform.

Complete the steps in the Use IBM Watson IoT Platform section of the workshop in my GitHub repository.

Step 3. Create a simple Node-RED app

In Step 3 you will create a Node-RED app using IBM Cloud Starter Kits. The deployment will be automated using the Continuous Delivery cloud service. You will then connect the recently deployed Cloud Foundry application (i.e. your Node-RED app) to the Watson IoT Platform service. This will generate an API Key that will be used in the near future to authenticate your app. Next, you’ll develop a simple application using the Node-RED web IDE. The application will receive acceleration data streamed from the device via MQTT. The figure below shows a simple Node-RED flow.

simple flow

Last, you’ll test the simple application you just created. The application will display the streamed data in the debug console.

Complete the steps in the Use Node-RED programming tool section of the workshop in my GitHub repository.

Step 4. Create a live dashboard using Node-RED

In Step 4 you will create a live dashboard that will display the acceleration data streamed from the device in real-time. You will learn how to manipulate of the original msg object to extract the relevant information to be passed along to the dashboard nodes. The figure below shows a Node-RED flow showing acceleration.

acceleration flow

You will then add linear acceleration to the list of sensor measurements streamed by your phone, and add linear acceleration data to the live dashboard. You will learn how to copy, paste and edit existing nodes for better reuse of an existing flow. The figure below shows the linear acceleration flow.

linear acceleration flow

Last, you’ll add a rudimentary shake detection feature to the dashboard using the absolute acceleration as a figure-of-merit. When a shake is detected, a notification will be displayed. The figure below shows the notification flow.

notification acceleration flow

Complete the steps in the Create live dashboard on Node-RED section of the workshop in my GitHub repository.

Step 5. Build an AI Classifier using IBM Watson Studio

In Step 5 you will create a Node-RED flow that stores the measured acceleration data into a Cloudant database. The sensor data is labelled with a Boolean class identifier that represents whether the device was being shaken or not during data collection.The figure below shows the training flow in Node-RED.

node red training flow

Next you will create an empty project on IBM Watson Studio alongside its companion Cloud Object Storage service instance and create an empty notebook to develop a classifier model. You will then create a service credential to allow your IBM Watson Studio project access the data stored in your Cloudant database, load the sensor data from Cloudant into a Pandas DataFrame and use PySpark machine learning functions to train a logistic regression classifier model. Last, you’ll translate the trained classifier model into a Javascript function and use it to add a shake detection probability visualization to your Node-RED dashboard. The figure below shows the completed Node-RED flow.

classifier acceleration flow

Complete the steps in the Build an AI classifier using IBM Watson Studio section of the workshop in my GitHub repository.

Summary

You’ve now connected your Android device to the cloud and developed analytics models to analyze sensor data. You can now set up your own IoT hub for streaming, storing and processing device data.