Overview

Skill Level: Intermediate

It is recommended to be familiar with the Watson IoT Platform and Raspberry Pi Sense HAT before beginning this recipe.

The Watson IoT Platform provides the ability to evaluate rules on device data sent to the cloud and initiate actions based on emerging conditions. In this recipe, see how those rules can run at the edge on a Dell Edge Gateway using the EAA SDK.

Ingredients

Step-by-step

  1. Introduction

    The IBM Watson IoT Platform Edge Analytics Agent (EAA) enables analytics to be run at the edge on devices and gateways.  Running analytics at the edge can be advantageous for a number of reasons:

      1.  Reduce network traffic, including costs and general congestion, since data is analyzed close to the source instead of sent to the cloud

      2.  Reduce reliance on always-on internet access

      3.  Decrease latency and execute mission or time critical analytics in near real-time 

      4.  Rules are managed in one central location in the cloud and then pushed to edge

    In this recipe we will see how we can enable the Dell Edge Gateway in order to run rules and take actions at the edge using the Edge Analytics SDK.  The following diagram illustrates the general architecture of the edge analytics environment:

    edge_arch_3

    In this recipe, we will use a Raspberry Pi with a Sense HAT as our IoT Device and the Dell Edge Gateway as our IoT gateway running the edge analytics.  

    The Edge Analytics Agent (EAA) is available as an SDK for devices that support Java.  The SDK supports a number of different types of connectors between the gateway, IoT devices, and the Edge Analytics Agent, including MQTT, Kafka and file connectors.  We will configure the EAA on the Dell gateway to communicate and manage our Raspberry Pi using an MQTT connector. Here is a simplified view of the architecture with the Raspberry Pi and Dell gateway:

    pi_dell_arch_small_2

    By the end of this recipe, we will have created a rule in the cloud to monitor sensor data coming from the Raspberry Pi. ¬†We will then push that rule to the Dell Edge Gateway and take an action when the conditions of the rule are met. ¬†Let’s get started by first registering our gateway with the Watson IoT service. ¬†

  2. Register Dell Edge Gateway with Watson IoT service and create application API key

    Launch your IoT Platform service and register your gateway.  If you are unfamiliar with how to register a gateway, check out this recipe for help:  https://developer.ibm.com/recipes/tutorials/how-to-register-gateways-in-ibm-watson-iot-platform/

    At the end of registering your gateway, you should receive an authentication token.  Make sure you save this information for later use in this recipe.

    credentials

    We now have our gateway registered with the Watson IoT platform service.  Make sure it is registered as a gateway as shown below:

    gw_registered

    Next, we need to create an application API key in order for the EAA to securely communicate with the Watson IoT Platform. ¬†First, click the “APPS” tab on the left hand side of the page and then click the +Generate API Key:

    app_api2

    Copy and save the API Key and Authentication Token.  We will need these later.  Leave the rest as default and click the Generate button.

    api_key

    We now have our gateway registered and our app API key for use later in this recipe.

  3. Configure the Dell Edge Gateway

    Before our Dell Edge Gateway can act as a gateway running the EAA, we need to install and configure some packages.  Open an SSH session and connect to your Dell gateway:

    ssh_session2

    We will use a couple different tools to help us run the EAA on the gateway. ¬†The first is Docker. ¬†If you are unfamiliar with Docker, check out this link to learn more. ¬†The second is Mosquitto. ¬†Mosquitto will be used to enable MQTT communication between the gateway and the Raspberry Pi. ¬†Later in this recipe we will run Mosquitto in a Docker container. ¬†For now, let’s just install Docker.

    In your ssh session, type the following command to install Docker:

    sudo snap install --devmode docker

    Working in Ubuntu Core Snappy can be constraining due to permissions and security reasons. ¬†To make things easier for installing and compiling the EAA SDK, we need to install the “classic” snap. ¬†The classic snap allows us to run the Ubuntu classic environment. ¬†

    In your ssh session, type the following command to install the classic snap:

    snap install classic --edge --devmode

    Once classic is installed, we can enter classic mode with the following command:

    sudo classic

    Now that we are in classic mode, let’s get any updates:

    sudo apt-get update

    And now install any updates:

    sudo apt-get -y upgrade

    Once the updates are complete, let’s install the nano file editor:

    sudo apt-get install nano

    We will need the unzip utility:

    sudo apt-get install unzip

    And finally we will need a Java Development Kit (JDK) to compile and run the EAA SDK:

    sudo apt-get install default-jdk

    Our Dell Edge Gateway is now setup for us to install, compile and run the EAA SDK.

  4. Download, configure, compile and run the EAA

    The first thing we need to do is download the latest EAA SDK.  We can find the latest in the IBM Edge Analytics Community page here:  IBM EAA Community

    In the “Files” section of the page, download the “edge-sdk-1.0.0.zip” file.

    eaa_sdk_link

    Next, we need to transfer the edge-sdk-1.0.0.zip to the Dell gateway using SCP.  The mechanism for using SCP may depend on your operating system.  The following example uses a terminal window on a Mac.  Note that admin is the user name, 10.0.0.9 is the IP address of the Dell gateway and /home/admin is the directory to place the zip file in this example.

    scp edge-sdk-1.0.0.zip admin@10.0.0.9:/home/admin

    Once the edge-sdk-1.0.0.zip has been copied over to the Dell gateway, bring up the SSH session from step 3 of this recipe and make sure you are in classic mode.  Change to the /home/admin directory if you are not there already:

    cd /home/admin

    Now unzip the SDK using the following command:

    sudo unzip edge-sdk-1.0.0.zip

    You should now have a directory in your /home/admin directory called “edge-sdk-1.0.0”. ¬†Now we need to update a config file with our gateway registration information we obtained in step 2 of this recipe. ¬†Change to the¬†/home/admin/edge-sdk-1.0.0/samples/sample-config directory:

    cd /home/admin/edge-sdk-1.0.0/samples/sample-config

    We can use the nano editor we installed earlier to update this file:

    sudo nano sample_app.conf

    Edit the file and provide your organization, API Key and API Token, Gateway Type, Gateway Device
    and Gateway Device Token that you received from step 2 of this recipe:

    samples_config2

    Save the file by hitting control-X and then Yes to save.  Now we are ready to compile a sample application that uses the EAA.   Navigate to the following directory:

    cd /home/admin/edge-sdk-1.0.0/samples/src/main/java/com/ibm/iot/analytics/edgesdk/samples

    In this directory there are some sample applications.  Use nano to open the SampleApplication.java file if you would like to explore the code in this sample.  This sample application will create a couple simulated devices and send some simulated data to the Watson IoT Platform. To compile the application, run the following command:

    sudo javac -cp /home/admin/edge-sdk-1.0.0/libs/edge-sdk-1.0.0.jar SampleApplication.java

    Once compiled successfully, navigate to the following directory:

    cd /home/admin/edge-sdk-1.0.0/samples

    Finally, run the EAA sample application using the following command:

    sudo java -cp "/home/admin/edge-sdk-1.0.0/libs/edge-sdk-1.0.0.jar:\
    /home/admin/edge-sdk-1.0.0/samples/src/main/java" \
    com.ibm.iot.analytics.edgesdk.samples.SampleApplication

    We should now see that the Dell gateway has successfully connected to the Watson IoT Platform:

    gw_connected

  5. Examine simulated devices

    The SampleApplication.java program compiled in the previous step creates 2 simulated devices that communicate to the Watson IoT Platform through the EAA running on the Dell Edge Gateway.  The gateway registers these two simulated devices automatically.  Launch the Watson IoT Platform and have a look at the Devices tab:

    sim_devices

    Here we can see the two simulated devices, “SampleStringDeviceId” and “SampleCsvDeviceId.” ¬†We can also see the icon next to our Dell Edge Gateway shows us that the gateway is actively connected and sending data to the platform. ¬†Furthermore, if you click on the SampleCsvDeviceId, you will be able to see the JSON messages coming from the simulated device as well as the sensor information.

    samplecsv1

    samplecsv2

    Congratulations!  Your Dell Edge Gateway is now registering simulated devices and sending data from those devices to the Watson IoT Platform.  Now the real fun begins.  In the remainder of this recipe, we will add a real physical device, a Raspberry Pi with a Sense HAT, to the Dell Edge Gateway.  We will then explore how we can create rules and actions that execute at the edge.

  6. Run Mosquitto in a Docker container on the Dell Edge Gateway

    There are many different ways to connect sensors to a gateway.  Lots of protocols and interfaces exist today such as Bluetooth, Wifi, ethernet, serial (RS-232, Modbus), ZigBee, NFC, LPWAN and many, many others. As mentioned in the introduction to this recipe, we will have our Raspberry Pi Sense HAT communicate to our Dell Edge Gateway using the standard MQTT protocol over Wifi.  This requires us to have an MQTT broker on the gateway.  In this example, we will use an open source broker called Mosquitto.  If you are interested in learning more about Mosquitto, have a look here:  https://mosquitto.org/

    A simple way to run Mosquitto on the Dell Edge Gateway is to use a Docker container. ¬†Let’s go ahead and fire up Mosquitto on the gateway. ¬†Start a¬†new SSH session into the Dell gateway. ¬†Since we already installed Docker in step three of this recipe, we simply need to run the following command to start Mosquitto:

    sudo docker run -ti -p 1883:1883 -p 9001:9001 toke/mosquitto

    If successful, you will see mosquitto now running:

    mosquitto

     

  7. Create a Node-RED flow for the Raspberry Pi Sense HAT

    Node-RED provides an easy way to gather sensor data from the Raspberry Pi Sense HAT and send it as an MQTT message.  If you are unfamiliar with the Raspberry Pi Sense HAT and Node-RED, I recommend you have a look at the following recipe before proceeding any further: https://developer.ibm.com/recipes/tutorials/connecting-a-sense-hat-to-watson-iot-using-node-red/

    The first thing we need to do is create a simple flow that gathers some sensor values from the Sense HAT and sends them to our Mosquitto MQTT broker running on the gateway.  Go to your Node-RED flow editor on the Raspberry Pi.  This is typically accessed by browsing the URL http://<ip address of pi>:<port number for Node-RED>/  

    For example, http://10.0.0.48:1880/ where 10.0.0.48 is the IP address of the Raspberry Pi and 1880 is the port number for Node-RED.

    Start the flow by dragging on a Sense HAT input node and configuring it to output Motion events:

    node_red1

    Next, we want to limit the rate of messages we send to the gateway.  We can do this using the Delay node.  Drag the delay node onto the flow and configure it to Limit rate to 1 second and check the box drop intermediate messages.  

    node_red2a

    Next, we need to shift the data a bit. ¬†We need to move the “roll” data point to the top of the payload hierarchy. ¬†To do this, drag on a change node and configure it with Move Roll for a name and change the Rule to Move from msg.payload.orientation.roll to msg.payload.roll.

    node_red2b

    Now we want to send the data to the MQTT broker running on the gateway.  We do this using the mqtt output node.  Drag the mqtt output node onto the flow and connect the four nodes together.

    node_red3a

    Configure the mqtt output node by typing “motion” for the Topic and then click the pencil icon to Add new mqtt-broker‚Ķ

     node_red4

    In the Add new mqtt-broker config node, enter the IP address of the gateway into the Server field. ¬†Next enter the client ID in the format¬†d:<name of organization>:<device type>:<device ID>¬†¬† You can get your organization ID from step two of this recipe. ¬†For device type and device ID you can use whatever you want. ¬†Just make sure you take note of those values because you will need them later in this recipe. ¬†In the example below, we can see “ejvbaz” was used for the organization ID, “RasPiSenseHat” was used for device type and “b827eba44272” was used for the device ID. ¬†Leave all the rest of the fields as default and click the¬†Add¬†button and then click¬†Done¬†in the Edit mqtt out node dialog box.

    node_red5

    The Node-RED flow is now complete.  Click the Deploy button in the upper right hand corner of the Node-RED editor to execute your flow.  You should see that the Sense HAT and motion mqtt nodes are connected:

    node_red6a

    You should also see the new client connected in the SSH session running the Mosquitto broker from step six of this recipe:

     mosquitto2

    Now that we have our Raspberry Pi sending sensor data to the gateway, let’s create an MQTT connector using the EAA SDK and see the gateway register our device and forward the data to the cloud.

  8. Use the EAA SDK to create an MQTT connector

    We need to update the SampleApplication we compiled in step four of this recipe.  Open an SSH session to your gateway and enter classic mode:

    sudo classic

    Navigate to the following directory:

    cd /home/admin/edge-sdk-1.0.0/samples/src/main/java/com/ibm/iot/analytics/edgesdk/samples

    Edit the SampleApplication.java file using the nano editor:

    sudo nano SampleApplication.java

    Locate the code ‚ÄúLOGGER.info(‚ÄúDone submitting the file data event connector.‚ÄĚ);‚ÄĚ at approx line 66 and add the following code after. ¬†*NOTE*¬†make sure you replace¬†‚Äúb827eba44272‚ÄĚ and ‚ÄúRasPiSenseHat‚ÄĚ with the device ID and device type you used in step seven of this recipe.

         final StringConnector mqttConnector = createMqttConnector();

        eaa.deviceData("b827eba44272", "RasPiSenseHat", "mqttEvent", mqttConnector);

        LOGGER.info("Done submitting the mqtt subscriber connector.");

        System.out.println("Done submitting the mqtt subscriber connector.");

    Next, locate the line ‚Äúprivate static StringConnector createMqttConnector() throws IOException {‚ÄĚ and modify the function by replacing ‚Äúlocalhost‚ÄĚ with the IP address of the gateway and ‚ÄúedgeSdkTest‚ÄĚ with the topic ‚Äúmotion‚ÄĚ. ¬†For example:

     eaacode

    Now click control-X and yes to save and exit the nano editor.  Just like we did in step four of this recipe we now need to compile and run the EAA SampleApplication.  

    If not there already, navigate to the following directory:

    cd /home/admin/edge-sdk-1.0.0/samples/src/main/java/com/ibm/iot/analytics/edgesdk/samples

    To compile the application, run the following command:

    sudo javac -cp /home/admin/edge-sdk-1.0.0/libs/edge-sdk-1.0.0.jar SampleApplication.java

    Once compiled successfully, navigate to the following directory:

    cd /home/admin/edge-sdk-1.0.0/samples

    Finally, run the EAA sample application using the following command:

    sudo java -cp "/home/admin/edge-sdk-1.0.0/libs/edge-sdk-1.0.0.jar:\
    /home/admin/edge-sdk-1.0.0/samples/src/main/java" \
    com.ibm.iot.analytics.edgesdk.samples.SampleApplication

    If successful, we should see that the EAA has connected to the Watson IoT Platform and it is done creating the MQTT connector:

     eaa_mqtt

    We should also see the client gateway has connected to the Mosquitto broker:

    mosquitto3

     Finally, you should now see the Raspberry Pi device in the devices tab of the Watson IoT Platform.  

     device

    Clicking on the device allows us to view the motion sensor values from the Sense HAT:

    sensorsa

    Take a look at the roll data point.  It should be close to 0 or 360 when the Raspberry Pi Sense HAT is lying flat.  If you are looking at the front of the Raspberry Pi where the ethernet port is, you can roll the Pi to the left or right to see the roll data point change.

     

  9. Create message schema, rules and actions

    Ok, now that we are able to see the Raspberry Pi Sense Hat sensor data coming from the Dell gateway, we can begin to put some rules in place and take actions when certain conditions are met.  First we need to create a message schema.  Back in the Watson IoT devices tab, click the Manage Schemas tab:

    schema1

    Next, click the Add Schema button, choose the device type you used for the Raspberry Pi Sense HAT and then click Next.

    schema2

    In the next dialog, click the Add property button, then the From Connected tab, check the box next to Property to select all properties and finally click OK.

    schema3

    Back in the add schema dialog, click Finish to complete the creation of the message schema. ¬†Next, let’s create an Edge Rule. ¬†First, click the Rules button on the left hand side of the browser. ¬†Next, click the Create Edge Rule button. ¬†Fill in the details for the Name, Description and Applies to fields and then click Next.

    rule1

    In the rule editor, click the new condition box and set the property to roll, the operator to > and the value to 40, then select OK.

    rule2

    Click the + symbol next to AND and add the condition for roll < 320.  The condition should now look like the following.  This condition checks to see if the Sense HAT is rotated more than 40 degrees or less than 320 degrees.

    rule3

    Now to complete the rule, we can add an action.  Click the new action box and then the Add action link.

    rule4

    Give the action a name and choose the type to be Forward event to cloud then click Next.

    rule5

    Leave the default radio button selected and give the event name rollAlertEvent and click Finish.

    rule6

    Select the newly created action and click OK.

    rule7

    Lastly, now that the rule is complete, click Save.

    rule8

    Now that the rule is complete, we can see how it works in the next step.

  10. Activate rule and examine results

    We can activate a rule by clicking the Manage Rule button next to the rule in the browse rules dialog:

    manage_rule

    In the resulting dialog, make sure the Dell gateway is selected, Activate is selected and then click the Submit Command button.  If successful, you will receive a notification.

    activate_rule

    Now that the rule has been activated at the edge, let’s go back to the devices tab and have a look at the Raspberry Pi Sense HAT device.

    device 

    Click on the Raspberry Pi Sense Hat device to open up the device information.  Notice that there are no longer any messages coming in:

    device2

    The reason is because our Dell gateway is now examining the data at the edge and only sending data to the cloud when the “roll” data point breaks the rule. ¬†To try this out,¬†look at the front of the Raspberry Pi where the ethernet port is, and then¬†roll the Pi to the left or right. ¬†If you roll far enough in either direction, you will start to see “rollAlertEvents” coming in as we specified in the action of the Roll Alert rule. ¬†You can also see the current sensor values at the time of the alert.

    sensorsb

    Congratulations!  You have now created a rule in the cloud and pushed that rule to the edge onto a Dell Edge gateway.  With the power of the Watson IoT Platform and Bluemix, you could now easily send text messages or email alerts when the rollAlertEvent occurs.  Or even more powerful, automatically create and schedule work orders in an asset management tool such as IBM Maximo.

  11. Conclusion

    In this recipe, we have seen an example of how we can connect sensors (in the form of a Raspberry Pi with a Sense HAT) to a Dell Edge Gateway 3000.  We took a look at how we configure, compile and run the IBM Edge Analytics Agent on the Dell Edge Gateway 3000.  Most importantly, we examined how we create rules in the cloud using the Watson IoT Platform and then push those rules to the edge to run on the Dell Edge Gateway using the Edge Analytics SDK.  This allowed us to run the analytics close to the physical device at the edge and only send data to the cloud when certain conditions were met.  

    There is a lot more we can do with the IBM Watson IoT Edge Analytics SDK and the Dell Edge Gateway.  For example, handling alerts and device actions at the edge.  If you want to learn more, check out this recipe: https://developer.ibm.com/recipes/tutorials/handling-alerts-and-device-actions-using-the-ibm-edge-analytics-sdk-from-watson-iot-platform/

    Lastly, here is a picture showing the major parts of the completed architecture: 

    pi_dell_arch_3

1 comment on"Getting Started with Watson IoT Edge Analytics on a Dell Edge Gateway 3000"

  1. Great recipe! Easy to follow and well-written.

    For those grabbing the most recent edge-sdk, be sure to update your sdk path on the command line. One thing of note, is that with the current version of the SDK (1.0.1 as of May 1, 2017), the packaged sdk jar is still at 1.0.0 in the libs directory.

    Here is a short shell script I used to invoke the agent from the command line. Simply set the path to the root location where you downloaded the SDK.

    #!/bin/bash
    EDGE_SDK_HOME=”/home/rjv/edge-sdk-1.0.1″
    cd $EDGE_SDK_HOME/samples
    java -cp “$EDGE_SDK_HOME/libs/edge-sdk-1.0.0.jar:\
    $EDGE_SDK_HOME/samples/src/main/java/com/ibm/iot/analytics/edgesdk/samples:\
    $EDGE_SDK_HOME/samples/src/main/java” com.ibm.iot.analytics.edgesdk.samples.SampleApplication

Join The Discussion