Overview

Skill Level: Beginner

For anyone who wants to carry out analytic on the edge device using IBM Edge Analytics Agent

Recipe to show case how to use IBM Edge Analytics Agent to carry out analytic on the edge and forward the events to Cloud for further analytic.

Ingredients

  • Bluemix account.
  • Chrome Browser.
  • Install DGLux Tool bundled in Distributed Service Architecture (DSA).
  • Get license for DGLux Tool.

 

Step-by-step

  1. Introduction

    Edge Analytics is an approach to data collection and analysis in which an automated analytical computations are performed on data close to the source instead of waiting for the data to be sent back to a centralized location. These analytical services can run on a router, switch, gateway, or server in the operational environment. Analyzing data as it is generated decreases latency in responding to emerging conditions, and increases scalability by filtering out low value data and reducing the network burden of moving large volumes of data.By distributing data processing and analytics out to the edge of the network, you can more easily apply a rich set of analytics on site with flexibility to start small and scale up and still maintain continuous operations even if the network goes down.

    IBM announced its initial support for Edge Analytics with Cisco in June, 2016. In this recipe, we are going show how Edge Analytics works with a Raspberry Pi / System with the help of IBM Edge Analytic Agent and IBM Watson IoT Platform. IBM Edge Anaytics Agent is built on top of Apache Edgent. 

    Edge Analytics provides the opportunity to implement analytics on an Edge device that is deployed at the ‘edge’ of a network where data from connected devices is collected. This allows for Actions to be taken against the device data prior to that data being forwarded on Watson IOT Platform in the Cloud. Analyzing data at the Edge can decrease latency in the decision-making process on connected devices.

    IBM’s Watson IoT Platform Analytics provides an integrated cloud-and-edge analytics programming model that allows control and optimization over the data flowing between edge devices and the Cloud. A common interface allows you to create a rule to send only the critical data to the cloud for further analysis by filtering data at the Edge. That same interface allows Cloud-based rules to be created to perform analysis on the data received in the Cloud and perform actions when specific conditions in the data are met. When Edge Analytic Rule(s) are pushed to the edge device, the IBM IoT Edge Analytic Agent (EAA) running on a IoT Gateway(Edge device) knows how to process and respond to messages from the Watson IoT Platform. That is, the EAA accepts the defined analytics and executes the defined analytic(s) on the subsequent messages received from IoT devices that are attached to the Gateway.

    Below given diagram describes with the steps at the high level, the working of Edge Analytics:

    EAA_Arch

    In this recipe, we are going to see the steps involved in getting the IBM EAA and installing it in DGLux tool which is bundled with Distributed Services Architecture(DSA) Archive, and then defining some rules and actions to carry out Edge Analytics on Device Data using IBM Watson IoT Platform Analytics.

  2. Setup Watson IoT Platform

    We need a Gateway Device to be registered in Watson IoT Platform Instance, that’s going to be used in further steps below. Refer to Registering Gateway Recipe to setup Watson IoT Platform and Register Gateway Device with Watson IoT Platform.

    Note down the credential details for the device created under registered Gateway as these are needed while configuring IBM EAA Agent in next coming sections below.

    By now, we should have a Gateway Device added into the Watson IoT Platform as shown below:

  3. Setup IoT-DSA Platform

    DSA is an open source IoT platform that facilitates device inter-communication, logic and applications at every layer of the Internet of Things infrastructure. To know more about DSA, please refer to the link http://iot-dsa.org/.

    In this section, we are going to showcase the steps involved in downloading and installing IoT-DSA Platform.

    Setting up IoT-DSA Platform on Raspberry Pi 3 :      Install IoT-DSA on Raspberry Pi.

    Setting up IoT-DSA Platform on Laptop / VM :

    • Download IoT-DSA platform using Google Chrome Browser.
    • Make sure Java 1.8 is installed on the system.
    • Select System DS Link from the list provided.

    system-link

    • Follow the install instructions on the download page and Install DSA on the system.

    Logging in to DSA Server irrespective of the device :

    • These steps are common for working with DSA on Raspberry Pi as well as on System
    • Open a Terminal and change to the dglux-server directory in the extracted ZIP location
    • Run chmod 777 bin/*.sh
    • Run ./bin/daemon.sh start
    • Access DGLux at: http://localhost:8080.
    • Login to DGLux with default credentials : dgSuper / dglux1234

    dglux_login

    • After logging in, request for DGLux License:

    dglux_license

    • Fill the required details and click on Submit Request:

    dglux_license_fields

    • Requires a day time to receive license for the DGLux tool on the provided Email.
    • As the DGLux license is received on the Email, login to DGLux to agree for DGLux End User License Agreement

    • With that, we’ll be able to see a welcome screen in which we choose to select blank project followed by giving appropriate name to the project:

    • After successful creation of the blank project, click on Cancel to close Welcome Screen and then we’ll be able to see Project and Data tabs in DGLux tool, select Data tab for the created project:
    • After selecting the Data tab for the project, we’ll be able to see downstream and sys components.
    • Expand sys and we’ll be able to see sub-component links.
    • The new links are going to be installed under links
    • New nodes get added under downstream component after successful connection to links.
    • Expnand downstream and links to get feel of initial contents:

    ¬†This concludes this section’s purpose¬† to download , install and logging in to DSA Server either on Raspberry Pi or Laptop / VM and get initial feel of DGLux Tool.

  4. Setup IBM Edge Analytics Agent

    IBM EAA is a software component built on top of Apache Edgent to carry out Edge Analytics and this sits in IoT Edge/Gateway Device.

    In this section, we are going to showcase the steps involved in installing and configuring IBM EAA component with the DGLux tool.

    • Download the IBM EAA DS Link from Watson IoT Platform.

    download_eaa

    • Extract DS Link from the downloaded archive and place it into the path DSA_PATH/dglux-server/dslinks.
    • Stop DSA by running command from terminal – DSA_PATH/dglux-server/bin/daemon.sh stop
    • Start DSA by running command from terminal – DSA_PATH/dglux-server/bin/daemon.sh start
    • Now, if we click on links component, we should be able to see new link installed with the name ibm-watson-iot-edge-analytics-dslink:

    • In above pasted snippet, we should be able to see state as connected under the Metrics window.
    • With this, we’ll be able to see a node added under downstream component for the IBM EAA link.
    • Right click IBM EAA Node present under downsream and select Configure.
    • Input Watson IoT credentials and click Invoke button:

    • After clicking on Invoke button in the previous step, now we go to Watson IoT Platform page and Refresh on Browse Devices pages, we should see that the defined Gateway Device connected to platform and this shows that the credentials that we provided are correct and DSA platform is setup properly on the system:

     In this section, we showcased the steps for downloading the IBM EAA from Watson IoT Platform. Then installing and configuring IBM EAA using DGLux tool.

  5. Setup System DSLink

    The DGLux tool comes with the implementation of “System” DSLink which can be installed and configured to get the system properties. So, this section mainly describes the steps involved in installing the default “System” DSLink into DGLux tool and then configure to add device and devicetype to Watson IoT Platform setup in the above steps.

    • As mentioned in Setup IoT-DSA Platform step, under download and installing DSA, we selected System DS Link along with DGLux tool to get downloaded.
    • Right click on Links -> Install Link -> from Repository and select System from drop down list, give the name as System then click on Invoke button. Wait till System DS Link gets installed and we’ll be able to see success message and the state as true with green color.

    install_system_link

    install_system_link_success

    • Right click on System Link and select Start Link option.

    • We should be able to see state as connected under Metrics window and a new node gets added to downstream component starting with the name – System.

    • Select Menu in Metrics window to enable Show Attributes to add attributes to System Node.
    • By selecting the System Node from downstream, we should be able to see the system properties under Metrics window.¬†
    • Next we need to add 2 attributes to System Node under downstream component.

    • Right click System and select @addAttribute, add type:device and devicetype:edgeDevices.

    • By selecting the System Node from downstream, we should be able to see the system properties under Metrics window along with the added 2 attributes which internally adds the Device and Device Type to Watson IoT Platform.

    • To see the automatic addition of the device and devicetype, click on Refresh button on Watson IoT Platform page
    • If we don’t see the Device and Device Type, Stop and Start the IBM EAA and System links from DGLux tool and then refresh Watson IoT Platform devices page.

     

    • Now, if we click on System Device from the Devices Page, we should be able to see all the System Properties have been sent to Watson IoT Platform wihtout any filtering:

     The steps in this section demonstrated how to install System DSLink and configure it with Watson IoT Platform credentials to automatically add the required device into the Platform.And we are also able to see that all System Properties have been sent to platform without any filtering action.

  6. Setup Rules

    This section covers important information needed for implementing Analytics on an Edge device.

    In this section, we describe the steps involved in defining the Schema, Edge Rule, An Action followed by pushing the defined rule to a Gateway by Activating the rule on Watson IoT Platform. We are going to describe the steps needed to filter out the device data with CPU usage >= 20% to be sent to Cloud Platform using Edge Analytics with the help of IBM EAA.

    • By default, all collected device data is sent to the Watson IoT Platform after successful connection to the platform. To view this data, click on the device from the Watson IoT Devices page:

     Define Schema

    • First step is to define the schema for our Edge Rule. Select Manage Schemas from the platform page and click on Add Schema:

    • Select the device type for those devices that are connected to our gateway and click on Next to proceed:

    • Clicking on Next, we’ll be presented with properties window.
    • Clicking on Add Property option, appears window with sources to select properties for the schema.¬†

    • Select “From Connected” device option to select required properties from the connected device. Select any properties that we intend to build an Analytic Rule (Cloud or Edge). For this recipe, we are selecting CPU usage to define the rule and then click on Finish:

    • Click on Finish to complete creation of Schema step.

    • Now, we’ll be able to see the defined schema in the Manage Schemas window as shown below:

     

    Create Edge Rule

    • Next step is to define condition as part of an Edge Rule. To do so, from the left menu panel, select Rules option and we’ll be presented with Browse Rules window in which click Create Edge Rule:

    • After clicking on Create Edge Rule from Browse Rules window, we’ll be presented with Add New Edge Rule dialog.
    • Fill in the Rule Name, Description and select an already defined/added Device Type and then click Next to proceed:

    • A dialog window appears with the added new edge rule along with options to add conditions and actions as shown in below pasted snippet:

    • Click on New condition to set new condition, a dialog window appears.
    • Select the property as CPU_Usage, defined earlier, and set condition as >= 20 and then click OK,

    • With this, we are done defining condition.

    Define An Action:

    • Next step is to define an action as part of Edge Rule. i.e what needs to be done when device’s CPU_Usage crosses 20%, so we are going to define the action to forward only such events to Cloud.
    • A primary benefit of implementing an Edge Rule with this Action would be to reduce the volume of data that is pushed to the Cloud.
    • To define, click on New Action, a dialog window appears, click on Add Action:

    • After clicking Add Action, Create Action Dialog appears, in which provide action name, description and select action type as “Forward Event to Cloud” and then click Next:

    • Next step is to configure the defined action.
    • Clicking Next, retain the default selected option and provide a name to event and click to Finish to complete the Action Creation step:

    • After successful completion of Create Action step, next step is to Set the Action.
    • In the next appeared dialog, select the created action and click OK:

    • With these steps completed, we can see the defined Condition and the Action as part of created Edge Rule.
    • As part of next step, click on Save to save the created edge rule:

     

    Activate the Edge Rule on the Gateway

    • After saving the Edge Rule, a dialog window appears to activate the Edge Rule in the Gateway.
    • Select the rule and the Activate operation, then click on Submit Command.¬†

    add_edge_rule

    • If implementation has multiple gateways, more than 1 gateway would be listed.
    • After command submission, we’ll get a notification about the result of command submission and click Close:

    add_edge_rule

    • Click on Browse Rules to check the latest status of the Edge Rule. We’ll be able to see that the submitted Edge Rule is Active as shown in below pasted snippet.

    Validate the Edge Analytics Result

    • To get confirm only the device data where CPU Usages >= 20 are being forwarded to the cloud, go to Browse Devices and click on System device. It should show only the device data where the CPU Usage >= 20 are being forwarded to the Cloud for further analytics.

     

    This concludes the steps required to perform Edge Analytics using IBM EAA. In this section, we described about creating the schema, defining the Edge Rule with Condition and Action to be taken when the condition is met. Also, we showcased how to activate the defined Rule in the Gateway to carry out the Edge Analytics and validate the activated Edge Rule correctly forwards the required events to cloud for further analysis.

  7. Update Rules

    The main focus for this section is to refine the condition(s) and action(s) associated with already defined Edge Rule based on requirement followed by updating the Edge Rule to take effect to perform the set action(s) based on the refined conditions. As an sample example, we are going to show the steps how to refine the above defined Edge Rule with condition set to CPU Usage between 15 to 25.

    • To update the condition associated with the existing Edge Rule, double click on the existing active Edge Rule:

    • Next we’ll able to see a dialog to update the condition and action for the Edge Rule:

    • Click on existing condition and refine the condition on the appeared dialog as CPU_usage >= 15 and then click on + to add new condition as CPU_usage <=25

    • We are going to retain the action part as same as existing action to forward the triggered every event to Cloud for further analysis.
    • Click on Save on the next Dialog Window to save the updated condition and action associated with the Edge Rule:

    • As soon as we click on Save to make the changes done to Edge Rule get saved, a dialog window appears with the existing state of the activated Edge Rule which clealry indicates that the Older Edge Rule is active.
    • Applicable actions on the Edge Rule are provided as drop down box, from them select Update, also select the Edge Rule to be updated and click Submit Command:

    update_edge_rule

    • After few seconds clicking Submit Command, we’ll see a notification window saying Update Submission Successful, click on Close to close the notification window:

    update_edge_rule_notification

    • Refresh Active Rules page by clicking on Browse followed by clicking on View button to see the latest status of the Edge Rule, we should be able to see that the Edge Rule has been updated and active state:

    • With this updated Edge Rule, if we go to Devices page and select the System device to find out what all triggered events from the Edge are getting forwarded to Cloud, we should see the events with CPU_usage in between 15 to 25 getting forwarded to Cloud.

     

    Similarly, we can also carry out deactivating any active Edge Rule as well as completely removing the deactivated Edge Rule by selecting Deactivate Action and Remove Action from the drop down list available along with the Update Action for the Edge Rule.

    With all these described steps, this seciton concludes on how we can Update the existing Edge Rules and activate them on the Edge to trigger the device data which meet the re-defined conditions present in the newly updated Edge Rule.

     

  8. Conclusion

    With the steps described in this recipe, we have showcased how to perform Edge Analytics for the device data with CPU usage more than 20%. To achieve this Edge Analytics, we have used :

    • IBM Edge Analytics Agent in the form DS Link which internally uses Apache Edgent to carry out the required analytics.
    • DSA bundle which comes with DGLux tool to Install and Configure required DS Links.
    • Installation of System DSLink and configuring the same to send events to Cloud.
    • Creation of Edge Rule to perform analytics on the device data at the Gateway Level and forward events to Cloud.
    • Update/Deactivate/Remove the Edge Rule.
  9. Next Steps

    We have next level of edge analytics use case where we show case the handling of alerts and device actions based on Edge Rule activated in the Edge Gateway. Refer to the recipe – Handling Alerts and Device Actions with Edge Analytics for detailed steps.

9 comments on"Getting started with Edge Analytics in Watson IoT Platform"

  1. I-Logix_Disciple September 30, 2016

    Nice recipe. Word of caution to others who try to do this recipe. I was not able to get DSA to work on a Raspberry Pi 2. I had to switch to a Pi 3 and then things worked ok.

  2. GonzaloValle November 29, 2016

    Step 4 fails if the Raspberry Pi hasn’t got English as the configured language.
    In my case I had Language: Espa√Īol (Spanish) and Country: Espa√Īa

    If you are at Raspberry Pi GUI desktop, do the following:
    – Go to: Menu > Preferences > Raspberry Pi Configuration
    – Select Localisation tab
    – Press Set Locale button
    – Check that your locale has the following values:

  3. I have successfully followed all the steps in the recipe with Raspberry Pi 3 model B v1.2. Had initial problems with the installing DSA + DGLogik link in step 3 initially due to link down for a few days sometime in Dec 2016.
    Good tutorial, I have better understanding on DSA with DSLink and DSBroker now…

  4. Hello, I’m trying to set up this recepi with my Raspberry. Installing dglux I see in sys-> links the new edge IOT but the state of the connection is not equal to connected. The state of the connection is continuosly changing from started to stopped.
    I have inserted the dglux license as explained. Any idea of what I’m not making correctly?

  5. AashishChaubey February 06, 2018

    Hey, I’m following this recipe and in step 5, towards the end, I should see the device ‘System’ of type ‘edgeDevices’ being connected to my device list in the IBM Watson IoT Platform, but somehow I am not able to see that. I’ve correctly followed the steps until now in the recipe. Please suggest a fix for the same!

    • AashishChaubey February 06, 2018

      I figured out the fix to the problem!

      So, for the people using Bluemix, it seems the authmethod should be set to token instead of use-token-auth while configuring the ibm-iot-edge-analytics.

Join The Discussion