Contents


Overview

Skill Level: Any Skill Level

Developers interested in analytics at the edge.

This recipe will show you how to connect a range sensor up to your Raspberry Pi, launch an Edgent sample that reads sensor data and runs analytics locally, then sends filtered data to Watson IoT Platform where we can visualize it and integrate it with the

Ingredients

  • Raspberry Pi
  • HC-SR04 Range Sensor (you can also use a simulated sensor)

Step-by-step

  1. Introduction

    edgent_logo_large_transparent

    [Editor’s Note: Apache Edgent has been renamed to “Apache Edgent.” The post below was written before the name change. All references to Edgent in this post now apply to the Apache Edgent project.]

    Apache Edgent (incubating*) is a programming model and micro-kernel style runtime that can be embedded in gateways and small-footprint edge devices. It makes it easy to process sensor streams on edge devices like a Raspberry Pi, run local analytics, then send only the useful information to cloud services like Watson IoT Platform.


    This recipe will show you how to connect a range sensor up to your Raspberry Pi, launch an Edgent sample that reads sensor data and runs analytics locally, then sends filtered data to Watson IoT Platform where we can visualize it and integrate it with the rest of the world.

    Once you're done with this recipe, you can move on to Part II where you will connect to the Streaming Analytics service on Bluemix. Here is a quick demo of what you will have after completing these two recipes:

    For an in-depth introduction to Edgent on Raspberry Pi, you can also check out this video from Queenie Ma:

  2. Hook up Range Sensor Hardware

    First, you will need to hook up your HC-SR04 range sensor to your Raspberry Pi. Here is a good tutorial on how to do it (skip the portion with Python).

    For the code we show, make sure that you set up your Trig pin on pin 16 and your Echo pin on pin 18.

    Note: If you don't have an HCSR04, you can still use this recipe by using a simulated sensor or another sensor of your choice.

  3. Install Pi4J on your Raspberry Pi

    From a terminal, run this command:

     $ curl -s get.pi4j.com | sudo bash   


    Next export the location of the Pi4J libraries (the default location on a Pi is: /opt/pi4j/lib/):

    $ export PI4J_LIB=/opt/pi4j/lib/

    Note: Pin numbering in the Pi4J is non-standard. See numbering diagrams here if you are having problems.

  4. Download and Build Apache Edgent (incubating)

    You can download a prebuilt version of Edgent from March 7 here. For the prebuilt verson, export the location of your java8 target (Note: packages from the prebuilt version will not match the latest code due to the name change):

     $ export EDGENT=/homes/myUser/quarks/java8/

    If you would like to download and build the latest code, follow these steps:

    1. Download the source here: https://github.com/apache/incubator-edgent/archive/master.zip
    2. Check the setup section of this link to see if you have the required software for building:  https://github.com/apache/incubator-edgent/blob/master/DEVELOPMENT.md. If you do not, install them. When your system meets the requirements, you can unzip the source and use the ant command to build:
      $ unzip master.zip            
      $ cd incubator-edgent-master/
      $ ant
    3. Export the location of your Edgent Java 8 target build (we will use this later to compile):
      $ export EDGENT=/homes/myUser/incubator-edgent-master/target/java8

       

    Optional: Set up a development environment outside of Rasperry Pi

    If you would like to set up an Edgent development environment on a development machine, there are instructions for setting Edgent up in Eclipse in the Getting Started Guide.

    For the purposes of this tutorial, we recommend adding these JARs to your environment as well:

    <edgent-java8-target>/samples/lib/edgent.samples.apps.jar
    <edgent-java8-target>/samples/lib/edgent.samples.connectors.jar
    <pi4j-lib>/*

     

     

  5. Getting Data from a Sensor on Your Pi

    Now the real fun starts. As mentioned in the introduction, Apache Edgent is a micro-kernel style runtime that enables stream processing at the edge. A typical Edgent programming model involves reading data from one or more sensors, processing and transforming that data, then only sending necessary data over the network to a service like Watson IoT Platform. This section will focus on getting the sensor data.

    1. Create a sensor class to encapsulate the the logic of reading from the range sensor. The RangeSensor class implements the Supplier<Double> interface. It will provide the sensor reading value as a double in the get() method when it is called.
      import java.util.function.Supplier;

      public class RangeSensor implements Supplier<Double> {


      @Override
      public Double get() {
      // TODO Auto-generated method stub
      return null;
      }

      }
    2. Create a Edgent topology that will poll data from the sensor every second. For more details on creating topologies and what they are, read here.
      DirectProvider dp = new DirectProvider();

      Topology topology = dp.newTopology();

      RangeSensor sensor = new RangeSensor();

      //poll Range sensor every second for distance reading
      TStream<Double> distanceReadings = device.topology().poll(sensor, 1, TimeUnit.SECONDS);
    3. Now we move to the Pi4J sensor code. The range sensor we use sends a sonic signal and times the echo to measure distance. We do this by sending a signal to the trig pin, then timing how long it takes to detect the reflected sonic wave on the echo pin (this is sonar). The time to signal detection divided by the speed of sound, gives the distance the signal traveled. Below are some of the basic steps taken using Pi4J in the range sensor which would apply to many sensors.
      Create a GPIO Controller and assign pin numbers (Check out the non-standard pin diagrams here):
      public class RangeSensor implements Supplier<Double> {

      private GpioController gpio = GpioFactory.getInstance();

      private Pin echoRPin = RaspiPin.GPIO_05; // PI4J custom numbering (pin 18 on RPi2)
      private Pin trigRPin = RaspiPin.GPIO_04; // PI4J custom numbering (pin 16 on RPi2)
    4. In the constructor of the RangeSensor class, provision input and output pins:
      public RangeSensor(){


      GpioPinDigitalInput echoPin = gpio.provisionDigitalInputPin( echoRPin );
      GpioPinDigitalOutput trigPin = gpio.provisionDigitalOutputPin( trigRPin );

      }
    5. Poll your sensor for data.
      1. Send a signal from your trig pin. This sets your echo pin to a high state and sends a sonic signal:
        trigPin.high();
        ...
        trigPin.low();
      2. Measure how long the echo pin stays on high to indicate how long it takes to receive the reflected sound signal:
        long start = System.nanoTime();

        while( this.echoPin.isHigh() && cycles < MAX_CYCLES){
        //only iterate through MAX_CYCLES times before giving up
        cycles++;
        }
        long end = System.nanoTime();
      3. Convert your time to a distance:
        long microSeconds = (long)Math.ceil( ( end - start ) / 1000.0 );

        Double distance = reboundTimeMicroSeconds * SPEED_OF_SOUND / (2 * 10000); //gives distance in cm

         

    6. The pins are now ready to be used. As discussed earlier, the sensor will be polled once every second. The get method of the RangeSensor class is responsible for reading the sensor values and providing them to the Edgent topology. In the get method:
      @Override
      public Double get() {
      return getDistance();
      }
  6. Sending Data to Watson IoT Platform from Edgent

    1. Make sure you have registered your device with Watson IoT. If you have not, these instructions make it easy. You will need the following information to create a device.cfg file:
      org = <orgname>
      type = <devicetype>
      id = <deviceid>
      auth-method = token
      auth-token = <authtoken>
    2. Declare an IotpDevice and pass in your config file (if we don't specify a config file parameter, the default location to look is the running directory):
      IotDevice device = new IotpDevice(topology, new File(deviceCfg)); 
    3. Create the sensor data stream, then run some analytics on the stream. As shown above, our data stream comes from polling our range sensor every second (this calles the overridden get method). We want to filter out bad readings, convert the format to JSON, do some aggregation, then only send readings to IoTF when the mean distance is closer than 30 cm.

       

      //poll Range sensor every second for distance reading
      TStream<Double> distanceReadings = device.topology().poll(sensor, 1, TimeUnit.SECONDS);

      //filter out bad readings that are out of the sensor's 4m range
      distanceReadings = distanceReadings.filter(j -> j < 400.0);

      TStream<JsonObject> sensorJSON = distanceReadings.map(v -> {
      JsonObject j = new JsonObject();
      j.addProperty("name", "rangeSensor");
      j.addProperty("reading", v);
      return j;
      });

      // Create a window on the stream of the last 10 readings partitioned
      // by sensor name. In this case we only have one range sensor so there will be one partition.
      TWindow<JsonObject,JsonElement> sensorWindow = sensorJSON.last(10, j -> j.get("name"));

      // Aggregate the windows calculating the min, max, mean and standard deviation
      // across each window independently.
      sensorJSON = JsonAnalytics.aggregate(sensorWindow, "name", "reading", MIN, MAX, MEAN, STDDEV);

      // Filter so that only when the mean sensor reading is that an object is closer than 30cm send data.
      sensorJSON = sensorJSON.filter(j -> Math.abs(j.get("reading").getAsJsonObject().get("MEAN").getAsDouble()) < 30.0);
    4. Send your data stream to Watson IoT. In this example, the data stream is sensorJSON and we're using “sensors” as the event identifier:

      // Send the device streams as IoTF device events
      // with event identifier "sensors".
      device.events(sensorJSON, "sensors", QoS.FIRE_AND_FORGET);

    That's all it takes with Edgent! Two lines of code related to Watson IoT, a data stream, and a config file. To learn more about data streams and Edgent basics, the Getting Started page is a great place to begin.

  7. View Data on Watson IoT Platform

    1. Login to Bluemix and from the Dashboard find your Internet of Things Platform service.

    2. Click on Launch dashboard.
    3. Click on the “devices” tab:
    4. You should see your device. Here is what mine looks like:
    5. If you click on your device, you can scroll down to the Sensor Information section and view your sensor readings as they come in.

     

    Now that you can view your data, you're ready to connect your edge device to the rest of the world!

  8. Building and Running Your Edgent Application

    1. To compile your application, you will need to include the Pi4J JARs as well as Apache Edgent JARs. The simplest way to do this is to use the samples libraries in Edgent (the following command assumes you exported $EDGENT and $PI4J_LIB as instructed above):
      $ javac cp $EDGENT/samples/lib/'*':$PI4J_LIB/'*' d bin/ src/*.java 
    2. To run your sample, you will need to use sudo so that the Pi4J libraries can control your Pi. You will also need to provide the location of the device.cfg if you don't have it hardcoded:
      $ sudo java cp $EDGENT/samples/lib/'*':$PI4J_LIB/'*':bin/ IotfRangeSensor device.cfg 

       

    Tip: If you are using Eclipse to do your development outside of the Pi, you can export a JAR and save yourself having to build on your Pi by following these steps:

    1. File -> Export…
    2. Expand Java and select JAR file. Click Next.
    3. Fill in the name you want for your application JAR. Click Finish.
    4. Now you can scp or transfer your JAR file to your Raspberry Pi and run without building. Here is a sample run command:
      $ sudo java cp $EDGENT/samples/lib/'*':$PI4J_LIB/'*':IoTFRangeSensor.jar IotfRangeSensor device.cfg  

       

  9. Next Steps

    You now have a range sensor whose readings are intelligently handled by Edgent and sent to Watson IoT Platform where you can view the data and start integrating it with other applciations on Bluemix.

    Happy Streaming!

     

    *Apache Edgent is currently undergoing Incubation at the Apache Software Foundation.

3 Comments on "Apache Edgent on Pi to Watson IoT Platform"

  1. KiranShashi May 29, 2016

    Interesting application, but is there a Python version of it ?
    Shashi

  2. Stan Bradbury March 14, 2017

    Just a nit. The word Quarks needs to be selectively replaced in two places in NOTE at the beginning. It currently incorrectly states:

    [Editor’s Note: Apache Edgent has been renamed to “Apache Edgent.” The post below was written before the name change. All references to Edgent in this post now apply to the Apache Edgent project.]

    • I’ve tried editing in 3 different browsers over 3 days, with no luck at being able to save my changes. Here is how it should read:

      Note: Apache Quarks has been renamed to “Apache Edgent.” The post below was written before the name change. All references to Quarks in this post now apply to the Apache Edgent project.

      Thanks for heads up!

Join The Discussion