Taxonomy Icon

IoT

In the first article in this series, “Integrating LPWAN networking into your IoT solutions,” you learned how to monitor hay barns for humidity and temperature to identify dangerous conditions. However, identification is only a partial solution. Ideally, you would want to fix those conditions automatically. Because Internet access can be slow, spotty, or even non-existent, the analytics to decide what to do should happen on the “edge” of the network, right at the barn.

To do this edge analytics, you can use two appliances, both powered by a smart plug:

  • A fan (for use when the temperature is high, but not too high because that means there are flames to fan)
  • A heater (for use when the humidity is too high, and the temperature is low)

A Raspberry Pi controls the smart plug using Bluetooth.

What you’ll need to build your application

Because you are adding on to the IoT system that you built in the first tutorial in this series, you need to have completed the first tutorial before working on this one. To build this edge analytics system for your IoT monitoring system, you need:

  • A Raspberry Pi with wifi and Bluetooth support. I used a Raspberry Pi 3 B+. If you use an earlier model, you might need a wifi and/or Bluetooth USB dongle.
  • A microSD card (8GB is more than enough).
  • A Switchmate smart power outlet.
  • A heater and a fan (you can verify that the system works without having them).
  • Basic knowledge of JavaScript.

Why edge analytics?

If possible, it is best to perform analytics in the cloud. The cloud has strong computers, elastic processing capacity, and access to all the information available on the Internet. However, it is not always possible to do all your analytics in the cloud. Internet access might be unavailable, unreliable, or have insufficient bandwidth available. Also, if you need the results in real time, a “best effort” network might not be the correct solution.

Edge analytics, which is analytics on the IoT device (or a gateway between the device and the Internet), can accomplish two goals. If Internet access is intermittent or slow, you can use it to send just the information that is needed on the cloud instead of raw data. For example, a security camera’s video feed is repetitive. There is no need to send 10 minutes of video when “this was the picture for the first 5 minutes and then the last 4 minutes, and here is the video of the 1 minute when somebody walked through the hallway” would do. The other goal of edge analytics is to encourage local decision making. In the case of the hay barn monitoring system, it is providing additional safety and should be running even in the absence of Internet access. It is best to rely on as few components as possible for higher reliability.

Edge analytics does not have to be complicated or require a lot of CPU power. In this case, it is enough to look at temperature and humidity to identify if it makes sense to activate any of the safety appliances (a fan to reduce the temperature or a heater to dry the hay).

Architecture

Figure 1 shows the architecture of the edge analytics system added to the existing IoT monitoring system. The sensors are made out of a NodeMCU and a DHT11, as explained in Step 1a of the first tutorial. The Raspberry Pi is a wifi access point that provides the sensors with their IP addresses and receives the data that they report using HTTP, as explained in Step 2c of the first tutorial. This Raspberry Pi — the one we are using for edge analytics — uses Bluetooth (which is not supported by the ESP8266 NodeMCU) to control the smart plugs. Those smart plugs power the heater and the fan.

Figure 1. The architecture of the edge analytics system as part of the IoT monitoring system

The source code

The NodeMCU Lua sensor program is the same one used at the end of the first article.

For the Edge Analytics Raspberry Pi, I follow an iterative development model. Three stages are available on GitHub:

  1. The 11_app.js program demonstrates how to use Bluetooth to control a Switchmate smart plug pair, which in turn controls the fan and heater.
  2. The 12_app.js program adds the web server that receives the sensor readings from the NodeMCU sensors.
  3. The 13_app.js program adds the actual edge analytics, and it represents the complete application.
1

Set up the Raspberry Pi

To start, connect the Raspberry Pi to your own wifi network to set it up.

1a

Set up the Raspberry Pi and its operating system

To set up the Raspberry Pi as a control station, complete these steps:

  1. Download and install NOOBS (the lite version is sufficient) on the microSD card.
  2. Insert the microSD card into the Raspberry Pi. Connect the HDMI, keyboard, mouse, and power (the micro USB connection).
  3. Configure the wifi network to connect to the Internet.
  4. Select Raspbian Lite and click Install.
  5. After the installation, reboot the Raspberry Pi.
  6. When you see the login prompt, log in as pi with the password raspberry. Note that there is no need to change the wifi settings — the wifi values you gave NOOBS are preserved.
  7. Change the default password to passwd pi.
  8. Enable sshd, as described in the Raspberry Pi documentation.
  9. Disconnect the Raspberry Pi, and reconnect it close to the smart plug.
  10. Use an SSH client (my favorite SSH client on Windows® is Bitvise) to connect to the Raspberry Pi. You might need to connect to your router to identify the IP address.
1b

Download and install Raspberry Pi software

With the hardware and the operating system running, the next step is to download the software that you need.

  1. Install Node.js. With this environment, you can program in JavaScript, a language that is similar to Lua. The npm application is the Node.js package manager.
    
    sudo apt‑get update
    sudo apt‑get install nodejs npm
    
  2. Install Noble, a Node.js Bluetooth library. Ignore the warning messages.
    
    sudo apt‑get install bluetooth bluez libbluetooth‑dev libudev‑dev
    sudo ln ‑s which nodejs /usr/bin/node
    npm install noble
    
    
    
  3. Install Express, the Node.js HTTP server library.
    
    npm install express
    
2

Configure the smart plug

You can control most smart plugs using standard Internet protocols over wifi. However, those designs are based on the assumption that the smart plug is on the Internet and that it is acceptable for the control channel to rely on the smart plug vendor’s servers. With hay barns, Internet access can be slow, spotty, or nonexistent. I chose to use the Switchmate smart plugs for this project because their plugs support Bluetooth in addition to wifi. Bluetooth is used for short distance communication, which is what we need.

2a

Pair your device with the smart plug

  1. Install the Switchmate application on your smart phone or tablet (they have versions for Android and iOS).
  2. Connect the Switchmate plug to power.
  3. Start the Switchmate software.
  4. Click Add Device > Power and follow the directions to pair your device with the smart plug.
  5. Turn the plugs on and off and verify that you hear the clicks of the relays inside the smart plug.
  6. Click the gear icon, and then one of the smart plug icons with a gear, to get to the device properties. Turn off LED backlight (which wastes electricity).
  7. Click Device Info. Note that the Bluetooth MAC address for the plug (mine is C9:87:08:B2:08:5D).
  8. Click Save.
2b

Get the smart plug configuration

Note: If you are using the exact same smart plug model that I am and you do not care how I got the Bluetooth parameters, you can skip this section.

To get the parameters for the smart plug, install a Bluetooth scanner. I use BLE Scanner (which is available for both Android and iOS).

To get the smart plug’s information, start a scan and connect to the MAC address that you found when you set up the smart plug.

You configure the Bluetooth device through characteristics, which are grouped into services. Services and characteristics, as well as descriptors, are collectively called attributes. The list of Bluetooth generic attributes (GATT) is available here. When I wrote this tutorial, there was no generic attribute for a smart plug, so it has to be in a custom service (one with a much longer identifier). The Switchmate smart plug has two custom services and many read/write attributes. We can rule out the attributes that take more than a single byte, and by testing the remaining ones, we can discover the two relevant attributes:

Service Attribute Meaning
0xA22BD383-… 0xA22B0090-… Top plug
0xA22BD383-… 0xA22B0095-… Bottom plug

The ellipses (…) for the service and the two attributes are EBDD-49AC-B2E7-40EB55F5D0AB. The same suffix appears for all the other attributes of the service.

3

Write the JavaScript program to control the smart plugs

To control the smart plugs, download this program. Use these steps:

  1. Download the program (the command is all one line).
    
    wget https://raw.githubusercontent.com/qbzzt/IoT/master/201801/Hay_Bale_Fire/11_app.js
    
  2. Run the program. Note that to control Bluetooth, this program needs to be running as root.
    sudo nodejs 11_app.js
    The top plug turns on or off each second. The bottom one does it every 0.99 seconds. Every time a plug’s status changes, you hear a ticking sound from the relay. At first, the two sounds happen at almost the same time, but as time passes, the time difference increases.

Let’s look at the source code. The first line loads the Noble library. As is usual in Node.js programming, you access the library through an object; in this case, the variable noble.

var noble = require("noble");

These lines specify the Bluetooth attributes, both service and characteristics.


var devTypeID = "ebdd49acb2e740eb55f5d0ab";

var plugService = "a22bd383" + devTypeID;


The characteristics are stored in an associative array.


var plugChars = {
  "top": "a22b0090" + devTypeID,
  "bottom": "a22b0095" + devTypeID
};

The Bluetooth scanning commands expect to get attributes as arrays (regular integer indexed ones, not associative arrays). This line creates a one-item array for the service.


var plugServices = [plugService];

Getting the characteristics from an associative array into a normal array is a bit more complicated. First, we use Object.keys to get an array for the keys of associative array. With the previous definition, this provides us with the array [“top“, “bottom“].

Next, we use the map function. The parameter to this function is itself a function, one that gets applied to every item in the array sequentially. In this case, if we call the function f(), the result is [f(“top“), f(“bottom“)]. The function is k => plugChars[k]. This function looks up the key in the associative array and returns the corresponding value, the universally unique identifier (UUID) of the characteristic. This gives us an array with all the characteristics.


var plugCharacteristics = 
  Object.keys(plugChars).map(k => plugChars[k]);

The plugAPI variable contains the objects for the characteristics that we can read and write. It starts as an empty associative array.

var plugAPI = {};

This function reads the status of a plug and runs a callback function with it. It receives two inputs, the discovered plug characteristic and a callback function. After reading the plug characteristic’s current value, it runs the callback with a Boolean value (true if the plug is turned on, false if it isn’t).


var readPlug = (plug, cb) => {
    plug.read((err, data) => {
        cb(data[0] !== 0);
    });
};


This function writes a plug’s status and then calls the callback. The first value for writing the characteristic is a Buffer to write. The Buffer.from function creates a buffer from an array of byte values. For the plug characteristics, the value is 1 byte long. Zero means turn it off, and any other value means to turn it on. The value is calculated using the ternary operator. If val is true, it is 1, otherwise 0.


var writePlug = (plug, val, cb) => {
    plug.write(Buffer.from([val ? 1 : 0]), true, (err) => {
        cb();
    });
};


This function toggles a plug’s value (on-to-off or off-to-on). Notice that the writePlug call is in the callback of the readPlug function. The callback for writePlug does nothing.


var togglePlug = (plug) => {
    readPlug(plug, 
      currentVal => writePlug(plug, !currentVal, () => {}));
};


This function is called when the Bluetooth scan discovers a device. The parameter (here called plugDevice) is an object with functions and values for the device. We know that the device is a plug because we are only looking for devices with that service.

var plugDiscovered = plugDevice => {

First, stop scanning.

noble.stopScanning();

Register a handler when the plug device is connected. We use .once so that the handler only runs a single time. The console.log call is write to standard output for debugging purposes.


plugDevice.once("connect", () => {
        console.log("Connected to the plug through Bluetooth");

Search for the services and characteristics that we need. We could attempt to discover all of them, but this takes less resources.


plugDevice.discoverSomeServicesAndCharacteristics(
  plugServices, 
  plugCharacteristics,
  (err, services, charObjs) => {

These lines find the proper characteristic object for each plug named in plugChars and write it to plugAPI. First, Object.keys(plugChars) creates an array of plug names. Then, .map runs a function on each one.


Object.keys(plugChars).map((plugName) => {

This function uses the filter function to find in the charObjs array the entry whose UUID is equal to the one for the plug characteristic. The result of filter is an array, so we take the first entry, indexed 0, and assign it to plugAPI with the correct plug name.


plugAPI[plugName] = 
charObjs.filter(c => c.uuid === plugChars[plugName])[0];
                });
            
                console.log("APIs: " + Object.keys(plugAPI));


Because top and bottom are legitimate identifiers, we can use either plugAPI.top and plugAPI.bottom or plugAPI["top"] and plugAPI["bottom"]. It would be the same object.


setInterval(() => togglePlug(plugAPI.top), 1000);
                setInterval(() => togglePlug(plugAPI.bottom), 990);            
        });  // plugDevice.discoverSoServicesAndCharacteristics

    });    // plugDevice.once("connect")

    plugDevice.connect();
};


Start the scan. To save resources, scan only for those devices that advertise the Switchmade smart plug service. It would make more sense to do this after registering the event handlers, but Noble does not support registering them until the Bluetooth is turned on.


noble.startScanning(plugServices);

Report when the state changes (either because the scan started or because a different process is using Bluetooth).


noble.on("stateChange", 
  state => console.log("Bluetooth state is now " + state));


When a plug is discovered, call the plugDiscovered function.

noble.on("discover", plugDiscovered);

4

Write the JavaScript program to receive the sensor readings

The next step is to receive sensor readings using HTTP. The format for these readings is explained in the first article, section 2.c. The NodeMCU sensor attempts to get a web page from URL http://///<temperature (centigrade)=””>/ to report that information. Note that the second double slash after the IP address is not a mistake here. It was a mistake in the first article, but I would rather keep it than change the NodeMCU program on the sensors at this point.

The program for the Raspberry Pi to receive sensor readings is here. The URL for the raw program (to get it into the Raspberry Pi) is here. To verify that it works, you can go to the Raspberry Pi’s IP address from a browser (for example, the URL http:////cpuid/15/99). Alternatively, you can modify the NodeMCU Lua program to connect to your wifi network (lines 13 – 15, see the list of parameters in the NodeMCU documentation) and the IP of the Raspberry Pi (line 19).

Here are the new parts:

The web server package is Express. You run the function returned by the module to create a new HTTP server.


var express = require('express');
var app = express();

The app.get call specifies how to handle requests for a particular path. The path is the first parameter. The second parameter is a function to call when the path is requested. This function has two parameters: a request object that includes the HTTP request and a response object used to send back a response. The callback function uses the res.send function to send the response. The exact response does not matter here; the NodeMCU ignores it.

When a path includes words that start with a colon, such as :cpu, it means that that particular path component can be anything (as long as it is composed of valid characters for a URL path component). It is a parameter for the function that generates the reply, and Express encodes it in req.param.<parameter name>. For example, if you browse to http:///myCpu/25/34, the req object sent to the callback will include these values:

req.param.cpu myCpu
req.param.temp 25
req.param.humidity 34


app.get("/:cpu/:temp/:humidity", (req, res) => {

  res.send("Hello, world");  // Just to respond with something

Template literals are strings in JavaScript that are enclosed by back ticks (). When a template literal includes ${_<expression`>_}, that part is replaced by the value of the expression. This way, it is simple to embed values, such as the NodeMCU’s CPU serial number or the temperature. A template literal can include newline characters, so to split the line (for readability) without affecting the response, we put the newlines inside the expressions, where JavaScript treats them as any other white space character.


console.log(${
  req.params.cpu} reports ${
  req.params.temp} C and ${req.params.humidity}%);
});

The app.listen call starts the actual HTTP server. In this case, it listens on port 80, on all the IP addresses that the Raspberry Pi has. When the server starts, it writes that fact.


// Start the web server. Listen to all IP addresses
app.listen(80, '0.0.0.0', () => {
  console.log("Web server started");
});

5

Write the JavaScript program for edge analytics

Looking at the article about hay bale fires, these are the states and the correct actions for them:

Humidity Temperature (b0F) Temperature (b0C) Heater Fan Explanation
(less-than 20% (less-than 125 (less-than 51 Off Off Everything is fine.
> 20% (less-than 125 (less-than 51 On Off Dry excess humidity.
Any 125 – 175 51 – 80 Off On Increase air circulation to reduce temperature.
Any > 175 > 80 Off Off Fire or hot spots likely, avoid fanning the flames. Call the fire department.

Before we write the code, we also need to decide which reading to use because we could have multiple sensors in the same barn. It makes the most sense to go with the highest recent reading. If any sensors report a temperature above 175 b0F, there is probably a hot spot nearby and it would be a really bad idea to turn on the fan and encourage air circulation. If any sensors report a temperature above 125 b0F (but below 175 b0F), that’s enough to pay the small cost of running the fan to avoid the much higher cost of a fire. Finally, if no sensor reports a temperature above 125 b0F but any of them reports a high humidity, it is best to turn on the heater and hopefully dehydrate the bacteria before they can raise the temperature to dangerous levels.

To do this, we preserve the last 10 minutes’ worth of readings and get the top temperature and humidity readings.

The full program is here. You can download just the program to your Raspberry Pi. Run the program, and then browse to the IP of the Raspberry Pi to make sure the expected behavior happens.

URL Expected behavior
http://(less-thanRaspberry’s IP>/cpu/20/10 Both plugs are off.
http://(less-thanRaspberry’s IP>/cpu/20/30 Only the heater plug (the bottom one) is on.
http://(less-thanRaspberry’s IP>/cpu/60/30 Only the fan plug (the tom one) is on.
http://(less-thanRaspberry’s IP>/cpu/85/10 Both plugs are off.

If you want to verify that only readings from the last ten minutes are applicable, browse to a URL that turns one of the plugs on, wait nine minutes, see that it is still on, and then wait a bit more than a minute, and see that it is off.

These are the new parts of the application. The readings are stored in an array called recentReadings.


// The recent readings, an array of structures, 
// each with temp, humidity, and timestamp.
var recentReadings = [];

The functions that manipulate recentReadings are grouped together for convenience.

// recentReadings manipulation

The max function is trivial, but having it as a function makes the getMax function more legible.


// Get the maximum between two numbers
var max = (a, b) => a > b ? a : b;

This function gets the maximum temperature and humidity from the recentReadings array using the reduce function. Reduce receives a function parameter that itself takes two parameters. If the array has multiple items, it runs that function on the first two, then on the result, and the third item, and so on until the array is reduced to a single value. If there is a single value in the array, reduce returns that value. If there are no values, reduce can return the optional second parameter.

In this case, the array is composed of associative arrays with a temp and a humidity (as well as a time stamp that is not relevant). The function that is passed to reduce takes two of those associative arrays and returns an array with the higher temperature and the higher humidity between the two. After reducing the entire array, we get the maximums of temperature and humidity between all the readings. The default value is {temp:0, humidity:0}. If there are no values, getMax returns this value, which turns both the heater and the fan off.


// Get the maximum temperature and humidity
var getMax = (arr) => {
  return arr.reduce((a,b) => {
       return {temp: max(a.temp, b.temp), 
humidity: max(a.humidity, b.humidity)};}, 
{temp:0, humidity: 0}); // End of arr.reduce
};    // End of getMax 

This function removes old entries using filter. JavaScript counts time by milliseconds, and we only keep those entries whose time stamp is more recent (higher) than the time stamp now minus 10(minutes) * 60 (seconds/minute) * 1000 (milliseconds/second).


// Remove old (>10 minutes) readings
var removeOld = () => {
    recentReadings = 
      recentReadings.filter(
        a => a.time > Date.now()‑10601000);
};


With the functions previously defined at our disposal, we can finally do the edge analytics in the update function.


// The actual analysis

var update = () => {

The names fanPlug and heaterPlug are easier than the top and bottom plugs.


var fanPlug = plugAPI.top;
  var heaterPlug = plugAPI.bottom;
               

Remove the old readings, if any.

removeOld();

If the Bluetooth code hasn’t provided us with the plugs yet, there is no point doing analytics.


  // No point in updates until we have the plugs
  if (fanPlug === undefined || heaterPlug === undefined)
    return;
               

Get the structure with the maximum temperature and humidity.

var maxValues = getMax(recentReadings);

Here’s where the actual analytics happens. Both plugs need to be off, except during specific conditions.


// Desired states, off by default
var fanPlugState = false;
var heaterPlugState = false;


If the temperature is in the 50 – 80 degrees centigrade range, turn on the fan.


// Try to reduce the temperature using the fan
  if (maxValues.temp < 80 && maxValues.temp > 50)
    fanPlugState = true;

If the temperature is lower than 50 and the humidity above 20 percent, turn on the heater. In this case, the fan won’t be on (because the temperature is outside the range). Otherwise, we’d have to decide if there are any cases that justify using the fan and the heater at the same time.


// Try to dry the hay using the heater
  if (maxValues.temp < 50 && maxValues.humidity > 20)
    heaterPlugState = true;

This console.log call uses a template literal, as explained previously.


  // Show what is happening
  console.log(Update. Maximums: ${
JSON.stringify(maxValues)
}, fan: ${fanPlugState}, heater: ${heaterPlugState});

  // Write the states
  writePlug(fanPlug, fanPlugState, () => {});
  writePlug(heaterPlug, heaterPlugState, () => {});
};

The program is getting complicated. To make it easier to debug problems, it is useful to be able to view the state information, possibly from a smart phone or laptop (the Raspberry Pi does not have to be connected to a screen). The easiest way to do this is to use the web server we already have, and return the requested value instead of HTML.


// Debugging information

app.get("/readings", (req, res) => {
  res.send(recentReadings);
});


app.get("/max", (req, res) => {
  res.send(getMax(recentReadings));
});


app.get("/plugs", (req, res) => {
    readPlug(plugAPI.top, topState => {
      readPlug(plugAPI.bottom, bottomState => {
        res.send(top plug: ${
topState
}, bottom plug: ${bottomState});
      });   // readPlug bottom
    })      // readPlug top
});         // app.get("/plugs"


In addition to the new code, there are two places in the existing code that need to change. One is the handler for readings. When we get a new reading, we need to parse the URL components as numbers (except for the CPU identifier, which is irrelevant), and push a new associative array into the recentReadings array. We also need to run update because the new reading might change the maximum temperature or humidity.


// Process readings
app.get("/:cpu/:temp/:humidity", (req, res) => {
  res.send("Hello, world");  // Just to respond with something
  recentReadings.push({
    temp: parseInt(req.params.temp),
    humidity: parseInt(req.params.humidity),
    time: Date.now()
  });
  console.log(${req.params.cpu} reports ${
req.params.temp} C and ${req.params.humidity}%);
  update();
});


The other place for a code change is the Bluetooth code. There is no point in running update before the plugs are discovered (the reason update checks if the plugs are available is in case of getting readings early). Once the plugs are discovered, we want to run update every minute in case we need to change plug status because an old reading is no longer relevant (older than ten minutes).


plugDevice.discoverSomeServicesAndCharacteristics(
    plugServices, 
    plugCharacteristics,        
    (err, services, charObjs) => {
        Object.keys(plugChars).map((plugName) => {
            plugAPI[plugName] = charObjs.filter(
                c => c.uuid === plugChars[plugName])[0];
        });

        console.log("APIs: " + Object.keys(plugAPI));

            // Update every minute even if there are no new readings
            setInterval(update, 60*1000);
    });  // plugDevice.discoverSoServicesAndCharacteristics


Preparing for deployment

The way you deploy this system in a barn depends on the exact configuration. If the barn is close enough to use the farmhouse wifi, you can just give the Raspberry Pi a constant IP address and have the NodeMCU devices connect to the farmhouse wifi. Then, those NodeMCU devices can use the farmhouse wifi for both purposes:

  1. Update the Watson IoT Platform (using MQTT, for example). Use the same code as in this example.
  2. Update the Raspberry Pi using HTTP, as shown in this example.

If the barn is more remote, it depends on whether you want to update the IoT Platform or not. If you don’t, all you need to do is configure the Raspberry Pi as an access point. Ignore the part of the linked page that deals with routing and masquerading; it is not relevant in this case. Then, you can use the long distance NodeMCU program without any changes.

If you do need to update the IoT Platform using LoRa, use the barn ESP32 as shown in the first article in the series. In that case, put the Raspberry Pi on the same barn-net network with a different IP address (for example, 10.0.0.254). Duplicate lines 19 – 22 in the requestUrl function (in this program) to inform both the barn ESP32 and the Raspberry Pi of the readings.

Conclusion

In this article, you learned how to use a Raspberry Pi with Node.js to do simple edge analytics to activate safety equipment without relying on the Internet. To do this, you learned how to control a smart plug using Bluetooth and how to write a Node.js HTTP server.

In the final part in this series, I’ll show you how to create a dashboard for your IoT system to visualize and take action on the sensor data.