Build your IoT skills by developing a door monitoring system
Dive deep into the code for building a scalable IoT solution
In the “Get started developing IoT solutions by building a simple home automation system” tutorial, you assembled a door monitoring device using multiple sensors connected to a Raspberry Pi, used Node-RED to design IoT flows, and then used the Apple HomeKit framework to monitor the system using an iOS application. That approach involved little to no code.
In this tutorial, we’ll build a similar but slightly more complex IoT system, a door monitoring system. Our door monitoring system will focus on the architecture and design, analytics, security, and other aspects of an IoT system. To minimize fiddling with hardware and wires, we’ll use a smartphone device in this project as a proxy for the collection of discrete hardware sensors. I’ll use an old Android phone during development, which includes an accelerometer, a proximity sensor, and a light intensity sensor. Most smartphones, even the older ones, will have these basic sensors. That’s why a real smartphone will serve as a good alternative for actual sensors.
Our door monitoring system has the sensors (accelerometer and proximity sensors in a smartphone) attached to the door. Using the proximity sensor in the smart phone, you can detect if someone is approaching or near to the door. Using the accelerometer, you can determine if someone is knocking at the door. This sensor data can be used to trigger different actions like ringing a bell or sending notification. In short, our door monitoring system consists of a Smartphone stuck to the door, sending data to a gateway device (which is Raspberry Pi), and the Raspberry Pi in-turn forwarding that data to a Node-RED dashboard, which the reader can use to see when the door is knocked on when the accelerometer graph changes.
To make a working prototype, you will need:
- An IBM Cloud account. Learn more about IBM Cloud by reviewing the Getting Started documentation. Although you need a pay-as-you-go account, all of the services we use are either part of a Lite or free tier or you will not meet the free tier threshold for the service.
- An Android Smartphone (acting as a sensor node for the door)
- Android Studio installed on your laptop (optional)
- A Raspberry Pi 3B+ or Raspberry Pi 4B with Raspberry Pi OS installed.
- A DHT22 or DHT11 sensor for sensing temperature and humidity.
- A micro USB cable and adapter to power the Raspberry Pi. You can also use an Android Smartphone USB cable and power adapter.
- A Raspberry Pi case or enclosure to prevent damage to it
If you are going to use some other single-board computer instead of Raspberry Pi, make sure that it has wifi in it.
Architecture of our IoT system
A simple IoT architecture can be seen as the one consisting of two layers: the device layer and the cloud layer. The devices are connected directly through the cloud via the network. The cloud orchestrates the devices and uses their data for analytics, monitoring, and so on. This structure can work for simplistic use cases, but as the system scales, we might have to add more complexity.
A typical IoT architecture involves many heterogeneous IoT devices, with sensors that produce data in different formats at different rates, which is then processed and analyzed to gain insights. IoT devices can be connected either directly to a network or through a gateway device to a network, which enables the devices to communicate with each other and with cloud services and applications. The devices can be geographically separated and can transmit information over any of the wireless protocols like Bluetooth, Zigbee, WiFi, Cellular, RFID, or over-the-wire protocols like Ethernet.
So it helps to add a layer between the devices and the cloud where these heterogeneous protocols and data formats can be aggregated before sending them to the cloud. This layer can be called gateway or edge layer, and it can be integrated into your router or on another dedicated device altogether.
Our IoT system will have these three layers:
- Device layer, or sensor nodes, which mainly collect sensor data and communicate that data to a gateway, while keeping the power consumption as minimum as possible.
- Edge layer, or gateway. A gateway that performs local computations, such as filtering unnecessary noise or performing logical operations on the data, is called an edge device. The edge layer acts as an abstraction layer between the cloud and the sensor nodes.
- Cloud layer, where data storage, visualization, and analytics usually take place.
The architecture can become complex if the solution needs to be scalable. For example, IBM has an IoT reference architecture that covers almost all of the aspects of an IoT system while respecting scalability needs.
System architecture of our IoT system
In our Door Monitoring System, we will use a single sensor node (our device layer) which will be an Android smartphone (which has by default accelerometer and proximity sensors in it) running an App. Then, we will use a Raspberry Pi as a gateway (our edge layer). The Raspberry Pi filters and aggregates the data before sending it to our dashboard. Finally, in IBM Cloud (our cloud layer), we will store our sensor data in an IBM Cloudant database, create a Node-RED dashboard to visualize the data, and use Watson Studio to do some data analysis.
The system architecture of our door monitoring system looks something like this:
IoT protocols for our IoT application
In an IoT system, there may be heterogeneous devices using different protocols to communicate with the gateway. The most prevalent protocol in the IoT world, however, is MQTT. MQTT is a machine-to-machine messaging protocol. It is designed to be lightweight and fast, keeping in mind the power and resource constraints of a typical IoT device. The protocol runs over TCP/IP and operates on the publish-subscribe model.
The key entities in a functional MQTT messaging system are:
- MQTT client
- Message broker
- Topics and messages
The broker is the central entity responsible for sending messages at the correct time to the correct client devices. The messages are sent along with a specified topic so that the clients subscribed to that topic can receive the messages.
For our door monitoring system, we will use MQTT to send messages between the gateway (a Raspberry Pi) and the MQTT broker, which we install on a baremetal server in IBM Cloud, and the Node-RED dashboard, also running in IBM Cloud. The communication between the smartphone and the gateway will be done over UDP. HTTP is not used in this tutorial to keep things simple.
Now that we’ve settled on the architecture for our system, and chosen the protocols we’ll use, let’s start coding!
All of my code is available in my GitHub repo.
To build our door monitoring system, complete these steps:
- Step 1 explains how to install an MQTT broker on a virtual server. The MQTT Broker is the central part of this project as all of the data communication will be sent and received through the MQTT Broker.
- Step 2 describes how to install and configure Node-RED as a standalone IoT app on IBM Cloud.
- Step 3 explains how to configure a Raspberry Pi as a gateway (edge device).
- Step 4 describes how to create the Node-RED dashboard.
- Step 5 explains how to store the sensor data in a Cloudant database.
- Step 6 describes how to analyze the sensor data using Watson Studio. This is an optional step for advanced analytics.
- Step 7 explains how to set up an Android smartphone as a sensor node.
- Step 8 describes how to visualize the data in a Node-RED dashboard.
Step 1. Installing an MQTT broker on a virtual server
We will use Mosquitto as our MQTT broker, and so we need to install Mosquitto on a virtual private server so that our IoT system can use it as a central entity in our system to send and receive messages among different MQTT clients. You can use IBM Cloud, AWS EC2, or any other cloud service or VPS provider where you can get a Ubuntu Server 20.04. Make sure that you have access to the Security Group of that Server to open in-bound port 1883 for MQTT.
On your Ubuntu-based VPS, open the terminal, and execute the following commands to install the mosquitto MQTT broker.
sudo apt update sudo apt upgrade sudo apt install mosquitto -y sudo apt-get install mosquitto-clients sudo apt clean sudo service mosquitto stop sudo nano /etc/mosquitto/mosquitto.conf sudo apt install ufw sudo ufw allow 1883 sudo ufw allow 22 sudo ufw enable
These commands will open up the file
mosquitto.conf in a terminal-based text editor. Put the following two lines at the end of the
listener 1883 allow_anonymous true
Then, press CTRL+O and CTRL+X to save and exit the file.
Next, start our mosquitto MQTT broker.
sudo service mosquitto start mosquitto -v
Now you have an MQTT broker running on the port 1883 with the address, which is the public IP address of your Ubuntu server.
Step 2. Installing and configuring Node-RED
We will use Node-RED to visualize the real-time accelerometer data sent from the smartphone.
To install Node-RED on IBM Cloud, follow the steps in this tutorial, “Create a Node-RED starter application,” to install Node-RED and deploy our Node-RED app to Code Engine, a managed serverless platform.
Then, we need to add some nodes to the Node-RED palette. Open Node-RED with your App URL link, and open Menu > Manage Palette > Install. Then, search for
node-red-dashboard, and click Install to install it.
Step 3. Configuring a Raspberry Pi as a gateway (edge device)
We will use Raspberry Pi in this project as a gateway device. We will run a python application to make the Raspberry Pi act as an IoT gateway device. (You can also use any other Linux device like a laptop to continue with the tutorial.) The main task of the gateway is to acquire the sensor data from the smartphone, filter it, and then aggregate the data before sending it to the Node-RED dashboard.
Connect the sensors to the gateway
If you have temperature or humidity sensors, such as a DHT11, connect sensors them to the Raspberry Pi. This environment data will be sent to the Node-RED dashboard just like the accelerometer and proximity data from the Android smartphone.
Raspberry Pi has 40 pins GPIO header for connecting sensors and actuators. We will use GPIO21 for the data pin connection of DHT11, GND for ground connection, and +5v for providing 5 volts to the sensor. The final circuit will look like as shown in the following figure:
Installing the latest OS on the Pi
Next, we need to install the latest Rasperry Pi OS on to our Pi. Raspberry Pi can run almost any Linux-based distribution and Raspberry Pi also provides their own Debian-based OS named Raspberry Pi OS. We will be using Raspberry Pi OS in this project because it is easier to install and set up. Follow the instructions below to get Raspberry Pi flashed on your SD Card.
- Download and Flash Raspberry Pi OS on your SD Card by following the instructions on the Raspberry Pi site.
- Put the SD Card in your Raspberry Pi, connect the keyboard and mouse to your Raspberry Pi and power it on.
- Once your Raspberry Pi is turned on, configure the WiFi by clicking on the network/WiFi button on the top right of the screen
- Then open the terminal from Menu > Accesories > Terminal.
Now, update the Raspberry Pi by executing the following commands.
sudo apt update sudo apt upgrade
Installing our gateway code on the Pi
If you haven’t done so already, download the code from my github repo. You can clone the repo using the following commands:
cd ~/Desktop git clone https://github.com/Nauman3S/DoorMonitoringSystem.git cd DoorMonitoringSystem/
We now have the gateway code on our Raspberry Pi and it is ready to run. You can run the code using the following command.
The gateway (Raspberry Pi) will start receiving the data from the smartphone and will forward it to the Node-RED dashboard.
Open another terminal and execute the following command to get the IP Address of the gateway device to make a connection with the smartphone.
The data is acquired from the smartphone, and then filtered and aggregated using the code in
udp_listner.py file. Once the data is ready to be published to the Node-RED dashboard, it is sent using the MQTT handler code in the
Step 4. Create the Node-RED dashboard
I created the Node-RED dashboard and saved the Node-RED flow to a JSON file that I stored in my GitHub repo for this tutorial.
You can import this flow into your Node-RED app to create the Node-RED dashboard.
- Copy the JSON file from my GitHub repo.
- Open your Node-RED app, and from the Node-RED menu, select Import.
- Paste the JSON file into the Clipboard tab.
- Click Import.
The overall Node-RED flow, which represents our Node-RED dashboard for visualizing our data, looks like this:
Now, you need to configure the MQTT node to use your mosquitto broker. Double-click the purple-colored MQTT node, and specify the public IP address of your Ubuntu server (that you created in Step 1) in the Server field while keeping all other settings as default.
This flow includes nodes for our Cloudant database and for Watson Studio, but we’ll come back and configure those nodes later in this tutorial.
Now, deploy your Node-RED app (your Node-RED dashboard) by clicking Deploy.
Once it is deployed, open the Node-RED dashboard by going to this URL
http://AppURL/ui and replacing
AppURL with your Node-RED app URL.
Step 5. Storing the sensor data
We are sending the sensor data to our IoT app (our Node-RED app), and then it is storing the sensor data in a Cloudant NoSQL database for future analytics and visualizations. We can store the events in an IBM Cloudant database, which is a fully managed, distributed NoSQL database.
In Step 2, when you created the Node-RED app, you created a Cloudant database. You need to need copy the service credentials into our Cloudant node in our Node-RED flow.
Log in to your IBM Cloud account.
Go to your resource list. Find your Node-RED app in the Apps list and click on it.
On the App Details page, in the Services section, expand the credentials for the Cloudant service, and click the Copy button in the top right corner of the code block box.
After getting the credentials, we need to bind the application client to the Cloudant service and configure the connector, destination, and rules to send the events to the Cloudant database.
a. Put the credentials in the
application.pyfile on the Raspberry Pi in the already downloaded
application.pyfile handles the database connections.
b. Put the same credentials in the Node-RED Cloudant node in the Node-RED flow. Double-click the Cloudant node in the Node-RED dashboard flow to enter the credentials.
In short, we configured our application with the Cloudant service in a way that all the Android smartphone sensor events (change in accelerometer and proximity sensor values) go to the Cloudant database, and all the Raspberry Pi status events (iot-2) are sent to another database. Both databases are bucketed by the interval of a day.
If you go to the Cloudant service dashboard, you will have generated the following databases. For more information on how to see the existing or newly created databases, you can refer to this article, “Using the IBM Cloudant dashboard” in the IBM Cloud docs.
And they will have your events data in the form of documents. Click the JSON button to view the event data in JSON format:
Step 6. Analyzing the sensor data using Watson Studio (optional, advanced analytics)
We already have Node-RED dashboard configured for analysis but for advanced analytics we can use IBM Watson Studio. By using Watson Studio, you can run and manage AI workloads and analyze (and visualize) data. Watson Studio is one of the core services in Cloud Pak for Data as a Service.
- Log in to your IBM Cloud account.
- Create a Watson Studio service using the lite plan.
- Click the Launch in IBM Cloud Pak for Data button.
- Under projects, click New project. Then, click Create an empty project.
- Specify the name of the project, and then add storage for the project, and click Create.
You can start adding resources to it by clicking the Add to project button.
Select Notebook, enter a name and description, and choose Spark with Python environment.
To use our Cloudant data, we need to add a connection resource to the project. For that, go to the project page of Watson Studio, click Add to project again, and choose Connection asset, select your Cloudant service, and click Create.
In your notebook, you’ll be able to see the Cloudant service, click on credentials, and insert the credentials from Step 5 above in the field.
Copy the Python code from my GitHub repo, and run it in the notebook.
We have all the smartphone events data available as a PySpark
dataframe object, which can be used for analysis and visualization via PySpark and Python.
Step 7. Setting up your smartphone as a sensor node
First, we need to download, compile, and install our smartphone app, which sets up our smartphone as a sensor node. Then, we need to run the app to start gathering and sending the sensor data to our IoT app, running in the cloud.
Installing and configuring the smartphone app
This project makes use of Android Phones to send sensor data to the Node-RED dashboard for visualization. We have multiple options in this step. We can use Bluetooth, MQTT, web-sockets, or even HTTP. Whatever protocol we select, we’ll have to implement appropriate logic at the Gateway. Our goal is to make this solution as simple as possible.
- Clone or download my GitHub repository to your laptop.
Now, you have two options:
a. Send the
sensor-udp.apkfile from the downloaded repository to your smartphone. b. Compile the whole project using Android Studio and send the compiled app to your smartphone.
Install the app on your Android smartphone.
- Transfer the python files to your Raspberry Pi.
Open the app on your Android smartphone. The app looks like this:
Put the IP address of the gateway device that we noted in Step 3 in the IP address field and click the Start Data button to start sending the data. Once the data starts to be received by the gateway device, you will see something similar to this in the Raspberry Pi (edge device) terminal:
Step 8. Visualizing the data in the Node-RED dashboard
You can now visualize the real-time data being sent from your Android smartphone to the Node-RED dashboard. To test the system, you can try to shake your smartphone and the acceleration values on the dashboard will change. You can put something on the proximity sensor of your phone (mostly present near the front camera of the phone) and the proximity sensor values will change on the dashboard as well.
The dashboard shows the following sensor data and gateway-related parameters:
- The light intensity graph gets the fluctuations in proximity values over time and plot it so that one can see the timeline of change in proximity sensor values.
- Accelerations graph shows the raw accelerometer sensor values in all three x, y and z dimensions while the Abs Acceleration graph is the absolute acceleration change. The absolute acceleration graph makes it easy to infer the events of door opening or closing with respect to the time.
- The proximity sensor shows if someone is in the proximity of the smartphone and it is shown as a simple number in the dashboard.
- The gateway (Raspberry Pi) memory usage and CPU usage is shown in percentages.
In this tutorial, we discussed various components of a typical IoT system. Although IoT is a vast topic and its use cases cover multiple domains, the knowledge of the different concepts discussed in this tutorial will enable you to get going with most IoT use cases.
In this sample door monitoring system, we set up a complete IoT system using cloud services and readily available hardware devices (an Android smartphone and a Raspberry Pi). The data can not only be visualized but can also be stored in databases, and then we can perform multiple operations easily like knock detection. You could extend this project with some simple functions like sending email or SMS messages when someone knocks on the door by using a service like IFTTT or adding some code to identify how many times the door is opened or closed.
This sample and tutorial were originally developed and written by Satwik Kansal.
The authors used the circuit diagrams from these fritzing components to create her own circuit diagrams for this tutorial:
- AdaFruit library:
- DHT11 sensor module:
- PIR Sensor and relay: