Rapidly develop Internet of Things apps with Docker Containers
Develop, test, deploy, and update IoT apps using container-based virtualization
The Internet of Things, the highly connected network of smart devices, such as environmental sensors, medical trackers, home appliances, and industrial devices, is growing rapidly. By 2020, a predicted 20 billion devices will be connected, which is more than twice the number of PCs, smartphones, and tablets combined. Developers are rapidly starting to create applications for the IoT, and using containers can help them in different ways.
Containers are a lightweight approach to virtualization that developers can apply to rapidly develop, test, deploy, and update IoT applications at scale. Many web and mobile app developers make use of hypervisors, such as VirtualBox, to run virtual machines (VMs) that virtualize physical hardware as part of a cross-platform development, testing, and deployment workflow.
Container-based virtualization (sometimes called operating-system-level virtualization) is much more lightweight. Each container runs as an isolated user space instance on top of a shared host operating system kernel. Although the operating system is shared, individual containers have independent virtual network interfaces, independent process spaces, and separate file systems. These containers can be allocated with system resources like RAM by using control groups that implement resource isolation. Compared to hypervisor-based virtualization, where each VM runs its own operating system which only increases its use of system resources, containers use much less disk and memory resources.
Docker is an open platform for container-based virtualization on Linux. Docker makes it fast and easy to build containers and to deploy them just about anywhere: in a private or public cloud, within a local VM, or on physical hardware including IoT devices. IBM Containers are a IBM Cloud feature based on Docker and Kubernetes for delivering and deploying containerized applications on the IBM Cloud platform.
Setting up Docker for IoT development
IoT applications target a wide variety of device platforms. During prototyping and early phases of development, IoT projects are often developed using generic microcontroller-based development boards or single-board computers (SBCs) like the Raspberry Pi.
To get set up with Docker for Raspberry Pi, you’ll need a Raspberry Pi with wifi (for example, a Raspberry Pi Zero W or a Raspberry Pi 3), with a microSD card imaged with raspbian Jesse, and SSH enabled. (Watch this Hands-on IoT video for how to set up a Raspberry Pi.) You can enable SSH by creating a file named ‘ssh’ on the boot partition on the microSD card after it has been imaged. You can access the Raspberry Pi in headless mode from a Mac or PC via SSH (the default password is raspberry):
From the raspbian command line, you can install Docker by running the following script:
$ curl -sSL get.docker.com |sh
You’ll want to add the default user (pi) to the docker group to avoid permissions issues, and then switch user (su) to pi so that the changes take effect, or reboot the Raspberry pi.
$ sudo usermod ‑aG docker pi $ su pi
Once you have Docker installed on the Raspberry Pi, you can run docker ps to confirm that Docker is running.
$ docker ps CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
Working with containers, images, and registries
Each Docker container contains one or more running processes. The Docker container is launched from an image that specifies these elements:
- The configuration information for an application to be run inside the container
- Its dependencies, for example, libraries and shared binary files
Docker makes use of a union file system to store images as a series of layers. Layers are cached during the build process, which makes building derivative images fast and efficient.
This layered approach also means that Docker images are small and portable, which makes them easy to share by publishing them to a public or private registry. Docker Hub is the largest registry of Docker image repositories; it lists more than 100,000 repositories and hosts numerous private repositories. If you are using the IBM Cloud platform, it supports pulling public images from Docker Hub and also provides hosted private image repositories which allow you to share private images within your organization.
If you’re using a popular open source framework or service, it’s likely that you’ll be able to find a pre-built public image in Docker Hub, with many of the images maintained by the open source communities that are associated with those projects. The Docker Hub web interface features a small list of official repositories, which is a curated list of repositories that include images that are tested by the Docker team for known security vulnerabilities.
Avoid using older Docker images, even from reliable sources, as many older images are provided for archival and testing purposes only. Older images are tagged as supported in Docker Hub if they are still maintained. The majority of images in the public registry are community contributed and may not be well documented or maintained. When in doubt, create your own image using one of the official images as a base.
You can create an image by completing these steps:
- Pull a base image from the registry
- Run a series of commands interactively
- Commit the result as a new image
For example, the following list of commands run on a Raspberry Pi with Docker installed pulls the latest ARM-based Alpine Linux image from the Docker Hub, launches a container from the image to run a command that installs Git into the container, and then lists the ID of the container that was created.
Listing 1. Creating an image
$ docker pull armhf/alpine:latest $ docker run ‑t ‑i armhf/alpine apk add ‑‑no‑cache git $ docker ps –l
You can save this container (cb23e345fde0) as a new image (named demo/git) by issuing the following command:
$ docker commit cb23e345fde0 demo/git.
If you are running Docker on a PC or Mac, to run Docker commands locally by using the IBM Container Extensions tool, change the
docker command to
ice --local. For example, use this command to pull Ubuntu 16.04:
$ ice --local pull ubuntu:16.04
If you need to run more than one or two commands to set up your application environment, you’ll want to create a Dockerfile. Dockerfiles are text files that specify a list of instructions for building an image. The Dockerfile in the following code listing creates a base image for running an Nginx web server.
Listing 2. Dockerfile that creates a base image for running an Nginx web server
FROM armhf/alpine:latest RUN apk update && apk add nginx RUN mkdir ‑p /var/www/html RUN chown ‑R nginx:www‑data /var/lib/nginx EXPOSE 80 443 WORKDIR /etc/nginx CMD "nginx"
The first instruction in each Dockerfile is a
FROM instruction, which indicates the base image. Each subsequent instruction in the Dockerfile is stored as a layer:
RUNinstructions run Linux commands such as
COPYinstructions add application files into the container.
EXPOSEinstructions open ports.
ENVinstructions configure environment variables.
Finally, each Dockerfile contains
CMD instructions to specify how and where to run the application when the container is launched.
Running Docker containers requires very little overhead, so you can break your application up into a set of services that run in separate containers that can be linked by name when you run them. For example, you might split your application into a container running a Node.js application linked to another container running a Redis key-value store.
Maintaining a consistent IoT development environment using containers
The embedded applications that are developed for IoT devices can later be upgraded to run on custom prototype development boards and then finally on production devices. You might start using off-the-shelf hardware like Raspberry Pi and later evaluate a range of potential microcontrollers and system-on-a-chip (SoC) devices, with distinct and sometimes incompatible device driver requirements. (Review the different IoT hardware available in the IoT hardware guide in my other developerWorks article.) Each device or device revision might also require different versions and configurations of the development toolkits that are used for flashing, monitoring, and communicating with the device.
Containers can be used to capture a development environment that is known to work for each device revision, and to share this environment among a team of developers. For example, if your team was working with several types of Arduino-compatible development boards, you can create an image containing the Arduino command line toolkit and a dumb-terminal emulation program for serial communication as your baseline development image. You can then use that base Arduino development image to create new variant images for each of the development boards that require specific custom drivers.
Docker’s layered file system makes efficient use of space in this situation, since only the unique layers are stored when a new image is created. The docker history command displays a list of the various layers making up an image, including the instruction that created each layer and the size of the layer, such as:
IMAGE CREATED CREATED BY 7483ffb80dd6 5 minutes ago apk add ‑‑no‑cache git
The layers are cached, so you can quickly try out new variations of the image. For example, you might want to apply updates to drivers or development tools. Layers also make it painless to roll back to an earlier version of the image and try a different approach if something goes wrong.
If the changes to your development environment are successful, the new image can be pushed to your private team registry, to rapidly propagate the changes to the rest of the team. When each team member pulls the updated image, they’ll already have most of the layers cached, so getting the new image up and running will typically take only a few seconds to a few minutes for Docker to run through the instructions to create the new layers. Compared to pushing and pulling snapshots of VM images or building entire images from scratch for each variation, layers save storage space but also more importantly your developer’s time.
Deploying containers to IoT devices
The appeal of a Dockerized application is that after you have built an image, you can ship and run it almost anywhere. If your IoT device runs Linux, you might be able to deploy containers directly on your device. Despite the limited system resources that are typically available on such IoT devices, it is feasible to deploy Docker containers because their runtime overhead is almost zero. You can even run multiple containers on your device, for example, if you wanted to run different versions of your application side-by-side for comparison.
Docker requires a modern Linux kernel with support for kernel namespaces and control groups, so suitable base images might not yet be available for your preferred IoT device. You can expect to see more Linux-capable IoT devices being developed as the price point of suitable SoCs continues to drop (for example, the $9 C.H.I.P. development board); then, a wider choice of Linux distributions targeting these platforms will become available.
Docker Hub contains a number of ARM-based images for popular hobbyist SBCs based on the ARM architecture, including the Raspberry Pi, Orange Pi, and BeagleBone Black. The arm32v7, arm32v6 and deprecated armhf images on DockerHub are the official images built for the ARM architecture as part of DockerHub’s multi-arch support. The resin organization on DockerHub also provides third-party base images designed for use with resin.io and resinOS.
IoT developers face the challenge of keeping a network of connected devices updated and running on potentially unreliable and low-bandwidth wireless connections. Also, they must consider how to maintain the security of sensitive and highly personal data that many of these devices collect. Docker containers can help with IoT security by supporting isolation through user namespaces, introduced in Docker 1.10, and Docker secrets, available since Docker 1.13, which can be used to store and encrypt keys or access tokens used for authentication and secure communication between devices or upstream to cloud services.
IoT devices might only be connected sporadically, with some devices set to sleep on different schedules to conserve power. Other IoT devices might be connected by using a mesh network topology where only parts of the network are reachable at any given time. Push-based updates are likely to fail when network connectivity drops out, and the danger of applying incomplete or potentially corrupted updates is that devices can enter an inconsistent state or be rendered inoperable.
Docker provides a possible solution to this update problem. A device that issues a pull request for the latest version of its application image will be sent just the image diffs over the air rather than the entire image. Diff-based updates will complete much more quickly, which reduces the amount of time that the device needs to be connected and reduces the probability of failure, thus putting less stress on low-bandwidth networks. This makes it possible for updates to be applied more frequently.
Some IoT devices allow limited user interaction through physical buttons or small touchscreens, however, for many current generation IoT devices the primary means of user interaction is with mobile applications. It should come as no surprise that Docker containers are increasingly being adopted to speed up automated testing, continuous integration, and delivery of mobile applications.
Integrating IoT devices with the cloud
Deploying applications to smart devices and mobile devices is only part of the Internet of Things development story. Edge analytics may be performed within containers running on IoT devices and gateways, however, many of the current generation of IoT devices also publish sensor data and events to cloud services for further processing.
Containers are frequently adopted for cloud services. However, containers are ephemeral, so any data written to the file system within a running database container should be considered temporary; that is, the data will be lost if the container is shut down. In a sense, using containers forces you to establish the good habit of developing stateless cloud applications. Any data to be persisted should be stored using a data volume. Data volumes persist after a container shuts down, and can be shared between multiple containers.
As the number of connected devices increases, cloud-based applications for the IoT will need to scale to handle the volume of data being generated. Fortunately, we can take advantage of the growing ecosystem of orchestration tools built around Docker for developing scalable cloud applications, including the Docker Machine, Swarm, and Compose tools. These tools are available on a number of cloud platforms, including EC2 Container Services, Microsoft Azure, and IBM Cloud. IBM Cloud also currently supports running groups of containers for load balancing and failover.
To meet the predicted demand for applications for the Internet of Things, developers will need to adopt tools and practices to enable them to rapidly develop IoT applications and services to run on smart devices, mobile devices, and in the cloud. With the ability to ship and run applications anywhere, on a broad range of devices, with minimal runtime overhead, with support for automation, and with a layered file system that results in lightweight portable images and fast image builds, Docker containers are a great tool for IoT developers.