MQTT: Enabling the Internet of Things

Dave Locke
Published on 26/04/2013 / Updated on 13/01/2014

Our world is filling with devices – and this isn’t just about connecting people. Nowadays we are also building systems of systems; of machines talking to machines – often across low bandwidth and unreliable comms links. These systems have to work with mobile devices, so we also have to minimize CPU and power usage. MQTT is a transport protocol that enables minimized pub/sub messaging across mobile networks. It significantly reduces message sizes, and by replacing polling with pub/sub it speeds up communications and at the same time dramatically reduces CPU use, power consumption and network traffic.

For the first 100 years, instant electronic communications meant people telephoning each other using fixed devices connected by cables. With the arrival of wireless, satellites and the web, phones were no longer tied to places and became associated with individual people. Then computing devices became so small, cheap and reliable that they could be installed almost anywhere – and the Internet of Thingswas born.

But this isn’t just about scale. 5 years ago, your phone only talked to other phones. It sent data to others when you asked it to, and mobile phone companies led the market. Today, your phone can connect to almost anything. It shares and receives information automatically, and computer and content companies drive the market. Tomorrow, what will replace your phone? Who will it talk to? What companies will lead?

And this isn’t just about connecting people. Today we are building systems of systems. The latest generation car has

• 100 electronic controllers
• 10 million lines of code
• its own IP address

and it was developed in just 29 months:

And this isn’t just about instrumentation. Most of today’s edge connectivity follows a similar pattern:

• Devices collect data for central processing.
• Decisions are made centrally and pushed out to devices.

What if we start adding intelligence throughout the network? What if we

• extend connectivity beyond enterprise boundaries to smart devices
• offer connectivity capabilities optimized for sensors and devices
• deliver relevant data to intelligent decision making assets to derive business value
• enable massive scalability of deployment and management of solutions

When the Internet of computers took off, interoperability was the key:

• Data transmission standards (Ethernet etc.)
• Addressing standards (IP, DNS, URI etc.)
• Discovery and routing standards (RIP etc.)
• Transport standards (TCP, UDP etc.)
• Data standards (HTTP, SMTP, POP etc.)

For the Internet of things, HTTP isn’t enough:

• The HTTP standard revolutionized how people consume data:
• A single simple model: Send a request, read the response
• Available via any tablet, laptop, phone, PC etc.

The Internet of Things has fundamentally different challenges:

• HTTP remains ideal for requesting data from a known source
• We also need an event-oriented paradigm:
• Emitting information one to many
• Listening for events whenever they happen
• Distributing minimal packets of data in huge volumes
• Pushing information over unreliable networks

And there are extra comms challenges for Mobile & M2M apps:

• Volume (cost) of data being transmitted (especially in M2M with limited data plans)
• Power® consumption (battery powered devices)
• Responsiveness (near-real time delivery of information)
• Reliable delivery over fragile connections
• Security and privacy
• Scalability

The MQ Telemetry Transport (MQTT) protocol is designed to meet these challenges. MQTT was invented by Dr Andy Stanford-Clark of IBM®, and Arlen Nipper of Arcom (now Eurotech), in 1999. The protocol is openly published with a royalty-free license, and has been widely implemented across a variety of industries. MQTT.org, defines the protocol as follows:

MQTT is a machine-to-machine (M2M)/”Internet of Things” connectivity protocol. It was designed as an extremely lightweight publish/subscribe messaging transport. It is useful for connections with remote locations where a small code footprint is required and/or network bandwidth is at a premium. For example, it has been used in sensors communicating to a broker via satellite link, over occasional dial-up connections with healthcare providers, and in a range of home automation and small device scenarios. It is also ideal for mobile applications because of its small size, low power usage, minimised data packets, and efficient distribution of information to one or many receivers.

As an example, consider the following system for collecting and using telemetry data received from sensors in a pipeline:

Without MQTT:

With MQTT:

Key facts about MQTT:

• It’s open
• Open royalty free spec designed for the world of “devices”
• Wide variety of clients and servers
• Hobbyist to enterprise
• Open source to commercial
• It’s lean
• Minimized on-the-wire format
• Smallest possible packet size is 2 bytes
• No application message headers
• Scalable
• Reduced complexity/footprint
• Clients: C=30Kb; Java=100Kb
• It’s reliable
• Three qualities of service:
• 0 – at most once delivery
• 1 – assured delivery but may be duplicated
• 2 – once and once only delivery
• In-built constructs to support loss of contact between client and server
• “Last will and testament” to publish a message if the client goes offline
• Stateful “roll-forward” semantics and “durable” subscriptions
• It’s simple
• Simple / minimal pub/sub messaging semantics
• Asynchronous (“push”) delivery
• Simple set of verbs — connect, publish, subscribe and disconnect

The following table compares messages per hour, and battery usage per message, between HTTP and MQTT networks (Source: http://stephendnicholas.com/archives/1217).

The realm of MQTT

You can think of the Internet of Things as being the coming together of all the mobile, web and m2m devices and applications that sit at the edge of the network. Businesses need a way to capture, process, and respond to this information. The MQTT protocol is designed to enable fast and efficient interaction between these devices and a central MQTT server (for example WebSphere® MQ or IBM MessageSight). Through the MQTT server, the business’s back-end systems can make use of, and respond to, the messages coming from those devices.

The computing requirements to connect the backend systems to the MQTT server are familiar: they are the traditional, often heavyweight, processes and protocols that are well suited to communications within robust and secure internal networks. What is new is the connection between the MQTT server and the edge of the network: this is conducted over networks which might be fragile, low bandwidth and expensive, and on devices where battery power must be protected and conserved. This is the environment that the MQTT protocol was designed for.

Note: To ensure that the disruption to existing backend systems is minimized, many MQTT servers (including those from IBM) can communicate across other protocols as well as MQTT. For example, your server can be configured to talk MQTT externally, and JMS internally.

In the following diagram, the shaded ellipse shows the realm of MQTT:

MQTT-related products and technologies

IBM, third parties and open source communities all provide client libraries and servers that support MQTT.

Available from IBM:

• Enterprise MQTT servers:
• WebSphere MQ
• IBM MessageSight
• Mid size/premises servers:
• WebSphere Sensor Events
• Edge servers:
• WebSphere MQ Telemetry Daemon for Devices (aka Really Small Message Broker (RSMB)) – ships with WebSphere MQ
• Client libraries:
• MQTT client for Java™ – for building native apps for (for example) Android, OS X, Linux or Windows devices
• MQTT client for C – for building native apps for (for example) iOS, OS X, Linux or Windows devices
• MQTT messaging client for JavaScript – for building browser-based web apps

Available from third party companies and open source communities:

• 16+ MQTT servers
• 40+ MQTT client libraries

Author