Good news! You are already an expert on messaging (you just don’t know it). Messaging is part of everyday life and critical to how stuff gets done. As human beings, we’re continuously sending and receiving information, and we’re incredibly good at processing data to understand how to function in real time in the world around us. Yet, we take our sophisticated skills for granted. Your phone rings, you answer it, and instantly you share data with the person at the other end.
What would a world with no messaging look like? This idea might seem far-fetched, but it’s a very real problem in the world of computing: How do you talk to software? How does software talk to other software? How does software talk to you?
This is where messaging software plays its part. We software developers hate having to solve the same problem over and over again; it’s an inefficient use of our time, inconsistencies creep in, compatibility suffers, and it’s bad news for scaling our application to reach more users. Because moving information around is fundamental to our application ecosystems, developers just want a simple hook into a way of doing messaging such that we can focus fully on the value and quality of our shiny new code.
What’s the difference between messaging and messaging?
Not all messaging systems and messages are equal. If you want to let a friend know you’re running a few minutes late, then you might send a short message “I’m going to be 10 minutes late.”
Exactly what happens when you click send depends on the technology behind the message service. Typically, this type of system is optimized for availability and speed because assuring delivery of the message can be computationally expensive, especially where there is a high throughput.
In short, the service will make a best effort to transport your message at most once and your friend will likely receive it, but there is some chance they might not. In this example, the impact of not receiving the message is probably fairly low and easily remedied with a simple apology.
Most messaging systems provide a range of what they call “Qualities of Service (QoS)” options to support the varied needs of application designers for the sending and receiving of messages. For example, the MQTT protocol defines three:
- QoS 0: at-most-once
- QoS 1: at-least-once
- QoS 2: exactly-once
Would you be as comfortable with an “at-most-once” QoS with a more important message?
Consider the example of a team of mountain climbers reaching a communication checkpoint:
[climber] > Dave has injured his leg. > Can you arrange for an emergency rescue team? > It's much warmer than we thought. > Can you send water to the next checkpoint?
In this example, it’s really important that the second and fourth messages are processed. Clearly, we need to send these messages with a QoS that ensures delivery; for example, an “at-least-once” or “exactly-once” QoS would seem desirable. Furthermore, half way up a mountain the network coverage will be patchy, and the climbers may not have the time to wait for a reply, so a “sent” acknowledgement really does have to mean sent.
Sometime later, the climbers receive the acknowledgement:
[support] > OK.
The reply makes some sense, but which message does it relate to?
This conversation between two parties in messaging parlance is often referred to as a request-response exchange. The “request-response” message exchange pattern provides a mechanism to correlate messages so we can work out which message the response is for.
Returning to our example:
[climber] > [messageId=002,correlationId=001] Can you arrange for an emergency rescue team? > [messageId=004,correlationId=002] Can you send water to the next checkpoint? [support] > [messageId=001,correlationId=001] OK. > [messageId=002,correlationId=002] OK.
Now we have positive confirmation that both messages have been processed by support and that help, and supplies, are on the way.
If you want to be sure your message is received, then one approach might be to keep sending it. And this “fire-and-forget” message exchange pattern works well for certain types of data in certain applications. A chatty temperature sensor might not be a bad thing. If a thermometer in a living room is communicating to a heating system every sixty seconds, then when a momentary network outage leads to a reading being missed, it’ll be at most sixty seconds until the next one arrives.
Although simple, this approach might not always result in the right outcome. For example, it wouldn’t be wise to keep clicking “Buy Now” on an internet shopping site every few minutes until the first acknowledgement arrives in the form of a delivery driver turning up at your home. In this scenario, we want the message related to our shopping session to be delivered “once-and-once-only” or expect that the processing application is smart enough to deduplicate purchase requests.
Message queuing to the rescue
It’s difficult to have a great conversation when no one is listening. Voicemail was invented for just this type of problem.
Most software applications are structured as a collection of interdependent services. Some of these might be owned in-house while other services could be owned and managed by third parties. Returning to our shopping example: the web front-end might be owned in-house while payment services, fulfillment services, and shipping services come from external providers. This collection of services depends on some form of asynchronous data exchange to facilitate frictionless communication. What happens if one of the services isn’t available or is just running slow? Our business can’t just grind to a halt as a result.
Message queuing allows for application decoupling so that we remove the need for long-running and brittle processes. Once a message is sent, it is stored safely on a queue until the consuming application is able to process it. The sending application can move forward and continue to process other orders. IBM MQ and the Java Message Service API are two such message queuing implementations.
Two messaging domains, or messaging styles, are used in messaging systems:
Point-to-point messaging – This style of messaging uses message queues, where messages are processed by a single consumer. Message producers are called senders, and message consumers are called receivers.
Publish/subscribe messaging – In this style of messaging, copies of messages are delivered to all interested consuming applications. Message producers are called publishers, and message consumers are called subscribers.
Apache Kafka is a publish/subscribe based messaging system. IBM MQ supports both messaging styles.
Security is essential. As consumers we instinctively check for the padlock symbol in our web browser’s address bar to ensure we have a secure connection when shopping online. Developers also need to ensure message exchanges between components within an application are safe and secure from hackers who might attempt to look at or manipulate them.
In software, security is achieved by employing industry-standard cryptography and encryption techniques. Software security is a broad term and the exact capabilities that an application needs influence the choice of messaging technology.
Typically, message security has these main considerations:
How users are authenticated and are they authorized to use resources such as queues and topics? Furthermore, does the messaging software integrate with authentication systems already in place?
How is data in transit protected? For example, is there support for Transport Layer Security (TLS) to provide encryption?
How is data at rest protected? For example, are the messages or their payloads encrypted when stored?
Messaging software provides a convenient, flexible and structured way to build communication into software applications. Supporting a variety of typical message exchange patterns and a range of qualities-of-service, messaging software frees up application developers to focus on the business value of their code. Used properly, messaging frameworks promote loosely-coupled application deployment, scaling of solutions, and help developers maximize reuse of existing services.