Transactions help application developers write reliable distributed applications and provide more ways to control modifications of application state. You can wrap important message exchange flows in ‘state wrappers’ that use features of MQ and the JMS specification. Transactions help to ensure that a number of actions like puts and gets either all happen or none do, and the actions can be reversed and then retried if something in a flow does not work as desired.
To effectively design a resilient solution that optimally uses the transaction capabilities that are provided by MQ and JMS, you need to understand several key concepts. This article introduces you to these concepts and explains how they interact.
Where do we start with transactions?
When we talk about messaging, we say queues are used to decouple applications, acting as shock absorbers, and allow us to build scalable and reliable solutions and applications.
Rather than connecting to each other, applications connect to queues, put and get messages and then disconnect. A collection of these actions combined to achieve a particular task, or operation, is called a unit of work. A unit of work is a set of or a sequence of logical tasks or operations that developers define. When a unit of work is wrapped in a transaction, either all the tasks will complete and the state of all the participants will be updated or something will fail and so all the participants will be returned to the state they were in before the transaction started. Transactions have boundaries.
In IBM MQ, a sync point defines a transactional boundary. A sync point is a logical point within a sequence of operations at which it is useful to synchronise data changes. When a transaction is committed, all work since the last sync point is committed. Conversely, when a transaction is rolled back, all work since the last sync point is rolled back.
Like many messaging providers, IBM MQ supports the JMS API which provides a convenient and ‘code portable’ way for developers to interact with the messaging layer. In the JMS specification, transactions are defined at the session level: “Each transacted session supports a single series of transactions. Each transaction groups a set of message sends and a set of message receives into an atomic unit of work. In effect, transactions organize a session’s input message stream and output message stream into series of atomic units. When a transaction commits, its atomic unit of input is acknowledged and its associated atomic unit of output is sent. If a transaction rollback is done, the transaction’s sent messages are destroyed and the session’s input is automatically recovered.” In computer science atomicity guarantees that each transaction is treated as a single “unit,” which either succeeds completely or fails completely.
The IBM MQ client libraries include support for transactions. JMS programmers control transactions at the JMS Session level by adding a simple parameter to a session or context object in their application. When ‘commit’ is called, all actions on the JMS session since the last commit call are completed.
If something goes wrong with the actions inside the transaction, everything is rolled back to the beginning of the transaction, including the state of all the participating objects or components.
With IBM MQ, we need to distinguish between local transactions and global transactions:
- Local transactions are transactions involving applications and a single resource manager – in this case a JMS provider like IBM MQ
- Global transactions are transactions involving several messaging servers, databases, or other transactional resources across a distributed topology.
Using transactions to build intelligent messaging applications
The following animated gif shows what a put and get operation looks like with one sender and one receiver connecting to the same queue. While you can handle exceptions and errors in your apps programmatically, and MQ helps you with reliability with auto-reconnect, transactions give you a more fine-grained control of what happens.
For example, when you add a put operation into a transaction (see the following figure), you have more control and further options for what to do with the message if things don’t go according to plan. For example, the fact that a message is being sent might be logged to a file. Should the output stream fail because the disk is full then the app might not call commit.
How can we use transactions to create a more sophisticated and intelligent messaging application? And, what else should you think about as a developer? The next few sections introduce you to the key concepts you need to be familiar with before you start working with transactions.
A transacted session is a JMS session with a transacted parameter and can include one or several actions that end with a commit to complete or a rollback in case something went wrong. While the actions are in progress, messages that are exchanged are not available on the queue. For example, if you’re sending three messages under a transaction, the first two messages will not be available on the queue to a consumer until the third message is put and the transaction committed. At this point, although the message is put to the queue, it is not available to other applications until the commit is called.
Without the transaction:
connection = cf.createConnection(); session = connection.createSession(false, Session.AUTO_ACKNOWLEDGE); destination = session.createQueue("queue:///DEV.QUEUE.1"); consumer = session.createConsumer(destination);
With the SESSION_TRANSACTED parameter:
connection = cf.createConnection(); session = connection.createSession(Session.SESSION_TRANSACTED); destination = session.createQueue("queue:///DEV.QUEUE.1"); consumer = session.createConsumer(destination);
A commit is a point at which a synchronization of state happens for all participants in a transaction. The JMS specification defines this as a method on the session. When you call a commit, any operations (puts or gets) will be committed. When the commit is called, a put that is under a transaction will make the message available on the queue for other applications to consume.
A rollback is the point at which, if a commit doesn’t happen, the state is rolled back to the previous point of synchronization, either since the last commit or since the start of the session (whichever is most recent). Any messages there were destined for queues will be deleted. You can easily test this by calling rollback instead of commit when putting a message.
In this first diagram, the put is under a transaction, the message gets to the queue, but it is not yet available.
When the transaction is ended with a rollback instead of a commit, the message is removed from the queue as the state is returned to the starting point before the transaction started (see the next diagram).
Dealing with the ‘poison message problem’
All this transaction stuff is great, but it leaves us with a problem: what if you can’t complete a task because of a persistent error? You don’t want to be stuck in an endless loop of trying to do something that is going to fail. Messaging developers refer to this situation as the ‘poison message problem’.
A poison message is a message that can’t be processed by the receiving application and is returned to the queue (this can be because of the wrong message format or another component in the exchange is not ready). The app will keep trying to get the message and then returning it, causing an infinite loop. To help deal with this scenario, two special queues are defined: a backout queue (a place to put messages when processing fails beyond a set threshold) and a dead letter queue (a place to put messages as a last resort). If the message cannot be put to the defined backout queue and is not able to be put to the DLQ then the message is discarded.
A backout queue is used for JMS applications and is defined for each queue. You can have a backout queue for multiple source queues. Messages can be moved to the backout queue by the MQ JMS client library classes or an application can have extra logic to do it. If the JMS classes handle the message, the backout count attribute in the message descriptor is incremented.
The backout count is an indicator as to a number of times a message has been unsuccessfully handled by an application. With IBM MQ, a backout threshold can be set on the target queue (DEV_Q) which when reached can be a signal to the application to put the message to a backout queue. The developer takes responsibility for this by determining the threshold programmatically within the application, or if you are using IBM MQ, you can set a backout threshold on the queue (for example, when a consumer can’t process the message, the MQ client moves that message to the backout queue). The backout count is an attribute of the message.
Dead letter queue
The dead letter queue (DLQ) is controlled by the queue manager. Every queue manager has one, and the queue manager is in charge of placing an undeliverable message to the DLQ . Summary and next steps
Summary and next steps
We have introduced transactions and related concepts in IBM MQ and JMS. Transactions help developers build resilient applications, provide mechanisms for controlling state and for all the components in the transaction being at the right sync point at all times, even allowing for system or component failure.
Now that you know a little bit about the components in play when it comes to transactions, you’ll want to make sure to use these resources in the most optimal way. Don’t worry, we have a set of samples to get you started. To see how local transaction with JMS and IBM MQ work in practice, have a look at our Transactions with JMS and IBM MQ sample in GitHub. You’ll be able to see how transactions work with commit and rollback, how to set up a backout threshold on the queue, how backout count is used and how to see if a transaction actually happened.