One of the new features in MQ Advanced 9.0.4 is the support for a new type of queue manager called a Replicated Data Queue Manager, or RDQM for short. As its name suggests, this type of queue manager replicates its data (rather than relying on remote shared storage) and this capability is used in 9.0.4 to provide a new approach to High Availability in MQ on Linux.
You may see this referred to as “Easy HA” which is a fitting name as we have tried to make it as easy to use as we can.
RDQM relies on the concept of an HA Group which is a collection of three nodes that host RDQM instances and that replicate the data for each RDQM from the Active node where an RDQM is running to the other two Passive nodes. Any given RDQM only runs on one node at any time but an HA Group can contain multiple RDQMs so it is possible to have RDQM1 active on node1, RDQM2 active on node2 and RDQM3 active on node3 for example. Each node should be sized so that it can run all of the RDQMs if that ever becomes necessary.
The use of three nodes means that if one node becomes isolated it detects that it is isolated and stops running any RDQM instances it was running, assuming the remaining two nodes can still form a quorum and start the RDQM instances that were running on the isolated node. Having three nodes and three copies of the data for the RDQM means that if one node fails or loses contact with the remaining two nodes, the RDQM will run on one of the remaining nodes and there will still be a second copy of the RDQM data.
Most of the High Availability approaches for MQ rely on a remote server providing shared data via some cluster filesystem which has to support the locking and sharing semantics that an MQ multi-instance queue manager requires. These approaches have a performance impact, both on the normal operation of MQ and on the restart of a queue manager following a problem.
It requires a large amount of effort and investment to provide a shared storage solution without a single point of failure in either the storage or the network, and with the desired level of performance. It is much simpler to give each of three nodes its own dedicated storage, ideally local, and let the nodes replicate the data between themselves.
RDQM also has a performance impact but there are some significant differences:
- The nodes in an RDQM HA Group are likely to be closer to each other than a remote file server is so the network latency is likely to be lower and the bandwidth higher
- A normal filesystem, ext4, is used rather than a cluster file system
- When data needs to be read, it is always read from a local/dedicated disk rather than reading from the remote shared disk
The main system requirements are:
- Red Hat Enterprise Linux 7.2 or 7.3 x86_64
- A volume group for the storage for RDQMs
- One, two or three IP addresses per node (the same number for each node)
Highlights of RDQM
We supply a script which installs the RDQM feature of MQ Advanced along with the necessary dependencies. The script can be edited to install additional MQ components if desired.
Configuring an HA Group can be as easy as creating a configuration file with the IP address information for each node and then running a single command.
Creating an RDQM can be as easy as supplying one new option to the MQ crtmqm command
Floating IP Address
An RDQM can optionally have a floating IP address associated with it which moves with the RDQM if it has to run on another node. This allows a client or another queue manager to be configured with a single IP address for the RDQM.
The official MQ Knowledge Center will be updated soon, if it has not already been.
Look out for more blog posts, videos etc. to be posted here.
I have created a GitHub community mq-rdqm which over time will have hints and tips, step-by-step guides etc.