IBM MQ Uniform Clusters Cheat Sheet

So you’ve set up a uniform cluster and connected some applications to it – that’s great! But now you’re seeing some strange behavior.

Are your applications refusing to get involved with rebalancing, or even leaving the uniform cluster altogether? Let’s explore some causes for these and other problems, so you can unleash the full potential of your uniform cluster.

My consuming application keeps disconnecting from the uniform cluster

This might be because there are not enough messages for them to consume, causing their "get" operations to time out and the applications to disconnect.

There are a couple of potential causes for this:

• You might have too few producers compared to the number of consumers connected to your uniform cluster, so there aren’t enough messages on the message queues for the consuming applications.

• If there is a good balance between the number of producing and consuming applications, it might be that the producing applications are not putting messages successfully and you need to debug within the application. To debug your producers and consumers, check out the IBM MQ development cheat sheet and the IBM MQ best practices articles.

• Producing and consuming messages might not be being rebalanced correctly across the uniform cluster. Read on to learn more.

My applications won’t rebalance across the uniform cluster

There might be a few reasons for this:

• Your applications don’t meet the essential criteria as described at the end of the “Setting up a uniform cluster” tutorial.

• Your applications might be in any of the states outlined in the table in the MQ docs, which means they are not eligible for rebalancing.

My applications are all connecting to one particular queue manager

Initially, you are likely to see all of your applications connect to a single queue manager, but this should change as the uniform cluster balances itself. If this is still the case after some time, it could be that the QMGR attribute specified by the application is for a particular queue manager and not the group of all queue managers in the uniform cluster. Check the CCDT that your application is using and ensure that the QMGR attribute in the application is set to the queue manager group name, prefixed by a *. For example, in our article on CCDTs, the queue manager group name was ANY_QM, so QMGR in the application was set to *ANY_QM.

I'm having trouble running the Uniform Clusters demo

If you are using Podman to run one of the demos found here, you might see this error when you initially try to create the uniform cluster:

Error: statfs /User/mq-uniform-clusters/demo/M1MacDocker/QMConfig/dockerVolume/UniCluster.mqsc: no such file or directory

Simply run podman machine stop followed by podman machine start to reboot your Podman machine. Attempting to create the uniform cluster again should then work as expected.

Running multiple JMS application instances with the multi-jms-sample-driver.sh script

When you connect your applications to your uniform cluster using the multi-jms-sample-driver.sh script as described in this tutorial, you might have come across some problems, which we’ll look at how to address these errors now.

“ERROR: MQCCDTURL not set”

Before running the multi-jms-sample-driver.sh script, execute this command to export the location of your CCDT:

export MQCCDTURL=file:///<your_CCDT_file>

“WARNING log file (logx).txt already exists. Please delete and run script again”

When the multi-jms-sample-driver.sh script runs, it will write to a log file for each JMS instance it is running. You might have existing logs that are named in the same way as the logs that are generated by this script (logx.txt) that you want to keep, so by default the script won’t overwrite them. If you always want to have your logs overwritten by this script, you can change the overwriteLogs variable in the script to true. Otherwise, manually move or rename your logs and run the script again.

The script doesn’t terminate after pressing Ctrl-C

You will find that when you first press Ctrl+C (or Command+C), the multi-jms-sample-driver.sh script doesn’t terminate. This is because pressing Ctrl+C once stops the logs from being printed to the console, but it doesn’t stop the script. Pressing Ctrl+C a second time should stop the script.

Your machine cannot handle running that many JMS instances at once

You can choose how many instances of the application are run either in the multi-jms-sample-driver.sh script by changing the numInstances variable, or by specifying the number when you run the script as follows:

./multi-jms-sample-driver.sh <jms_app_name> <number_of_instances>

Uniform Cluster Best Practices

We’ve now covered some common problems you might encounter and their solutions. Now, to ensure that your uniform cluster behaves optimally, here are some operational best practices to consider:

• Ideally, uniform clusters should not consist of too many queue managers - we recommend no greater than 10.
• Each of these queue managers should have near-identical configuration (as explained in this introductory article).
• The uniform cluster is typically scoped for use by a single application instance group, or groups of related application instances.
• The number of application instances connecting to the cluster should be greater than or equal to the number of queue managers to ensure there are no idle resources.
• There should be a good balance between the putting and getting application instances that are connected to the queue manager.

Summary and next steps

This article summarized the tips and tricks for debugging uniform clusters in IBM MQ, and looked at the best practices to consider when using a uniform cluster.

When you connect your application to a uniform cluster, it will be reconnected more often as part of automatic rebalancing. The ways in which your app needs to prepare for this rebalancing will depend on the type of messaging it will carry out in the uniform cluster. Have a look at this doc which describes the different scenarios and how to prepare your app to handle them.