Build and deploy an IBM MQ app to AWS Cloud

The IBM MQ messaging playground application allows developers to experience and explore common messaging patterns. The application design and deployment steps have been optimized for AWS to simplify the user tasks required to get up and running.

The application is structured around three components:

• A front-end web application that enables users to interact with messaging objects
• A back-end application that processes instructions from the front-end web app and performs messaging tasks
• A queue manager that hosts the messaging objects and stores the messages that are exchanged

This messaging playground application allows you to create your own instances of messaging producers and messaging consumers, interact with them, and explore how IBM MQ on AWS underpins the exchange of messages between different endpoints.

The playground provides a visual, hands-on experience of three messaging patterns:

• Point-to-point
• Publish/subscribe
• Request-response

The messaging playground application is based on the ticket reseller sample application that was introduced in the IBM MQ Developer Essentials learning path.

In this tutorial, you will learn how to build more sophisticated solutions with these patterns to handle a ticket reseller and management system for a theater event. The application components are available on our MQ Samples and Patterns GitHub under an Apache 2.0 license.

Prerequisites

• To familiarize yourself with the sample application, review the IBM MQ messaging app coding challenge in the IBM MQ Developer Essentials learning path.
• Docker and Docker Compose, version 2.0 or higher.
• To deploy the application in this tutorial to the AWS Cloud, you need to have completed the steps in the tutorial, “Get an IBM MQ queue for development in AWS Cloud.”

Messaging patterns

In IBM MQ Developer Essentials learning path, you learned about the fundamental messaging components (messages, queues, channels, and so on) and the basics of messaging patterns. Before we deploy the messaging playground application, which is packaged as a containerized application that is deployed on AWS, you need to familiarize yourself with the messaging components that make up the application and how they interact with the different messaging patterns (which are also known as message exchange patterns).

Point-to-point messaging

Point-to-point messaging is the simplest form of messaging in IBM MQ. Using this messaging pattern, loosely coupled applications can communicate without the need to know anything about each other. The only component that they have in common is the queue that they use to exchange messages.

Using point-to-point communication, one or more producers can send new messages to a specific queue without knowing which application will consume them. In the same way, consumers can receive messages by connecting to the queue where the messages are stored, without the need to have any information about the sender.

Let’s see how the messaging playground application can be used to explore point-to-point messaging for the ticket reseller and management system sample application.

In this section, you will learn how to:

• Create connections
• Put messages into a queue
• Consume messages
• Add new producers, consumers, or queues into the workspace

Create Connections

You can create connections by connecting a message producer (in the messaging playground app, the Tickets Generator) and a queue (for example, DEV.QUEUE.1). A single producer or consumer can be connected to a single queue, but an individual queue can support connections from multiple concurrent producers and consumers. A connection can be closed by clicking the x icon in the middle of the connecting line that joins them.

Figure 1: an example of a connection between a producer and a queue.

Putting messages into a queue

After your producer is connected to a queue, you are ready to send messages. You can set the number of messages you want to send by modifying the value of the Tickets created input field. Clicking the Create Bookings button will start the producer and send the batch of messages (in this case, “bookings”) to the queue that it is connected to. The number on the queue is shown as the number of requests on the queue object along with the percentage full (based on the queue’s max depth). This value increases when new messages are put by the producer and decreases when consumers get them.

Figure 2: An example of a producer putting messages into a connected queue.

Consume messages

Connections between consumers and queues can be created in the same way as producers. By default, the consumer application components are in an inactive state, which means they are not actively getting messages. This state can be changed by clicking the switch icon to the On position. After the switch is on, messages can be seen flowing from the queue to the consuming application. The payload of each message that is consumed is displayed in the Type field. The date and time that the message was created are displayed along with a cumulative total of messages that have been consumed by the receiving application.

Figure 3: An example of a consumer consuming messages from a queue.

Add new producers, consumers, or queues

You might want to explore more sophisticated producer-consumer patterns. You can add producer, consumer, and queue components from the sidebar by dragging them to the workspace.

The messaging playground app is deployed using containers with limited resources. To avoid overwhelming the resources available, we have limited the maximum number of concurrent queue objects within the application to three. If you have small containers, the load on the container resources should be limited. To limit the load, avoid sending messages with high frequency (one message per second).

Figure 4: An example of adding new UI components and creating new applications.

Publish/subscribe messaging

Publish/subscribe messaging is a widely adopted messaging pattern, and it is also implemented in IBM MQ. Unlike point-to-point messaging where messages are exchanged between a producer-consumer pair, in publish/subscribe messaging, a message that is published by a producer to a topic will be delivered to all subscribers of that topic.

A typical publish/subscribe messaging system likely has multiple publishers and subscribers interconnected over a network of different topics, often referred to as a topic tree. A topic tree can span the domain of more than one queue manager. In this messaging pattern, an application can be both a publisher and a subscriber to a range of topics.

The basic concepts of the publish/subscribe messaging pattern are:

• Topics are objects and they have properties.
• Messages are published to a topic by a publisher.
• Each publication is to a single topic. Subscribers register an interest in, or subscribe to, a topic.
• When a publisher publishes a message to a topic, all subscribers for that topic receive a copy of the message.
• For a message to be delivered to a subscriber, both the publisher and the subscriber must match the same topic. The subscribers will only receive messages that are produced from the time they made a subscription to the topic.

In this section you will learn how the messaging playground application can be used to explore publish/subscribe messaging and how to use the messaging playground application to:

• Create a topic
• Publish messages to a topic
• Create subscriptions
• Exchange messages
• Add new publishers, topics, or subscribers to the workspace

Create topics

To add topic objects to the playground and to create IBM MQ topics, you can drag a new topic component onto the workspace. A topic value must be assigned using the input text field before it can be made available for publishers and subscribers to connect to.

In this application topic, trees cannot include special characters, except for the # wildcard character, which can only be placed at the end of the topic. Direct publications to wildcard topics are not allowed. Wildcard topics will receive all the messages that are published to topics with the same tree. For example, a subscriber to topics/# will receive messages that are published to topics/classics and topics/VIP. However, the same subscriber would not receive publications to bookings/VIP as the parent tree topics does not match.

Figure 5: How to create topics

Subscribers are connected to topics by using the orange interface on the right side, while publishers are connected to topics by using the blue interface on the left side. Topic components can be deleted by clicking the x icon.

Publish a message to a topic

Publishers in the messaging playground app are represented by notification components. In this booking sample scenario, event organizers (publishers) need a notification system to send messages to all customers (subscribers). Each message can include a title and content, both of which form the payload of the message that is sent to all subscribers to that topic. After your message is ready, you can connect the publisher with the corresponding topic by drawing a connecting line between them. The message is published by clicking the Send Notification button. Optionally, the message can be set to publish automatically every five seconds by clicking the Notify every 5 seconds switch.

Figure 6: An example of a publisher application that is connected to a topic.

Create subscriptions

After you configure the topics in the messaging playground app, you can start adding subscribers by dragging subscriber components from the sidebar. Subscribers are connected to topics by drawing a line between the subscriber component and the topic component. The playground indicates the success or failure of the subscribe action by displaying a notification on the lower right of the browser window. You can rename the subscriber by overwriting the text on the upper left of the subscriber component. After subscribing to a topic, your subscriber applications can now receive messages that are published to that topic. The payload of incoming messages is displayed on the subscriber component under Last Notification received.

Figure 7: An example of a subscriber subscribing to a topic.

Exchange messages between publishers and subscribers

After both your publishers and subscribers are connected to a topic, you can start publishing. The banner that appears on the lower right of the browser window indicates whether the message has been published successfully. If the message has been correctly consumed by the subscribers, the title and the content of the message you sent will be displayed and the payload of the message sent is displayed by the subscriber component. If multiple subscriptions are made to the same topic, either explicitly or indirectly using a wildcard, then a copy of the message is delivered to each subscriber.

Figure 8: An example of a publisher publishing message to a topic, and a subscriber receiving messages from a topic.

Add new publishers, subscribers, or topics

You can add more publisher, subscriber, and topic components by dragging the required component from the sidebar. You can reset the workspace to the default example at any time by clicking on the Reset button.

Because this playground is a set of applications running on tiny containers, you should avoid adding more than 10 subscribers to reduce the stress on the containers.

Figure 9: Adding new UI components (topics, subscribers, publishers).

Request-response messaging

Request-response (sometimes referred to as request-reply) is a conversational messaging pattern where the application that sends a message requires a response from the receiving application. The request-response messaging pattern is often built on top of the point-to-point messaging pattern and can be a synchronous operation.

Typically, the sending application will wait for a set amount of time for the response message before entering a timeout routine. The request and response applications typically employ strategies to handle error conditions such as timeouts or temporary queue resources not being available. A requesting application might set a time-to-live value on the request message so that the message is deleted by the queue manager if it is not processed by the response application within a specified timeframe.

The sending application sets a reply-to-destination and a correlation-id so that the response message can be routed and matched to the originating request message by the requesting application.

In this section you will explore request-response messaging in the playground application and learn how to:

• Send messages and create a temporary queue
• Receive and reply to request messages

Send messages and create a temporary queue

To send messages and create a temporary queue, connect the request component (in the messaging playground app, the Participants checker) to a pre-defined destination queue (for example, DEV.QUEUE.3). Then, to submit a new request, click the Submit Request button. You can optionally decide the number of requests to send to your response application by adjusting the value of the Quantity Request input field. If the request has successfully sent the message to the request queue, a success notification will appear at the bottom of the browser window.

Figure 10: An example of a requester creating a temporary queue and sending a message to a responder.

When the request message is sent, a temporary queue is created. This queue is where the requesting application will expect the response application to reply and is often referred to as the “reply-to” queue. It is the responsibility of the requesting application to share the reply-to queue information with the response application.

Receive and reply to request messages

The response component will consume a new message from the requester when the switch on the right side of the Responder component is clicked. If a message is received correctly, a notification will appear at the bottom of the browser window, and the payload of the message is displayed by the response component.

The message sent from the requester contains two values: the payload itself and a property that contains the reply-to queue name. The reply-to queue is a temporary queue (in our example, APP.REPLIES.638...2001). After a new request message is received, you can insert your response into the textbox and reply to it by clicking the Reply button. After the reply message is received back to the requester, the reply-to-queue (which is a temporary queue) is deleted because the conversation is completed.

Figure 11: Responder receiving a message from a requester and replying to a temporary queue.

Now we have covered the basics, let’s set up the messaging playground application and get hands-on with the app!

Run the messaging playground application

In this tutorial, you learn how to deploy the messaging playground application on AWS ECS. This sample application helps expand your knowledge of messaging patterns. These messaging patterns will be essential for you to take the ticket reseller and management system to the next level in the next coding challenge later in this learning path.

Setting AWS workspace

Before obtaining the code for deploying and running the messaging playground application, you need to configure the AWS environment that will host the three containers.

In the previous tutorial in this learning path, you got an IBM MQ queue running in AWS Cloud. Now, you need to configure the AWS environment to host the three containers for running the messaging playground app.

All you need to set up the AWS environment is a command line with AWS CLI, and Docker Compose installed.

Create an ECR registry

The first thing to do is creating a new AWS ECR registry instance where the containers will reside in AWS. To do so, open your terminal and type the following command:

aws ecr create-repository --repository-name mq-app-cluster

You should see a response similar to:

{
    "repository": {
        "repositoryArn": "arn:aws:ecr:eu-west-2:600647096926:repository/mq-app-cluster",
        "registryId": "600647096926",
        "repositoryName": "mq-app-cluster",
        "repositoryUri": "600647096926.dkr.ecr.eu-west-2.amazonaws.com/mq-app-cluster",
        "createdAt": "2022-11-24T10:47:18+00:00",
        "imageTagMutability": "MUTABLE",
        "imageScanningConfiguration": {
            "scanOnPush": false
        },
        "encryptionConfiguration": {
            "encryptionType": "AES256"
        }
    }
}

Make note of the Repository URI, which in our case is "repositoryUri": "600647096926.dkr.ecr.eu-west-2.amazonaws.com/mq-app-cluster", where 600647096926.dkr.ecr.eu-west-2.amazonaws.com is the ECS Registry URI and mq-app-cluster is the cluster name.

You use these values for configuring the .env file as we will explain in later steps. We will authenticate by using the ECS Registry URI.

Run the following command to retrieve an authentication token and authenticate your Docker client:

aws ecr get-login-password --region eu-west-2 | docker login --username AWS --password-stdin 600647096926.dkr.ecr.eu-west-2.amazonaws.com

Important: Your ECR cluster and region will be different!

Obtain the messaging playground application code and set it up

The playground app is available as an open source project in our MQ Samples and Patterns GitHub repository.

To clone the messaging playground sample app code, open your terminal and run:

git clone https://github.com/ibm-messaging/mq-dev-patterns.git

After you download the code from GitHub, you must configure it before you deploy the playground sample app. You need to configure:

• Your own ADMIN user for the MQ manager
• Your own USER password for the MQ manager
• The AWS Image ECS Registry URI
• The AWS cluster name

These values must be configured before you deploy the messaging playground application by setting the variables “APP_PASSWORD”, “ADMIN_PASSWORD”, “IMAGE_ECS_REGISTRY_URI”, and “CLUSTER_NAME” that are located in the ~/mq-dev-patterns/ibm-messaging-mq-cloud-showcase-app/.env file.

Edit the ~/mq-dev-patterns/ibm-messaging-mq-cloud-showcase-app/.env file with your favorite editor.

• Set the APP_PASSWORD value with your own password: APP_PASSWORD=exampleAppPassword
• Set the ADMIN_PASSWORD variable with your own password (which will be different form the app_password): ADMIN_PASSWORD=exampleAdminPassword
• Set the IMAGE_ECS_REGISTRY_URI and the CLUSTER_NAME. You obtained these field values when you created the registry and the cluster as explained in the previous step.

After the .env file is configured, you are now ready to build the two containers (front end and back end) that will be pushed to the AWS registry. Run this command:

docker compose -f aws-docker-compose.yaml build

Then, to push to the ECR registry run this command:

docker compose -f aws-docker-compose.yaml push

Start the messaging playground application

For compose to be able to interact with AWS services, a context needs to be created that will allow Docker Compose to make use of AWS credentials.

From one of the terminals, run this command:

docker context create ecs mq-app-server

Next, you need to initialize the external storage that the queue manager will use for persistence. Run this command:

docker --context=mq-app-server compose -f aws-mq-init-compose.yaml -p mqonaws up

Start the messaging playground application by running this command:

docker --context=mq-app-server compose -f aws-docker-compose.yaml -p mqonaws up

This task can take up to 2 minutes to complete. It creates the 3 containers and initializes the services that are needed for running the messaging playground application.

To determine the address of the load balancer, run this command:

docker --context=mq-app-server compose -p mqonaws ps

Under the Ports column, you'll see the URLs that you need to be able to access the IBM MQ web console (port 9443) and the application (port 3000) via your local browser. An example is provided in the following screen capture.

After the three containers are running and the front-end web app has been compiled, you can run the messaging playground application by pasting this URL on your browser: http://<loadBalancerURL>:3000. The loadBalancerURL is the same address you used for accessing the IBM MQ Console.

Configure model queues for the messaging playground app

We need to create a model queue that will used by the queue manager to create temporary queues.

Log in to the IBM MQ Console by typing https://<loadBalancerURL>:9443/ibmmq/console on your local browser. The default Username is Admin and the password is the same as the one you configured for the ADMIN_PASSWORD variable in the .env file.

From the main Console page, click the Create a queue button.

Click the Model button to configure a new model queue.

Enter this name for the model queue DEV.APP.MODEL.QUEUE in the Queue name text input field, and then click Create. All the other fields can be left with their default values.

After you create the model queue, a green success message should appear on the Manage page, and your queues’ configuration should look like something like this:

You need to set some security permissions for allowing the messaging playground application to access the model queue. For the DEV.APP.MODEL.QUEUE, click the three dots in the last column on the right to open the configuration of the queue model.

Click the Security tab. Then, click the second row (configuration for the app user), and a sidebar is displayed that shows the current security permissions for the app user.

In the sidebar, click Edit, check the checkbox for the Display permission under the Admin access section, and then click the Save button.

The queue model is now configured with the right security permissions.

Troubleshooting

The environment and the communication between the three containers are fully managed by Docker. This means that you do not have to worry about the different versions of the OSs, packages, frameworks, and APIs that are used by the messaging playground application because the containers’ environments are fully set up and linked together by Docker.

Sometimes, after running the messaging playground application, errors can occur due to incorrect configuration of the three components (front-end web app, back end web server, and queue manager). Don’t panic! In this section, we’ll explore some common configuration problems, how to diagnose them, and how to get back on track.

The messaging playground application displays notifications on the bottom right of the browser window and will notify you about the status of these three containers.

If you observe connectivity errors (a component is not reachable) you should check that all the required variables in the .env file (~/mq-dev-patterns/ibm-messaging-mq-cloud-showcase-app/.env) are set with the right values as explained in the previous steps.

The containers might exit from their running status. This is not a problem because AWS ECS will restart them automatically.

Optional: Tear down the AWS services

When you are done working with the messaging playground application, you should shut down the AWS services to avoid unnecessary costs.

Remove the MQ container and all AWS infrastructure, except the EFS storage, by running the command.

docker --context=mq-server compose -p mqonaws down

If you restart the container, within the storage retention period, then the persisted MQ state will be restored.

When you no longer need it, you can remove the EFS Storage. You can list which volumes you have by running the following command:

docker --context=mq-server volume ls

Using the returned arn identifier remove the volume by running the following command:

docker --context=mq-server volume rm <full_hard_identifier_from_ls>

If you are using an existing Amazon ECS cluster or load balancer, then they will not have been removed by the compose down process.

Summary

You have successfully deployed an MQ application to AWS and explored common messaging patterns using IBM MQ and the messaging playground application.