This post covers developing and deploying Ruby apps using the Message Hub Service for Bluemix and its' MQ Light API support. By going through this post, you will be able to get a sample up and running, and see worker offload in action, understand how it works, and get started with developing your own Ruby apps using the Message Hub Service for Bluemix.

If you have already gone through Dave's blog on Getting started with Ruby apps using the MQ Light Service for Bluemix, then much of of this post is going to be familiar to you; such as what the sample is meant to do and the Ruby specific content. So you can skip those bits and focus on the Message Hub Bluemix-related details, along with updated options we'll cover around debugging and application development in purely local environment vs local only applications vs all in Bluemix. Its worth noting the MQ Light service is to be retired in favor of Message Hub so this tutorial is to show how simple it is to get your MQ Light Ruby applications up and running against the replacement messaging service.

The following blog posts cover getting started with the MQ Light API with the Message Hub Service for Bluemix for the various languages:

Tip: For more information about Message Hub in Bluemix, see Getting started with Message Hub.

1. Where can I get the sample?

This sample is available on GitHub. Either use the git clone command or grab a zip copy of the latest master version.

git clone https://github.com/ibm-messaging/mqlight-fishalive-ruby

2. What is this sample about?

The sample app is actually composed of two simple apps: a web front-end that sends messages to a back-end where they are processed - capitalizing the words - and then returned to the front-end.

sample_screenshot

This sample demonstrates:

  1. how easy it is to get disjoint applications talking to each other using the MQ Light API along with the Message Hub Service
  2. how the the Message Hub Service can be used to perform worker offload, which provides benefits such as:
    • the front-end will remain responsive while the back-end is processing your data (here the actual processing is not very intensive as it is just a simple example, but imagine if your application had to process a heavy workload, such as a video!)
    • you can scale the number of instances of the back-end app to have several workers processing the requests sent from the front end and get the work done much more quickly

You can also do more advanced stuff, like having several back-end apps in different languages processing the work load from the front end, but first things first...let's see the sample in action!

3. How do I get it running on Bluemix with Message Hub?

If this is the first time that you are going to deploy an app to Bluemix, you will need to:

  1. Sign up for Bluemix if you don't have an account yet
  2. Install and setup the Bluemix command line interface by following these instructions, which will allow you to manage your Bluemix applications from your machine. (don’t forget to 'bluemix login')
  3. Create and configure a Message Hub Service instance

    Once you have a Bluemix account and have the CLI, you will have to create a Message Hub Service instance that your apps can use when they are deployed to Bluemix.

    Tip: Alternatively you can create a Message Hub Service instance via your Bluemix dashboard.

    bluemix service create MessageHub standard MQLight-sampleservice
    bluemix service key-create MQLight-sampleservice credentials-1

    That'll create you a Message Hub Service instance named MQLight-sampleservice using the standard plan with a set of randomly generated service credentials with the key credentials-1.

    Now you must create a single partition Kafka topic named MQLight before you can use the API. In the example below we will use curl to create this topic, making use of the Message Hub administration RESTful API.

    Tip: If you do not have curl or you find grabbing API keys and the like rather long winded, you can create the new topic through your Message Hub Service instance admin panel via your Bluemix dashboard.

    To create the MQLight topic from the command line, firstly we must get hold of the service specific api key required in the curl request - this forms part of the randomly generated set of credentials created above.

    bluemix service key-show MQLight-sampleservice credentials-1

    This will output the key : value pairs held within the credentials key, for example the ones we care about are:

    {
     "api_key": "7gD5BWK6y2tyCqK79M4oOVSILBRWOFMtZ3JqKT1jjvlCBcMr",
     "kafka_admin_url": "https://kafka-admin-prod01.messagehub.services.us-south.bluemix.net:443",
    }

    Copy the api_key and kafka_admin_url values into the following curl request to enable the MQLight topic. Replace the values in the example with your own service values.

    curl -k -v -H 'Content-Type: application/json' -H 'Accept: */*' \
        -H 'X-Auth-Token: 7gD5BWK6y2tyCqK79M4oOVSILBRWOFMtZ3JqKT1jjvlCBcMr' \
        -d '{ "name": "MQLight", "partitions": 1 }' \
        https://kafka-admin-prod01.messagehub.services.us-south.bluemix.net:443/admin/topics
    
  4. Deploying the sample to Bluemix

    Now that you've created the service and enabled the MQ Light API support either from the command line or via the dashboard, you're ready to push the apps themselves, so navigate to the root directory where you downloaded the apps, and run:

    bluemix cf push

    First, this uploads the app files themselves, then sorts out all of the setup for you, taking information from the Gemfile to download the Ruby dependencies. This command also binds the service you created to both the apps which were pushed, and starts them up.

    Tip: To do all this, this command uses the manifest.yml that is in the root directory to facilitate the deployment. This file is optional, but avoids having to specify all of the arguments each time you run the command (another tip: have a look at bluemix cf push -h if you want to specify these manually)

    When it's done, you can run the bluemix list apps command to see the apps running and the bluemix list services command to see they are successfully bound to the Message Hub Service.

    $ bluemix list apps
    Getting CF applications in org 'ORG' / space 'SPACE' as email...
    Name                          Type             State     Instances   Memory   Disk   URLs   
    mql.fishalive.ruby.backend    CF Application   STARTED   2/2         128M     1G        
    mql.fishalive.ruby.frontend   CF Application   STARTED   1/1         128M     1G     mqlight-fishalive-ruby-barwise-cotwal.mybluemix.net   
    
    $ bluemix list services
    Getting services in org 'ORG' / space 'SPACE' as email...
    Name                    Type               Service      Plan       Bound CF Apps                                             Last Operation   
    MQLight-sampleservice   Service Instance   messagehub   standard   mql.fishalive.ruby.backend, mql.fishalive.ruby.frontend   create succeeded   
    

If everything went well, you will see the front-end, as presented in the screen shot in the Sample app section. When you click the 'Submit work' button, the app publishes messages to the mqlight/sample/words topic, which are received and processed by the back end workers (we deployed two of them when we used bluemix app push), and returned capitalized.

messages-recieved

Now that the app has been successfully deployed to Bluemix...how does it actually work? Have a look at the following section if you are interested.

What's under the covers?

Looking at frontend/sinatra_frontend.rb, you can get an idea of what is going on. Both frontend/sinatra_frontend.rb and backend/sinatra_backend.rb follow a similar style. Early on, the app checks to see whether its running in Bluemix by checking for the presence of environment variable VCAP_SERVICES. If its found, the app then checks for the presence of either the legacy MQ Light service name or the Message Hub Service name. The app then grabs the necessary credentials and connection details from Bluemix for the service type in question. If VCAP_SERVICES is not found, the app assumes its running locally against a local instance of the MQ Light Developer Tools.

Tip: The Message Hub specific VCAP_SERVICES lookup block below is the only difference you'll need to worry about if migrating your apps from the old MQ Light Service for Bluemix. The sample has been update to handle being run against either service type.

  if ENV['VCAP_SERVICES']
    vcap_services = JSON.parse(ENV['VCAP_SERVICES'])
    for service in vcap_services.keys
      if service.start_with?(mqlight_service_name)
        mqlight_service = vcap_services[service][0]
        uri = mqlight_service['credentials']['nonTLSConnectionLookupURI']
        opts[:user] = mqlight_service['credentials']['username']
        opts[:password] = mqlight_service['credentials']['password']
      elsif service.start_with?(messagehub_service_name)
        messagehub_service = vcap_services[service][0]
        uri = messagehub_service['credentials']['mqlight_lookup_url']
        opts[:user] = messagehub_service['credentials']['user']
        opts[:password] = messagehub_service['credentials']['password']
      end
    end
  else
    uri = 'amqp://127.0.0.1:5672'
  end

You can then start the MQ Light client using these options:

 set :client, Mqlight::BlockingClient.new(uri, opts)

After creating the MQ Light client instance, the frontend app subscribes on SUBSCRIBE_TOPIC = 'mqlight/sample/wordsuppercase', the backend app is similar however it subscribes on SUBSCRIBE_TOPIC = 'mqlight/sample/words' with a SHARE_ID = 'fishalive-workers'. The share id allows multiple backend apps to be spun up and process messages sent from the frontend app as a single set rather than each getting a copy and causing duplication.

When the frontend app has successfully started its MQ Light client instance it can then process data from its web page form. It does this by splitting the input into single words and sending each in turn to PUBLISH_TOPIC = 'mqlight/sample/words' where the backend app is subscribed waiting for messages.

settings.client.subscribe(SUBSCRIBE_TOPIC, share: SHARE_ID)

The processMessage function for the backend app uppercases the message received from the frontend app and sends it back via PUBLISH_TOPIC = 'mqlight/sample/wordsuppercase'.

def process_message(data)
  word = JSON.parse(data)['word']
  ...  
  # upper case it and publish a notification
  reply_data = {
    word: word.upcase,
    backend: "Ruby: #{settings.client.id}"
  }
  send_reply(JSON.unparse(reply_data))
end

def send_reply(message)
  puts "Sending response: #{message}"
  settings.client.send(PUBLISH_TOPIC, message)
end

The frontend app adds the messages received from the backend app(s) to the list of heldMessages to display on the frontend's web page.

loop do
  delivery = settings.client.receive(SUBSCRIBE_TOPIC, timeout: 1000)
  if delivery
    puts "Received delivery: #{delivery}"
    data = JSON.parse(delivery.data)
    settings.recv_queue.push(data: data, delivery: delivery)
  end

Now that you understand how this sample works, are you ready to develop your own Ruby apps? The next section gives a few tips about how to do that.

5. Developing your own Ruby apps for Bluemix

When writing your own app (which can be based on this sample if it helps!), you have a number of options:

  1. Run entirely in Bluemix:

    Modify the source code and push your apps to Bluemix to test them (this is what you've done above already)

  2. Run apps locally and connect to Bluemix service:

    Using the connection details and credentials generated by your Message Hub Service, run you app locally and remotely connect to your service instance in Bluemix.

    Rather than continuously push and repush your application as you work out the kinks, is it possible to connect MQ Light applications to a Message Hub Service instance from outside of Bluemix, i.e. running in your local development environment. This allows you to test out your apps more efficiently (easily connect debuggers and the like), and once you feel that your app is ready, then get it deployed onto Bluemix.

    Tip: You can mix and match app deployment options and run apps you've proven to be stable within Bluemix while running others still in development locally.

    This is pretty simple to do by locally populating the VCAP_SERVICES system environment variable the application uses with the details provided in Bluemix for your service instance. As you did when creating your MQLight topic using curl, get hold of the service's credentials.

    $ bluemix service key-show MQLight-sampleservice credentials-1

    This will output the key : value pairs held within the credentials key, for example the ones we care about are:

    "mqlight_lookup_url": "https://mqlight-lookup-prod01.messagehub.services.us-south.bluemix.net/Lookup?serviceId=18e08058-9012-4cfb-b109-81b7f8c42f7a",
    "password": "9M4oOVSILBrWOFMtZ3JqKT1jjvlCBcMr",
    "user": "7gD5BWK5y2tyCqK7"
    

    Define the VCAP_SERVICES and VCAP_APPLICATION environment variables, replacing the values in the example with your own service values.

    export VCAP_SERVICES='{
       "messagehub": [
          {
             "name": "MQLight-sampleservice",
             "label": "messagehub",
             "credentials": {
                "mqlight_lookup_url": "https://mqlight-lookup-prod01.messagehub.services.us-south.bluemix.net/Lookup?serviceId=18e08058-9012-4cfb-b109-81b7f8c42f7a",
                "user": "7gD5BWK5y2tyCqK7",
                "password": "9M4oOVSILBrWOFMtZ3JqKT1jjvlCBcMr"
             }
          }
       ]
    }'
    
    export VCAP_APPLICATION=true

    Tip: Setting the VCAP_APPLICATION environment variable is unique to the MQ Light Ruby client but is required to hoodwink it into thinking its running in a Bluemix-like environment.

    Now you have your environment setup to allow your app to connect to your Bluemix service instance, from each of the directories (frontend and backend), install the Ruby gem dependencies, as specified in the Gemfile:

    bundle install

    Tip: This installs the dependencies you need. When you push your app to Bluemix however, Bluemix will get the dependencies for you based on the Gemfile.

    When everything has finished installing, use the following to start each app:

    ruby sinatra_frontend.rb
    
    ruby sinatra_backend.rb

    Point your browser to localhost:3000 to get to the front end deployed on your local machine.

  3. Run entirely local:

    By installing the the MQ Light Developer Tools onto your own machine you can use this as a development environment for purely local application development.

    This option builds on the benefits of running local apps and comes with the added benefit of the MQ Light Developer Tools UI which will help you understand where your messages are going (or not going!) in the event of something going awry in your applications during app development - this is covered in more detail in the troubleshooting section below. You can also continue to develop and test your applications offline, if you do find yourself in a location where you do have connectivity to use Bluemix at any time.

    Follow the MQ Light Getting started guide and ensure that you have MQ Light and Node.js set up on your machine as described. When you have completed this process, you should have a web browser open and pointing at the MQ Light Developer Tools Web UI as this opens automatically when starting the MQ Light Developer Tools.

    If your app follows the same connection details logic as the sample, make sure the VCAP_SERVICES and VCAP_APPLICATION environment variables are not defined and your application should connect to the local MQ Light Developer Tools rather than attempt to connect to Bluemix. As with option 2 running with local apps to a Bluemix Service instance (above), pull down the Ruby prereqs using bundle install and once you feel that your app is ready, then get it deployed onto Bluemix!

6. Troubleshooting your own MQ Light Ruby apps

To troubleshoot your Ruby apps there are two useful sources of information: the MQ Light Developer Tools UI, available in your local MQ Light Developer Tools installation; and for the Message Hub Service for Bluemix, the logs from your Bluemix apps.

The MQ Light Developer Tools UI

To access the MQ Light Developer Tools UI, the UI should have been fired up when you started MQ Light, but if you closed it, point your browser to localhost:9180

You will be able to see senders appearing on the left-hand side, and receivers on the right-hand side, with any messages flowing in the middle.

For example, going back to the sample, after clicking on the 'Submit work' button, you should see something similar to the screenshot below. You can click on 'Topic List' for senders and on 'Details' for receivers to get more information about the connected applications. Click on 'Details' in any of the messages to get some more information on it, such as the payload, who sent it, to where, and whether or not anyone is waiting for it. This is perfect for troubleshooting messaging applications as you can see what your messages are actually doing.

Looking at Bluemix logs

To see the logs generated by your app on Bluemix, you can use the bluemix app logs command:

  • To tail your logs as they are being generated (from a separate command prompt while pushing for example)
  • To see your app's most recent logs using the --recent option

Tip: Only the logs following the latest app startup will be shown.

Tip: If you have several instances of an app running, use the drop down box next to 'Instances' (under 'Files and logs') to see the logs from each of them

7. Working with other MQ Light client languages

If you have already gone through the Node and/or Java blogs, then you will already have MQ Light back-end apps hooked into the same instance of the MQ Light service. If those are stopped, and you have only your Ruby apps started, you should see the following when you click the button:
just ruby

As you can see, it is only the Ruby back-end that is doing the work. If you start the back end apps you created in the other posts and click the front end's 'submit work' button, you should then see something like:
all

This time, each message sent by the front end is processed by all active back end apps. If other front ends are running you should see the same thing in their views. And just like that, you've got apps in different languages talking to each other.

Each message sent from the front end is delivered to a shared destination that all back end apps are subscribed to, as such these messages will only be delivered to one of the back end apps (to avoid them duplicating work), so in this way, we distributed workload among multiple workers, written in different languages! This pattern is great for offloading work and allows independent workers to tackle the same workload without stepping on each-other's toes.

If you didn't use shared destinations, each message would be received by each client subscribed to the topic in question. This might be desirable in some cases; perhaps a Node app could take messages and display them in a responsive web app, while a Ruby app stores the same messages in a database or does analytics without affecting the responsiveness of your front end at all.

Join The Discussion

Your email address will not be published.