IBM Developer Advocacy

Introducing OpenWhisk – Microservices Made Easy



Glynn Bird
4/27/16

In earlier blog posts I have described a microservice architecture that uses a queue, pubsub channel, or message hub to broker a list of “work”. Each item of work is a block of data—typically a JSON document—that is to be processed, saved, or acted upon in some way. I created the Metrics Collector Microservice, which collects web metrics data from mobile or web apps and writes it to a queue or pubsub channel using Redis, RabbitMQ, or Apache Kafka. A separate Metrics Collector Storage Microservice consumes the work and writes it to a choice of Cloudant, MongoDB, or ElasticSearch. I then described how other microservices could be added to aggregate the streaming data as it arrived. Fortunately Compose.io allows the deployment of Redis, RabbiMQ, MongoDB, or ElasticSearch and IBM offers Cloudant and Apache Kafka as services, so it’s very easy to get started but there are a lot of moving parts.

Today I’ll be using a new service, OpenWhisk, which makes it simple to deploy microservices and eliminates the need to manage your own message broker or deploy your own worker servers.

schematic

OpenWhisk is an open-source, event-driven compute platform. You send your action code to OpenWhisk and then deliver a stream of data that your OpenWhisk code works upon. OpenWhisk handles the scaling out of the computing resources needed to deal with the workload; all you deal with is the action code and the data that triggers the actions. You pay only for the amount of work that is undertaken, not for servers standing idle waiting for something to happen.

You can write action code in JavaScript or Swift. This means that web developers and iOS developers can create server-side code in the same language as their front-end code.

Getting started

The code snippets and command-line calls made in this blog post assume that you have signed up for the OpenWhisk beta programme in Bluemix and have already installed the “wsk” command-line tool. Visit https://developer.ibm.com/openwhisk/ and click the Try Now button to get started.

Hello World

Let’s create a JavaScript file called ‘hello.js’ containing a function that returns a simple object:

function main() {
   return {payload: 'Hello world'};
}

This is the simplest OpenWhisk action; it simply returns a static string as its payload. Deploy the action to OpenWhisk with:

> wsk action create hello hello.js
ok: created action hello

This creates an action called “hello” that runs the code found in hello.js. We can run it in the cloud with:

> wsk action invoke --blocking hello
{
  "payload": "Hello world"
}

We can also make our code expect parameters:

function main(params) {
   return {payload:  'Hello, ' + params.name + ' from ' + params.place};
}

Then update our action:

> wsk action update hello hello.js
ok: created updated hello

And run our code with parameters:

> wsk action invoke --blocking --result hello --param name 'Jennie' --param place 'The Block'
{
  "payload": "Hello, Jennie from The Block"
}

We’ve created a simple JavaScript function that processes some data and without worrying about queues, workers, or any network infrastructure we were able to execute the code on the OpenWhisk platform.

Doing something useful with our actions

We can do more complex things in our action, such as making API calls. I created the following action, which calls out to a Simple Search Service instance containing Game of Thrones data, passing in the q parameter:

var request = require('request');

function main(msg) {
   var q = msg.q || 'Jon Snow';
   var opts = {
     method: 'get',
     url: 'https://sss-got.mybluemix.net/search',
     qs: {
       q: q,
       limit:5
     },
     json: true
   }
   request(opts, function(error, response, body) {
       whisk.done({msg: body});
   });

   return whisk.async();
}

We can create this action and give it a different name:

> wsk action create gameofthrones gameofthrones.js
ok: created action gameofthrones

Then call it with a parameter q;

> wsk action invoke --blocking --result gameofthrones --param q 'melisandre'
{
    "msg": {
        "_ts": 1460028600363,
        "bookmark": "g2wAAAABaANkAChkYmNvcmVAZGI0LmJtLWRhbC1zdGFuZGFyZDEuY2xvdWRhbnQubmV0bAAAAAJuBAAAAACAbgQA____n2poAkY_7PVPoAAAAGHlag",
        "counts": {
            "culture": {
                "Asshai": 1
            },
            "gender": {
                "Female": 1
            }
        },
        "from_cache": true,
        "rows": [
            {
                "_id": "characters:743",
                "_order": [
                    0.9049451947212219,
                    229
                ],
                "_rev": "1-c68720782e2500311125768153d7170b",
                "aliases": [
                    "The Red Priestess",
                    "The Red Woman",
                    "The King's Red Shadow",
                    "Lady Red",
                    "Lot Seven"
                ],
                "allegiances": [
                    ""
                ],
                "books": [
                    "A Clash of Kings",
                    "A Storm of Swords",
                    "A Feast for Crows"
                ],
                "born": "AtufffdufffdUnknown",
                "culture": "Asshai",
                "died": "",
                "father": "",
                "gender": "Female",
                "mother": "",
                "name": "Melisandre",
                "playedBy": "Carice van Houten",
                "povBooks": "A Dance with Dragons",
                "spouse": "",
                "titles": [
                    ""
                ],
                "tvSeries": "Season 2,Season 3,Season 4,Season 5"
            }
        ],
        "total_rows": 1
    }
}

Writing data to Slack from OpenWhisk

Another task we could perform in an OpenWhisk action is to post a message in Slack. Slack has a great API for creating custom integrations: a Slack administrator can set up an “incoming webhook”, so posting to a channel is as simple as POSTing a string to an HTTP endpoint. We can create a Slack-posting action with a few lines of code:

var request = require('request');

function main(msg) {
   var text = msg.text || 'This is the body text';
   var opts = {
     method: 'post',
     url: 'MY_CUSTOM_SLACK_WEBHOOK_URL',
     form: {
       payload: JSON.stringify({text:text})
     },
     json: true
   }
   request(opts, function(error, response, body) {
       whisk.done({msg: body});
   });

   return whisk.async();
}

replacing MY_CUSTOM_SLACK_WEBHOOK_URL with the Webhook URL that Slack provided when the “Incoming Webhook” integration was created. Notice how this action is executed asynchronously and only calls back when the request has completed.

Then we can deploy and run it in the usual way:

> wsk action create slack slack.js
ok: created action slack
> wsk action invoke --blocking --result slack --param text 'you know nothing, Jon Snow'
{
    "msg": "ok"
}

screenshot

As it happens, Whisk has a built-in Slack integration, but it’s nice to build things yourself isn’t it? Then you can perform your own logic and decide whether a Slack message is posted or not based on the incoming data.

Writing data to Cloudant from OpenWhisk

It is relatively simple to write your own custom action to write to Cloudant because you can:

The disadvantage of this approach is that you’d have to hard-code your Cloudant credentials in the action code, just as we hard-coded the Slack Webhook URL in our previous example, which isn’t best-practice.

Fortunately, OpenWhisk has a pre-built Cloudant integration which you can invoke without any custom code. If you have an existing Cloudant account, then you can grant access to that Cloudant service on the command-line:

> wsk package bind /whisk.system/cloudant myCloudant -p username 'myusername' -p password 'mypassword' -p host 'mydomainname.cloudant.com'

Then you see at list of connections that OpenWhisk can interact with:

> wsk package list
packages
/me@uk.ibm.com_dev/myCloudant                               private binding

where me@uk.ibm.com is my Bluemix username (or the name of your Bluemix organisation) and dev is your Bluemix space.

You can write data to Cloudant by invoking the write command of the package:

> wsk action invoke /me@uk.ibm.com_dev/myCloudant/write --blocking --result --param dbname testdb --param doc '{"name":"George Friend"}'
{
    "id": "656eaeaed0fd47aa733dd41c3c79a7a0",
    "ok": true,
    "rev": "1-a7720095a32c4d1b994ce5e31fe8c73e"
}

Let’s take a breath

So far we’ve created and updated OpenWhisk actions and triggered individual actions as blocking, command-line tasks. The ‘wsk’ tool lets you trigger actions to run in the background and also chain actions together into sequences, but we are not going to cover those options in this post.

Our code has been simple JavaScript blocks where the “main” function is called. This means we can test our code using normal automated testing and continuous integration tools. Deployment of our code is a single command-line statement; no need to worry about servers, operating systems, or network hardware.

OpenWhisk is an event-driven system. You’ve seen how to create an event by deploying code manually. But how can we set up OpenWhisk to act upon a stream of events?

OpenWhisk Triggers

A Trigger in OpenWhisk is another way of firing events and executing code. We can create a number of named Triggers and then create rules that define which of our actions (our code) are executed against which of our triggers. Instead of invoking actions directly, we would invoke Triggers instead; the rules defined against the triggers decide which action(s) are executed. This lets us chain actions together so that one trigger causes several actions to occur and re-use code by assigning the same action code to multiple triggers.

triggers

Triggers can fire individually, or tie to external feeds such as:

  • the changes feed from a Cloudant database – every time a document is added, updated, or deleted, a trigger fires
  • the commit feed of a Github repository – every time a commit occurs a trigger fires

So we can use a Cloudant database to fire a trigger which writes some data to Slack:

> wsk trigger create myCloudantTrigger --feed /me@uk.ibm.com_dev/myCloudant/changes --param dbname mydb 

and configure that trigger to fire our Slack-posting action:

> wsk rule create --enable myRule myCloudantTrigger slack

Now every time a document is added, updated or deleted in the Cloudant database, my custom action fires, which in this case posts a message to Slack!

What would I use OpenWhisk for?

OpenWhisk lends itself to projects where you don’t want to manage any infrastructure. You pay only for the work done, and don’t waste money on idle servers. OpenWhisk easily manages peaks of activity, as it scales out to meet the demand.

Combining OpenWhisk with other “as-a-Service” databases, such as Cloudant, means that you don’t have to manage any data storage infrastructure either. Cloudant is built to store large data sets, cope with high rates of concurrency, and provide high-availability.

As the act of spinning up an OpenWhisk action is non-zero, it makes sense to use OpenWhisk for non-trivial computing tasks like

  • processing an uploaded image to create thumbnails, saving them to object storage
  • taking geo-located data from a mobile application and enriching it with calls out to a Weather API

It is also useful for dealing with systems that feature large amounts of concurrency such as:

  • mobile apps sending data to the cloud
  • Internet of Things deployments where incoming sensor data needs to be stored and acted upon

There are features of OpenWhisk that I haven’t touched on such as Swift support, the ability to use Docker containers as action code instead of uploading source code, and the mobile software development kit.

References

blog comments powered by Disqus