2021 Call for Code Awards: Live from New York, with SNL’s Colin Jost! Learn more

Create unit tests for pure OpenWhisk actions

The modular nature of OpenWhisk applications makes it easy to create unit tests for actions that are purely functional — meaning they have no side effects and do not rely on external state. In this tutorial, you’ll learn how to create these unit tests semi-automatically, and how to run the tests to verify that code changes do not break anything.

Pure functions are functions that fulfill two conditions:

  1. The result of the function depends only on the input arguments. Any time it is called with the same input arguments, it will produce the same result.
  2. The function does not have any side effects; getting the return value is the only effect of running the function.

When using a Function as a Service (FaaS) framework such as OpenWhisk, any actions that are pure functions can have automatic QA applied using the methods you learn in this article.

What you’ll need to build your application

  • A basic knowledge of OpenWhisk and JavaScript
  • An IBM Cloud account

Unit tests

A usable application is likely to consist of multiple modules, each performing a specific action. In addition to testing the application as a whole, it is useful to test each module separately. This type of testing is called unit testing, and it lets you identify where bugs are located and fix them relatively easily.

One of the challenges of unit tests is that you need to identify possible inputs and their matching outputs to each module. If the module is simple, then this process can be trivial — but with more complicated modules, it can be more difficult. This is particularly true for modules that are embedded deep in the application and don’t have human-readable input or output.

In the case of actions that act as pure functions, you can identify the inputs and outputs automatically by seeing how they are called when the application is in use. Then, when a programmer changes the module, you can rerun it with the same inputs to see if the change broke anything.

The sample application

The purpose of this tutorial is to show you the QA technique, so the sample application is very simple: a dictionary app that lets users choose a word in English and then provides the equivalent in other languages.

Create the application sequence

To learn how to write a user-facing application in OpenWhisk, read my tutorial, “Build a smart lock for a connected environment” (sections 1, 2, and 5). In contrast to that application, the app in this tutorial consists of three actions:

  1. getTranslation
  2. translation2tables
  3. tables2html

You create all three actions in OpenWhisk, and they are usually pure, but getTranslation can become impure if it gets a value in the error parameter.

After you create the actions, this is how you combine them into a sequence and make the sequence accessible from browsers:

  1. Click Develop in the left sidebar.
  2. Under MY ACTIONS, select the first action in the sequence, getTranslation.
  3. Click Link into a Sequence.
  4. Click the MY ACTIONS tile, select the second action, translation2tables, and then click Add to Sequence.
  5. Click Extend and repeat the previous step to add the third and final action, tables2html.
  6. Click This Looks Good.
  7. Name the new sequence UnitTestsApp. Click Save Action Sequence and then Done.

The sequence should look like this:

Application sequence

Application sequence

Make the sequence accessible from the Internet

Follow these steps to make the sequence accessible from the Internet:

  1. Click APIs in the left sidebar.
  2. Click Create Managed API.
  3. Name the API UnitTestsApp. Type the base path /UnitTestsApp.
  4. Click Create Operation and create an operation with these parameters:
Parameter Value
Path /translate
Verb GET
Package containing action Default
Action UnitTestsApp
Response content type text/html
  1. Scroll down, make sure Enable CORS is selected, and click Save & expose.
  2. Copy the route, paste it in a browser window, type /translate at the end, and browse to that URL (mine is https://service.us.apiconnect.ibmcloud.com/gws/apigateway/api/ec74d9ee76d47d2a5f9c4dbae2510b0b8ae5912b542df3e2d6c8308843e70d59/UnitTestsApp/translate).
  3. Make sure that when you click the blue buttons in the application you get translations. When you click the red buttons in the application, you may get translations or you may get an error message.
Sample translations

Sample translations

Capture (input, output) pairs

To create unit tests, you need to get inputs and matching outputs. The easiest way to do this is to edit the sequence to replace the existing action with a wrapper that calls the original action and stores its input and output.

Replace the action with a wrapper

To replace the action with a wrapper, you first need to be able to call the action remotely. You could add it to the API or make it a web action, but it is easier and safer to use the following method:

  1. Click Develop on the left sidebar and open the action (getTranslation).
  2. Click View REST Endpoint.
  3. Scroll down to the cUrl command and click Show Full Command. Copy the command to a text file.
  4. Use a base64 decoder (such as this one) to decode the authentication information, the part after Basic, as shown in the illustration below (the value is redacted). This value is your API key.
Authentication information

Authentication information

  1. Create a new action called callGetTranslation, with the contents of this file (make sure to put your own values in the api_key on line 4 and name on line 11).
  2. Run the action, observe that it is functionally equivalent to getTranslation.

So how does it work?

These lines use the OpenWhisk NPM library to construct a connection to the server. The connection needs the hostname and the API key (which you need to provide — I am not telling you mine because then you’d be able to execute all my OpenWhisk actions without my permission).

Calling another OpenWhisk action can take a long time in computer terms, possibly even a whole second. To avoid being charged for a process for all that time, you need to create and return a Promise object. The object constructor has one parameter, the function you run to get the result. This function receives two parameters — one function to call in the case of success, and another function to call in the case of failure.

return new Promise((success, failure) => {

When the Promise function is called, it calls ow.actions.invoke to run the original OpenWhisk action. The name parameter is the path to the original action (you’ll need to replace it with your own value). If the invocation is called without blocking, it returns immediately with an ID that you can use to query the result later — but for this use case, it is better to block until you have an answer.

var invocation = ow.actions.invoke({
     name: '/developerWorks_Ori-Pomerantz-apps/unit-tests/getTranslation',
     blocking: true,
     params: params

The invocation variable contains a Promise object. The then method gets the function inside the Promise called, and then calls the parameter function with the result.

invocation.then((resVal) => {

Create and log (to show the value) a variable with the input and output:

var logMe = {
    input: params,
    output: resVal.response.result

Return the result to the OpenWhisk infrastructure that called the original Promise object.


In Figure 4, you see the logs from two separate invocations, one with empty input and one with a word in the input.

Two invocation logs

Two invocation logs

Write the (input, output) pairs to Cloudant

For the unit test data to be useful, it needs to be preserved in a Cloudant database. First, create such a database (named expected), as explained in this article, section 4a (the name of the service does not matter). Then replace callGetTranslation‘s code with the contents of this file. Remember to change the relevant parameters in lines 4 (OpenWhisk API key), 9-13 (Cloudant credentials), and 26 (path to the action).

Here I explain just the new parts, those having to do with the database.

Cloudant database entries (called documents) are accessible using a string. The index you’ll use is the input parameters.

var dbKey = JSON.stringify(params);

Before you write an entry to the database, you should check if you’ve seen this input before.

// Did we already see this input?
mydb.get(dbKey, (err, body) => {

If you try to get a document and fail, you will get an error with status code 404. In that case, write to the database and then call the callback.

// No, write it.
 if (err != null && err.statusCode == 404)     {
         {"_id": JSON.stringify(params), data: logMe},
         () => {
         });   // end of mydb.insert call
 }  // End of "this value has not been found"
 else {  // Assume value has been found, for the sake of simplicity

If this input value is already in the database, compare the current output to the output in the database. If they are the same, that’s great. If they aren’t, then there is a problem. The action is not acting as a pure function, either because of a bug or because it wasn’t pure to begin with.

This sample code just reports the problem to the console. In a more realistic implementation, the code would instead log it to the issue tracking system for development and QA use.

// We found this before, but the output was different then
if (JSON.stringify(body.data) !== JSON.stringify(logMe)) {
    console.log("This action isn't a pure function.");
    console.log("For input:" + JSON.stringify(params));
    console.log("Old output:" + JSON.stringify(body.data.output));
    console.log("New output:" + JSON.stringify(logMe.output));

Regardless of what else happens, you need to return the output value.

        // Regardless, return the value.
    }  // End of "this value has been found, not new"
}); // end of mydb.get call

Integrate with the application

Finally, edit the sequence of the application to replace the old action, getTranslation, with the new one, callGetTranslation. Then run the application and select different options to get enough inputs and their matching outputs.

Note that if you look in the database you may see a lot of input parameters that you did not expect. This is because the first action in the sequence gets information about the HTTP request, and that is the action we trace in this example. These extra parameters don’t cause any problems.

Use unit tests

Now that you have the information in the database, you can use it to run unit tests. Create a new action with this content. Put the correct identifiers in lines 4, 13, and 38. When you run this action, it returns the following fields:

  • match — the number of test cases in which the action returned the expected output
  • noMatch — the number of test cases in which the action returned a different value
  • problems — a list of input values for which problems were detected

Most of this action uses components that you used earlier in this article. The one exception is the async.map call. This function receives three parameters, a list and two functions. It then calls the first function (the iteratee) individually for every item in the list, like a normal map in functional programming. The difference is that the function called can be asynchronous. It gets two parameters: the item from the list and a callback function to call when the asynchronous process finishes. This allows you to use the map with an asynchronous function — for example, calling a different OpenWhisk action. A third parameter (which is optional but used here) is a function that’s called after the map is done.

                (obj, callback) => {
                }, // end of the iteratable, which is called for every item
                (err, results) => {
                }  // end of the post map callback
            );  // end of async.map

The information is gathered in variables that are shared between the different calls to the iteratee. In some environments this would require you to worry about thread safety, but the single threaded nature of Node.js ensures that there won’t be two threads trying to change the variable at the same time.

Here is a sample result, shown on JSON formatter (to minimize the HTTP headers, which are not relevant):

Sample result

Sample result


In this tutorial, you developed a prototype for determining whether actions that are supposed to be pure functions (with the same input always producing the same output, and no side effects) actually have at least one property of purity. Unfortunately, you cannot check a black box such as an OpenWhisk action for side effects. I made this prototype simple in order to explain the concepts as clearly as possible. If you want to put it to real use, you need to add a few other features.

  • Fail gracefully. Right now, the code just assumes that errors do not occur. That is … optimistic.
  • Instead of calling the tested action once, call it multiple times. That way, you’re more likely to detect problems.
  • Add the name of the action being called to the database key (which is currently just the input parameters). That way, you can use the same database for multiple actions.

These three features are relatively easy to implement while still treating the OpenWhisk action as a black box. A more challenging feature would be to proxy the connections between the action and other systems, such as Cloudant, to turn actions that aren’t pure functions into pure functions; you could do this by providing the same state information from other systems as the state provided during a previous call with the same input, and removing any side effects that the action tried to create.