Build a real-time voting application

In this tutorial, you will build a serverless application that uses text messages to poll survey participants and collect their responses in real time. This diagram shows roughly how the app works:

app-flow-diagram

  • create-question does not have a user interface (UI). You invoke the function directly to create a new question in the IBM Cloudant database.
  • get-question occurs when survey participants query the database for a question with the ID from the React UI. If the question exists, they visit the question page.
  • By selecting a choice to vote for, respondents invoke the submit-vote function to send their vote to the Cloudant database, publish the vote to a PubNub channel that everyone else is listening to, and updates the vote count in real time.
  • get-all-votes is the function invoked when participants try to see the vote graphs. The app loads up all of the votes from the Cloudant database and then starts listening for any new votes.
  • handle-message function works with Twilio to deal with incoming text messages from survey participants and to respond appropriately.

Prerequisites

To complete this tutorial you will need the following:

  • IBM Cloud Functions

    This is a functions-as-a-Service (FaaS) platform based on Apache OpenWhisk that I am using for this tutorial. You can access it by signing up for a free IBM Cloud account. Alternatively, you can install a local version of OpenWhisk for testing with Docker Desktop and Docker Compose.

  • Node Package Manager

    For this tutorial, I am using Node.js as the programming language, but with OpenWhisk you can also write your functions in Swift, Go, Python, Java, Ruby, and PHP. In the unlikely case that none of the above are your preferred language, you can create a Docker image of your function and OpenWhisk will run that. So it’s safe to say OpenWhisk can run it all.

  • IBM Cloudant

    The free Cloudant Lite plan provides full functionality of the distributed database for development and evaluation purposes.

  • React

    This tutorial shows a very simple use of React for the front-end application. We won’t go much deeper into the use of React.js here.

  • Twilio

    This is a messaging platform that you will use to collect votes. Twilio has a trial account which provides a credit amount for you to try it out. You do not need to provide a credit card for the trial account, but the credit will deplete as you use it. Check the pricing per message details on the Twilio Pricing page for more information.

  • PubNub

    The real-time portion of the app depends on PubNub, which will give you a publish and subscribe platform to get real time data streaming in the app. The free tier will give you enough for this project and to tinker.

Estimated time

From end to end, this tutorial should take you 60 to 90 minutes to complete.

Step 1: Install IBM Cloud command line interface (CLI)

You can use the IBM Cloud CLI installer or the specific operating system shell installers below.

  • MacOS

     curl -fsSL https://clis.ng.bluemix.net/install/osx | sh
    
  • Linux

     curl -fsSL https://clis.cloud.ibm.com/install/linux | sh
    
  • Windows

     iex(New-Object Net.WebClient).DownloadString('https://clis.cloud.ibm.com/install/powershell')
    

    Note: If you are using Windows, you may see the following error: Exception calling “DownloadString” with “1” argument(s): “The underlying connection was closed: An unexpected error occurred on a send.” At line:1 char:1 This indicates an issue with your PowerShell Mutual TLS setting. In this situation, use the IBM Cloud CLI installer.

Step 2: Set up your local development environment to use Cloud Functions

Run the following command in your terminal to install the Cloud Functions plugin for the IBM Cloud CLI:

ibmcloud plugin install cloud-functions

Cloud Functions should have been setup with your IBM Cloud account creation. If for some reason, it does not automatically create the Cloud Foundry orgs space, set up your local development environment with this exercise.

Step 3: Clone the Git repository

Run the following command in your terminal to clone the realtime-polling.git repository, which contains both the web application and code for the functions that you will deploy:

git clone https://github.com/moficodes/realtime-polling.git

Step 4: Create a new PubNub app

Log into your PubNub account, and create a new App and keyset. Each keyset has a Publish Key (pubkey), Subscribe Key (subkey), and Secret Key (secretkey). You will need this information in future steps.

Step 5: Create an IBM Cloudant database

In this step, you will create a Cloudant database to hold the polling questions and participant votes.

  1. Log into IBM Cloud.
  2. In the top navigation bar, click Catalog.
  3. Search the catalog for “Cloudant.”
  4. Select the Cloudant box that appears as a result of your search.
  5. In the Service name field, type a name for the service (overwriting the prefilled information). I am naming mine mofi-workshop-db.
  6. From the Available authentication methods list, select the Use both legacy credentials and IAM value.
  7. Click Create. When your Cloudant database service is successfully provisioned and available for use, it will appear with a status of Provisioned on the “Resource list” page (under the Services category).
  8. On the “Resource list” page, expand Services and then select your database service to open it.
  9. On left side of your database service page, click the Service credentials tab:

    service-credentials

  10. Click on New credential.

  11. In the Name field, type a name for your credential or use the prefilled text. I am naming mine Service credentials-1.
  12. From the Select Service ID list, select the Auto Generate value.
  13. Click Add.

    add-new-credential

  14. Once your service credential is generated, click View credentials under the ACTIONS column to take a look at it. You will need the username and password information in a later step.

  15. Click Manage on the left side of your database service page and then click Launch Cloudant Dashboard on the right side of the page.
  16. Once the Cloudant dashboard loads, click the Databases tab on the left side and then click Create Database on the top navigation bar.

    databases

  17. Name the database questions and click Create.

The reason to do this is because when you create a poll question, you will put it in this particular database instead of having to explicitly check to see if the question exists every time the function runs. This will make more sense when we look at the function in Step 7. For now, just take my word for it. It’s also a place to view your data. If you ever need to look closely at your data, this is where you will do so.

Step 6: Create a Twilio project

  1. Log into your Twilio account.
  2. On the Create a Project page, click on the Products tab.
  3. Under COMMUNICATIONS CLOUD, select Programmable SMS and click Continue at the bottom of the page.
  4. Type a name for the project within the PROJECT NAME field (overwriting the prefilled information).
  5. Click Skip Remaining Steps.
  6. Within your project dashboard, select the second tab on the far left sidebar to open the Programmable SMS Dashboard, which looks like the screen shot below. (Note: By default, trial accounts in Twilio can only send messages to verified numbers. This means you could make your app and try it out yourself. But you will not be able to send messages to other people. This is fine for the needs of this tutorial.)

    Twilio

  7. Click Get Started.

  8. Click Get a number. A number will be assigned for you. Choose that number or search for a different number if you want. When finished, click Choose this Number.
  9. On the second left navigation bar, click SMS to open the Messaging Services page, which looks like this:

    Twilio-messaging-service

  10. Click Create new Messaging Service.

  11. In the “Create new Messaging Service” window, enter a name for your service in the FRIENDLY NAME field (I’m calling it polling).
  12. In the USE CASE menu, select Mixed. Click Create.
  13. On the Settings page, under the Inbound Settings section, select the SEND AN INCOMING_MESSAGE WEBHOOK check box. Two combination boxes appear, labelled REQUEST URL and FALLBACK URL. These are where you will use the handle-message function webhook to handle an incoming Twilio message. We will come back to these in a bit.
  14. Click Save.

Step 7: Understand the five functions of the app

I am using a total of five functions to manage the backend of this application. There is no actual setup in this step. But you should read this to know the tasks of each function and why they are there.

create-question does exactly as the name suggests and a little more. The things this function has to accomplish are:

  • Take user input for an id of the question, the question, and the options.
  • Connect to the Cloudant database using the username and password input.
  • Check if the id already exists.
  • Create a new record in the questions table with the id, question, and options.
  • Create a new table for the participant votes of that question to be stored.
  • Create an index on that table so that it can be sorted with timestamp.

Note: create-question is a pretty overloaded function to be honest. But most of these are one time things that needs to happen for the rest of the app to function (no pun intended) properly.

The get-question function is probably used in most places in the app:

  • Takes in the id, username, and password from the user.
  • Connects to the Cloudant database using the username and password input.
  • Queries the questions table for the id. If id exists in the table already, that means the id is taken as an error condition. If no error, the question and options return from the questions table.

The get-all-votes function is mainly used for the initialization of graph:

  • Get id input of a question.
  • Connect to the Cloudant database.
  • Connect to the table of the specific question.
  • Get all the records from the table.
  • Return all the questions.

The submit-vote function is probably the most used, as it is what drives our application:

  • Gets id input of a question.
  • Gets index of the option voting for.
  • Connects to a Cloudant database.
  • Tries to use the database for the question. If that table cannot be found, then the id was not valid and an error returns. Note: Because you create that table when the question is created, I think you can explicitly check to see if the question exists and create that table if it does. But I think it’s fine for our application.
  • Create a record in the table.
  • Publish the message to PubNub.

The handle-message function is used to deal with incoming messages to Twilio. The way this works is when a message is sent to your Twilio number, it invokes a web action and passes a bunch of data related to the message to the function. That data will look something like this:

    "params": {
        "AccountSid": "XXXXXXXXXXXXXXXXX",
        "ApiVersion": "2010-04-01",
        "Body": "Hello",
        "From": "+12101234567",
        "FromCity": "NEW YORK",
        "FromCountry": "US",
        "FromState": "NY",
        "FromZip": "10010",
        "To": "+12345678901",
          .
          .
          .
        "__ow_method": "post",
        "__ow_path": ""
    }

In this example, To is the Twilio number and From is where the message is being sent from. We will setup the Twilio stuff in the next step. This is what the handle-message function does:

  • Gets user sent text from the body key of the parameters sent to the function. The above example has Hello as the body. (Note: For the logic of dealing with the body text, I am taking a simple approach. If the text is ? or has the word help in it, you will send information on how to use the app via the message. If the message has one line, you will try to get the question and set a variable with the question and option. If the message body has two lines, we will use the first line as id and second line as the vote choice. If the function fails for any reason, you can set the variable with the appropriate message.)
  • Returns a twiml message without a message variable, which Twilio will interpret and send to the user.

Step 8: Start the React front-end application

  1. In your terminal, open the folder where you cloned the realtime-polling.git repository in Step 3:

    cd realtime-polling
    
  2. Copy the secret.template.json file to secret.json:

    cp src/secret.template.json src/secret.json
    

    This is what the secret.json file should contain:

       {
         "SUBMIT_VOTE": "submit-vote-api-key",
         "SUBMIT_VOTE_URL": "submit-vote-api-url",
         "GET_QUESTION": "get-question-api-key",
         "GET_QUESTION_URL": "get-question-api-url",
         "GET_VOTES": "get-votes-api-key",
         "GET_VOTES_URL": "get-votes-api-url",
         "SUBSCRIBE_KEY": "pubnub-subscribe-key",
         "PUBLISH_KEY": "pubnub-publish-key",
         "SECRET_KEY": "pubnub-secret-key"
       }
    

    You will complete this file later, but need this to start the application.

  3. Install the application dependencies:

    npm install
    
  4. Start the application server:

    npm start
    

You should see the application pop up in the browser at http://localhost:3000.

Step 9: Create the app functions

You can create your app functions using either the web UI or the IBM Cloud CLI.

From the web UI

  1. Log into IBM Cloud.
  2. From the Navigation Menu (hamburger icon) located in the upper left corner, select Functions.

    ibm-cloud-functions

  3. Select Actions from the Functions menu located on the left side of the window.

    ibm-cloud-functions-actions

  4. On the Actions page, click the Create button.

  5. On the Create Action page, fill out the fields for your first action:
    • In the Action Name field, type create-question.
    • Click Create Package and enter a name for how you want to organize all of the actions you are creating. I’m naming mine workshop.
    • In the Runtime field, select Node.js 10.
  6. Click Create located in the lower right of the page. (Note: This first time, you have to create a new package, but for the rest of this tutorial, you can select your package name from the drop-down list.)

    ibm-cloud-functions-action-name

    The editor page opens, showing the code of your web action:

     /**
     *
     * main() will be run when you invoke this action
     *
     * @param Cloud Functions actions accept a single parameter, which must be a JSON object.
     *
     * @return The output of this action, which must be a JSON object.
     *
     */
    function main(params) {
           return { message: 'Hello World' };
    }
    
  7. Click Invoke in the upper right to invoke this action.

  8. You can also change input to pass in a params. To set default parameters, click the Parameters tab on the left.
  9. Click Add Parameter.
  10. In your terminal, open the folder where you cloned the realtime-polling.git repository in Step 3, and go to the functions directory.

    cd functions
    
  11. All five functions are in their own folders. Copy the create-questio.js file from the create-question folder and paste it into the Web Action editor (within IBM Cloud).

       /**
       *
       * main() will be run when you invoke this action
       *
       * @param Cloud Functions actions accept a single parameter, which must be a JSON object.
       *
       * @return The output of this action, which must be a JSON object.
       *
       */
      const Cloudant = require('@cloudant/cloudant');
      let cloudant = null;
      async function main(params) {
        if (params.id === undefined || params.question === undefined || params.options === undefined) {
            return {
              error: "Not Enough Arguments",
            }
        }
        const reused = cloudant != null;
        var username = params.username;
        var password = params.password;
        if (cloudant == null) {
          cloudant = Cloudant({
            account: username,
            password: password
          });
        }
        var id = params.id;
        const database = cloudant.db.use('questions');
        const docs = (await database.find({
          "selector": {
            "_id": id
          },
          "fields": [
            "_id",
            "question",
            "options"
          ]
        })).docs;
        if (docs.length > 0) {
          return {
            error: "ID already Exists",
          }
        };
        const data = {
          _id: id,
          question: params.question,
          options: params.options,
        }
        const result = await database.insert(data);
        console.log(result.ok);
        if (result.ok) {
          dbcreate = await cloudant.db.create("questions-" + id);
          return {
            ok: true,
            payload: id,
          }
        }
        return {
          error: "Insertion failed",
        };
      }
      exports.main = main;
    
  12. Click Save.

    Note: From the code (and the Activations pane that appears on the right if you click Invoke), you can see that the params object is expected to have five things: id, question, options, username, and password. The first three will be passed in when the function is being called via API. You will set the username and password parameters next using default parameter settings.

  13. Click the Parameters tab in the left navigation column.

  14. Add the username and password that were created for your Cloudant service credentials (in Step 5) within the corresponding Parameter Name and Parameter Value fields.

    ibm-cloud-functions-parameters

  15. Click Save.

  16. Go back to the Code tab in the left navigation column.
  17. Click Change Input located in the upper right to open the Change Action Input dialog box.
  18. Paste the following JSON code:

    {
          "id": "001",
          "question": "Who was the best James Bond",
          "options": ["Daniel Craig", "Sean Connery", "Pierce Brosnan", "Roger Moore"]
    
    }
    
  19. Click Apply.

  20. Click Invoke. You should see something like this on the side:

    create-question

  21. If you click Invoke again, you should see an error because that was the logic we implemented. If the id already exists in the database, it returns an error like this:

    create-question-error

From the IBM Cloud CLI

Let’s create an action from IBM Cloud CLI. This time, we will create the get-question action.

  1. Login to IBM Cloud CLI:

    ibmcloud login
    
  2. Use your user name and password to login. (You can also use ibmcloud login --sso to log in using the single sign-on method using your browser and access token.)

  3. Target a Cloud Foundry org:

    ibmcloud target --cf
    
  4. Check that you have the Cloud Functions plugin enabled.

  5. Run ibmcloud fn. You should see the help page for the IBM Cloud plug-in.
  6. To see the actions in your account, run the following:

    ibmcloud fn action list
    

    Your output should show the other action (create-question) that you created in the previous step:

    actions
    /thisismofi@gmail.com_dev/workshop/create-question                     private nodejs:10
    
  7. To create the get-question action, change the directory into the folder containing the get-question.js file. If you are in the realtime-polling folder, it is located under function/get-question:

    cd functions/get-question
    
  8. Create the action:

    ibmcloud fn action create workshop/get-question get-question.js --kind nodejs:10
    
  9. You should see the action with ibmcloud fn action list.

  10. Set up the default parameter:

    ibmcloud fn action update workshop/get-question --param username "YOUR-CLOUDANT-USERNAME-HERE" --param password "YOUR-CLOUDANT-PASSWORD-HERE"
    
  11. You can now invoke the action from the CLI as well:

    ibmcloud fn action invoke workshop/get-question --param id 001
    

Your output should return what you inserted in the previous step:

       {
           "ok": true,
           "payload": [
               {
                   "_id": "001",
                   "options": [
                       "Daniel Craig",
                       "Sean Connery",
                       "Pierce Brosnan",
                       "Roger Moore"
                   ],
                   "question": "Who was the best James Bond"
               }
           ]
       }

Actions with external dependency

The submit-vote action has external dependency. Actions with external dependencies cannot be created using the web UI or IBM Cloud CLI, so use your terminal for this.

  1. Change directory in the submit-vote folder.
  2. Install the dependencies:

    npm install
    
  3. Package the files into a zip: (Note: The zip command will only work in a MacOS or Linux environment. For Windows, use a third party tool like 7zip or look at stack-overflow answer.)

    zip -r submit-vote.zip *
    
  4. Create the action as you would with IBM Cloud CLI:

    ibmcloud fn action create workshop/submit-vote submit-vote.zip --kind nodejs:10
    
  5. This action needs five default parameters: the Cloudant username and password, as well as the publish_key, subscribe_key, and secret_key from PubNub (look back at Step 4 to find these).

  6. You can setup the default parameters from either the CLI or the web UI. From the IBM Cloud CLI:

    ibmcloud fn action update workshop/submit-vote --param publish_key "YOUR PUBNUB PUBLISH KEY" --param subscribe_key "YOUR PUBNUB SUBSCRIBE KEY" --param secret_key "YOUR PUBNUB SECRET KEY" --param username "CLOUDANT USERNAME" --param password "CLOUDANT PASSWORD"
    

From the web UI:

  1. Go to Functions and click on the submit-vote action from the list of actions.
  2. Go to Parameters and add the parameters.

    submit-vote action

Finish the rest

This leaves two functions: get-all-votes and handle-message. These do not have any external dependencies. So feel free to create them from the web UI or IBM Cloud CLI. The get-all-votes function needs two default parameters: Cloudant username and password, which you learned how to add in the previous section.

Note about external dependency

If you look at the code, there are a few functions with dependencies on Cloudant and one with Openwhisk. These are not considered to be external dependencies in IBM Cloud Functions. There are a bunch of packages that come preinstalled in the environment. See a complete list.

Step 10: Create the API gateway to manage access to the functions

Now that you have your function, let’s review how to use it for your app. API gateway is great way to manage access to your function.

Get-Question API

  1. Open the IBM Cloud Functions dashboard.
  2. On the navigation menu located on the left side of the window, click APIs.
  3. Click Create a Cloud Functions API.
  4. Type get-question into the API name field
  5. In the Base path for the API field, type /get-question.
  6. Click Create operation.

    ibm-cloud-functions-api-info

  7. In the Create operation dialog box, just put / in the Path field since you will only have one API at that end point.

  8. From the Verb list, select GET.
  9. From the Package containing action list, select workshop.
  10. From the Action list, select get-question.
  11. From the Response content type list, select application/json.

    ibm-cloud-functions-create-operation

  12. Click Create.

  13. Under the Security and Rate Limiting section, enable application authentication by selecting Require applications to authenticate via API key, and choosing API key only from the Method list.
  14. Enable rate limiting by selecting Limit API call rate on a per-key basis and choosing a 20 calls for Maximum calls.
  15. Skip OAuth user authentication for now, but know that you could add social login with IBM Cloud App ID, Google, Facebook, or Github.
  16. Leave CORS enabled to allow your React app to call this API.
  17. Click Save.
  18. On the left side of your “get-question” API page, click Sharing.
  19. Within the Sharing Outside of Cloud Foundry organization section, click Create API key.

    ibm-cloud-functions-api-get-question

  20. Give your API key a name, such as get-question-api-key, and click Create.

  21. Once the API key is created, you should see an API portal link. If you click the API portal link, it will show you the API and how to access it via curl and seven programming languages including Java, Node, Go, and Python.

    ibm-cloud-functions-api-portal

  22. On the left side of your “get-question” API page, click API Explorer to find the endpoint for the API.

  23. Make a note of the API Key and the API Endpoint since you will need those for your React application.

Get-All-Votes API

Follow the instructions above for creating the API for the get-all-votes function.

Submit-Vote API

You can follow almost all of the same steps above for creating the API for the submit-vote function. However, in the Create operation dialog box, select POST from the Verb list (not GET like you did for the other APIs). If you are wondering why, read this.

Step 11: Put it all together

In the src folder of the application, there is a file called secret.template.json. Copy the file and save it as secret.json.

The content of the secret.json file is as follows:

{
      "SUBMIT_VOTE": "submit-vote-api-key",
      "SUBMIT_VOTE_URL": "submit-vote-api-url",
      "GET_QUESTION": "get-question-api-key",
      "GET_QUESTION_URL": "get-question-api-url",
      "GET_VOTES": "get-votes-api-key",
      "GET_VOTES_URL": "get-votes-api-url",
      "SUBSCRIBE_KEY": "pubnub-subscribe-key",
      "PUBLISH_KEY": "pubnub-publish-key",
      "SECRET_KEY": "pubnub-secret-key"
}

Replace the data with corresponding information you collected during the previous tutorial steps. If you are having trouble finding the PubNub keys, review Step 4 above. You obtained the URL and API keys in Step 10 above.

If everything else worked, you should be able to search the question that you created:

  1. Type in the id (for the question you created, the ID was “001.”)
  2. Click Enter.

    james-bond

  3. Open a new browser window, go to the question again, and then click Watch The Votes. You should see a bar graph of the question and the options appear. Vote on one of the options on the screen and you should see the numbers rise on the other screen.

    james-bond-answers

With that, your web application is done.

Step 12: Enable the Twilio webhook

To handle Twilio messages, you must convert the handle-message function into a web action.

  1. Open the Actions page in IBM Cloud Functions.
  2. Select the handle-message action.
  3. On the left side of the page, click Endpoints.
  4. Select the Enable as Web Action check box.
  5. Click Save.

    ibm-cloud-functions-endpoints

  6. Copy the URL.

  7. Within Twilio, open the Messaging Services page.
  8. Click your application name (I named it polling).
  9. Click Settings.
  10. Under the Inbound Settings section, select the SEND AN INCOMING_MESSAGE WEBHOOK check box. Two combination boxes appear, labelled REQUEST URL and FALLBACK URL.
  11. Paste the URL in the REQUEST URL combination box, but change the end to .http from .json. This tells Twilio to accept a HTTP response and not a JSON response. This is crucial because you are making use of TwiML to send a reply to the user.
  12. Test your offline capabilities by texting the Twilio number from your mobile phone.

    • If you text a ?, it should reply back with some helpful text.
    • If you text an id of a question, it will reply with the question and options.
    • If you text an id and an index separated by a new line, a vote will be submitted for the id of that index.

    Twilio-james-bond

Summary

This is a very simple use case. Using serverless architecture is a great way to build backend functionality for mobile apps, as well as web apps. In addition, with the power of Twilio, you can easily enable offline capabilities to your app. Now you know how to:

  • Create OpenWhisk functions.
  • Enable functions as web APIs.
  • Create functions with dependency.
  • Connect to a Cloudant database from OpenWhisk.
  • Receive a text message and respond to it using webhooks.
Mofizur Rahman