Create a voice-enabled COVID-19 chatbot using Node-RED

Learn how to create a voice-enabled chatbot using Node-RED and the IBM® Watson™ Assistant, Watson Speech to Text, and Watson Text to Speech services.

Learning objectives

In this tutorial, you will:

• Learn about Node-RED and see how to install it locally and on IBM Cloud
• Explore the node-red-node-watson Node-RED nodes
• Import and deploy a Watson Assistant chatbot example
• Build a Call for Code COVID-19 crisis communications voice-enabled chatbot solution

Prerequisites

• Install Node-RED locally or Create a Node-RED Starter application in IBM Cloud.
• Create a Watson Assistant COVID-19 crisis communications chatbot. Follow the instructions.

Estimated time

Completing this tutorial should take approximately 30 minutes.

Architecture diagram

The following diagram shows the workflow for a Node-RED chatbot that answers questions about COVID-19.

1. A user visits a voice-enabled Node-RED website with the COVID-19 chatbot and asks a question.
2. Node-RED records the speech WAV file and calls the Watson Speech to Text Service hosted in IBM Cloud.
3. Watson Speech to Text uses machine learning to decode the user’s speech.
4. Watson Speech to Text replies with a transcript of the COVID-19 question and Node-RED calls Watson Assistant hosted in IBM Cloud.
5. Watson Assistant uses natural language understanding and machine learning to extract entities and intents of the user question.
6. Source COVID-19 FAQ information from trusted CDC data.
7. Watson Assistant invokes an OpenWhisk open source-powered IBM Cloud Function.
8. IBM Cloud Function calls Watson Discovery service running in IBM Cloud.
9. Watson Discovery scans news articles and responds with relevant articles.
10. Watson Assistant invokes an OpenWhisk open source-powered IBM Cloud Function.
11. IBM Cloud Function calls COVID-19 API to get statistics.
12. Watson Assistant replies to the user inquiry and Node-RED sends the text transcript to Watson Text to Speech.
13. Watson Text to Speech encodes the message in the user’s language.
14. Node-RED plays the chat answer .wav file to the user.
15. The user listens to the chat answer.

Steps

Learn about Node-RED

Node-RED is an open source programming tool for wiring together hardware devices, APIs, and online services in new and interesting ways. It provides a browser-based editor that makes it easy to wire together flows using the wide range of nodes in the palette that can be deployed to its runtime in a single-click.

Install Node-RED dependencies nodes

After Node-RED is installed, add the dependencies for this tutorial:

• node-red-node-ui-microphone
• node-red-dashboard
• node-red-node-watson
• node-red-contrib-play-audio
• node-red-contrib-twc-covid19-tracker

Local installation instructions

npm install node-red-node-ui-microphone node-red-dashboard node-red-node-watson node-red-contrib-play-audio node-red-contrib-twc-covid19-tracker

IBM Cloud installation instructions

• Instead of adding the additional packages via Manage Palette, use the IBM Cloud Toolchain and the git repository in IBM Cloud to add the following packages to the package.json. Commit the change and the CI/CD toolchain will restage the CF application.

"node-red-node-ui-microphone":"0.x",
"node-red-dashboard":"2.x",
"node-red-contrib-play-audio":"2.x",
"node-red-contrib-twc-covid19-tracker":"0.x",

Explore node-red-node-watson Node-RED nodes

The node-red-node-watson GitHub repository includes a collection of Node-RED nodes for IBM Watson services. This package adds the following nodes to your Node-RED palette.

• Assistant: Add conversational capabilities into applications
• Discovery: List environments created for Watson Discovery
• Language Identification: Detect the language used in text
• Language Translator: Translate text from one language to another
• Natural Language Classifier: Use machine learning algorithms to return the top matching predefined classes for short text inputs
• Natural Language Understanding: Analyze text to extract metadata from content such as concepts, entities, and keywords
• Speech To Text: Convert audio containing speech to text
• Text To Speech: Convert text to audio speech

Import and deploy the COVID-19 crisis communication chatbot example

Now that you have installed the Node-RED dependencies, Watson nodes are available to integrate Watson AI services. Let’s build a voice-enabled chatbot example that uses Watson Assistant, Watson Speech to Text, and Watson Text to Speech.

1. Import the flow from our Call for Code Starter kit.

2. Deploy the flow.

Create Watson Services on IBM Cloud

Before the flow will execute successfully, you must configure the Watson Assistant and Watson Speech nodes with new service instances and API Keys.

Create a Watson Assistant instance

1. If you haven’t already, create a Watson Assistant instance.

2. Follow these instructions to provision a Watson Assistant chatbot for COVID-19.

Create a Watson Speech to Text Service instance

1. Create a Watson Speech to Text Service instance instance from the IBM Cloud Catalog.

2. Click Create.

   The Node-RED Watson Speech to Text node needs the apikey credentials for this new instance.

3. After the Watson Speech to Text service is created, click Service credentials (1).
4. Click the View credentials (2) twistie.

5. Copy the apikey (3) for use in the next section.

Create a Watson Text to Speech Service instance

Create a Watson Text to Speech Service instance from the IBM Cloud Catalog.

1. Click Create.

   The Node-RED Watson Text to Speech node needs the apikey credentials for this new instance.

2. After you create the Watson Text to Speech Service, click Service credentials (1).
3. Click the View credentials (2) twistie.

4. Copy the apikey (3) for use in the next section.

Enable Watson Speech nodes with API keys

1. Double-click the speech to text node, and paste the API key from the Watson Speech to Text Service instance.

2. Click Done.

3. Double-click the text to speech node, and paste the API key from the Watson Text to Speech Service instance.

4. Click Done.

Enable a Watson Assistant node with the COVID-19 workspace ID and API key

1. Double-click the assistant node, and paste the Skill ID into the Workspace ID field, Assistant Service Endpoint URL and the API key from the Watson Assistant service instance.

2. Click Done.

Parse intents and invoke API calls

Watson Assistant returns the intents related to your questions.

• The Switch node routes two intents to an http request node to query an external data source for current COVID statistics.
• A Function node adds up the summary statistics and builds a sentence to be spoken.

Deploy the Node-RED flows

• To deploy the Node-RED flows, click the red Deploy from your UI.
• To launch the Node-RED Dashboard, click on the dashboard tab in the right sidebar. Click on the arrow icon in the upper right corner.

Talk to your COVID-19 crisis chatbot

To talk to your chatbot, click the microphone input button and ask a question about COVID.

When someone asks a question, here’s the Node-RED flow that happens in the background:

Build a Node-RED COVID statistics dashboard

Bonus: This flow includes a Node-RED dashboard with several gauges to display COVID-19 statistics.

• Every hour, the Node-RED flow calls the TWC COVID-19 Disease Tracker API and collects dynamic COVID-19 infection statistics.
• The country data is aggregated and then the gauges are updated.

Learn more about the dashboard code

The following Node-RED flow is included in this tutorial.

The TWC COVID-19 Country Report node uses the TWC COVID-19 Global Disease Country List API to retrieve the daily information for all countries with infections.

Each function node then aggregates the Total Confirmed Cases, Total Fatalities, Total Recovered, and Total Countries and sends the results to the corresponding gauge node.

This is the code in the function node:

let totalConfirmedCase = 0;

msg.payload.Countries.map(function(line){
    totalConfirmedCase += line.TotalConfirmed;
});

msg.payload = totalConfirmedCase;

return msg;

You can expand the above dashboard to include additional COVID-19 data visualization using the node-red-contrib-twc-covid19-tracker data to build charts, tables, maps and enhanced chatbot user experiences.

Build a SMS enabled Watson Assistant COVID Chatbot

Import this flow and Deploy the flow.

For this example, you need to install some additional dependencies:

• node-red-node-twilio

This flow requires a Twilio Number with a SMS Messaging webhook configured to POST to the URL specified by the HTTP In node.

For Twilio to retransmit the incoming SMS to this Node-RED endpoint, Node-RED needs to be reachable and hosted on the Internet (not on your local computer). Follow the instructions above to deploy Node-RED on IBM Cloud.

Here’s what the Node-RED flow looks like after you enable Twilio:

And here’s a sample chat message from Twilio using the TWC COVID-19 tracker API.

Build a Call for Code Crisis Communications solution!

Now that you have completed this tutorial, you are ready to modify these example flows and Node-RED Dashboard to build a Call for Code COVID Crisis Communications solution.