Solution Overview and Environment Setup on Bluemix
In this recipe, we will:
- connect a sensor to the Watson IoT Platform
- send the sensor data to a dashDB table
- write a Watson Conversation dialog that will allow us to ask the sensor questions in plain English about the temperature and light data it sends to the Watson IoT Platform
- write a Node-RED application that will wire the Watson IoT Platform, dashDB and Watson Conversation services and create a chatbot interface for the questions and answers
Here is an architectural overview of the solution:
Let’s create the Node-RED application that we will use for coding this solution, and connect all required services to it.
- First of all, let’s make sure we start with a fresh Bluemix Space. Click on the top-right menu next to your Bluemix account, and click Create a space. Call it RECIPES and click Create. This is the space we will use for this recipe.
- On the next page, click Create app. In the Search field, start typing Node-RED and select the Node-RED Starter Boilerplate. Enter Talk to your Sensor Your Initials in the App name field, and click Create. This will lead you to the app dahsboard main page. NOTE: do append your initials at the end of the name (e.g. CL) as if you use just the name Talk to your Sensor, you will get Someone already used the host name ‘Talk-to-your-Sensor’ error message. Note that in the rest of this recipe, we will refer to Talk to your Sensor.
- Go to the Connections menu of your app page. Note that as part of the boilerplate, a Cloudant NoSQL databse has been created. We will now add the followings services:
- Click Connect New, and select the Internet of Things Platform service. Edit the service name to your liking, and click Create. Click Restage.
- Repeat previous step by adding and connecting first a Watson Conversation service, then the dashDB for Analytics service to your app.
- Note that after creating the dashDB service, you will not be asked to restage the application, and will be redirected to your Bluemix main dashboard – do wait a couple minutes, then reclick on the Talk to your Sensor app link.
Your Talk to you Sensor app’s Overview page should now look like this:
Connect your IoT sensor to the Watson IoT Platform
This recipe assumes that you know how to register your sensor to the Watson IoT Platform, and how to send your sensor data to the Node-RED Debug output.
But for reference, the steps to follow to do this for the CC2650 sensor tag can be found in this recipe: Texas Instruments CC2650 SimpleLink™ Bluetooth® Smart BLE SensorTag
Before proceeding to the next step, you should have:
- registered the CC2650 (or another sensor) to the Watson IoT Platform,
- created and deployed the simplest 2-nodes flow in Node-RED (i.e. 1 IBM IoT input node connected to 1 Debug node – as per picture below) that sends your sensor data to the Debug tab.
TIP: whether you use CC2650 of another sensor, take good note of the msg.payload as it appears in the Debug tab. This payload will be our reference for creating the dashDB table in Section 3 and the Watson Conversation Entities in Section 4.
Create a dashDB table and populate it with your sensor data
Let’s now create the dashDB table that we will populate with the sensor data.
- From the Talk to you Sensor app’s Overview page, click on the dashDB fo Analytics service, then Launch it.
- In the dashDB interface, click on the Tables menu, then click Add Table
- Copy-paste the following DDL code, and click Run DDL. This creates the TIDATA table.
CREATE TABLE "TIDATA"
You can check the successful creation of the TIDATA table in the Tables menu: select TIDATA as the Table Name, and check the Table Definition.
TIP: if, instead of CC2650, you use another sensor that sends different data values to the Watson IoT Platform, you must update the above DDL to match the values that your sensor sends.
The picture below depicts the previous steps and what your table should now look like.
Now we are ready to populate the TIDATA table with the values sent by the sensor to the Watson IoT Platform:
- We will use a Node-RED flow to populate the dashDB TIDATA table. The TalkToYourSensor-dashDB-NodeRed.txt Node-RED file is available here: https://github.com/XopheLucas/TalkToYourSensor (as well as 2 other files we will use in Sections 4 and 5). In this GitHub repository, click on Clone or download – Download ZIP. Unzip, open the file TalkToYourSensor-dashDB-NodeRED.txt, and copy all its content.
- From the Talk to you Sensor app’s Overview page, open the Node-RED app by clicking the link next to the green Running icon. In your Node-RED editor menu, click Import – Clipboard, and paste the TalkToYourSensor-dashDB-NodeRed.txt code you just copied.
NOTE: Once you have imported this code, you must check and update 2 things in the Node-RED flow: (a) in the imported IBM IoT App In node, make sure that the All or tickbox is clicked, with the name of your Sensor in it (e.g. TISensorTagCL), (b) ensure that on the dashDB TIDATA node, the dashDB for Analytics Service name is selected (this is the service we created in Section 1, and connected to the Node-RED app) and the name of the Table corresponds to the one you created in the above step 3.
Each message your sensor sends is now formatted and being sent as a row in the dashDB TIDATA table. To check this: open the dashDB service, click the Tables menu and select the TIDATA Table Name, then Browse Data tab.
In this picture, see (1) what your nodes should look like (in the Debug tab, note the msg.payload as it is sent to dashDB), (2) how the TIDATA table has been populated with the sensor data.
Create the Intents, Entities and Dialog in the Watson Conversation service
We will now create all the Waston Conversation Intents, Entities and Dialog required.
If you are not familiar with the Conversation service, you might want to follow this tutorial on Watson Developer Cloud.
We provide you with 2 options to achieve this step: OPTION A (just import the ready-to-use Watson Conversation workspace), or OPTION B (do it all yourself).
OPTION A: Just import the workspace
- From the Talk to you Sensor app’s Overview page, click on the Watson Conversation service, then Launch it.
- The TalkToYourSensor-WatsonConversation.json corresponding to the conversation is available in the same repository you downloaded and unzipped in Section 3 (https://github.com/XopheLucas/TalkToYourSensor).
- On the Create Workspace page, click Import, and select the TalkToYourSensor-WatsonConversation.json file you just downloaded. Select Import Everything (Intents, Entities, and Dialog).
- Now test the whole flow. Click the Ask Watson icon (top-right) and Try it out by having a conversation as per the last picture in OPTION B of this section.
- That’s it ! You can now move directly to section 5 of this recipe …
OPION B: Do it yourself !
From the Talk to you Sensor app’s Overview page, click on the Conversation service, then Launch Tool, and login with your IBM ID.
- Create the Workspace, the Intents and the Entities
In this section, we will create the Workspace, the Intents and the Entities of the conversation.
For this recipe, we will only create 3 Intents, which will correspond to the ‘only’ conversation topics that Watson can answer to. The #Greetings and #developerWorks Intents will just be used to start the chat with Watson, whereas the #Environment Intent will contain those questions that will lead Watson to understand we want to know more about the data our sensor sends. The 2 Entities (@sensordata and @type) will be used to make Watson understand we want the max/min/average/last (@type) values of either light or temperature (@sensordata).
TIP: if your sensor sends data other than light or temperature, make sure you adapt the Intents and/or Entities to something relevant to the data sent – e.g. if it sends Humidity values, update the @sensordata values (and their synonyms) accordingly.
- On the Watson Conversation homepage, click Create Workspace, and name it Talk to your Sensor
- On the next page, click Create Intent, name the 1st Intent #Greetings, and add the 5 User Examples provided below for #Greetings. Click Create (top right).
- Repeat above step 2. with the other Intents (#developerWorks and #Environment) and associated User Examples provided below.
- Click on the Entities menu, and Create the 1st Entity: @sensordata, add 2 values (light and temperature), and for these 2 values, add the synonyms provided below. Click Create (top right).
- Repeat above step 4. for the @type Entity.
The 3 Intents and associated User Examples
1. Hi Watson
2. Good Morning Watson
3. Good Afternoon
4. Are you there? Pleased talking to you again
1. Can you tell me about developerWorks ?
2. Can we talk about IBM's DW recipes ?
3. What is developerWorks ?
4. What is DW ?
5. Tell me more about DW
1. Are you able to help me with sensor and IoT data info ?
2. Can you give me sensor information ?
3. Can you provide me with IoT info like temperature or light ?
4. Do you have IoT data available for me ?
5. What is my sensor saying regarding environmental values ?
The 2 Entities, their Values and associated Synonyms
light - synonyms: brightness, brilliance, luminosity, visibility
temperature - synonyms: temp, cold, hot, heat, warmth
maximum - synonyms: max, highest, biggest, top
minimum - synonyms: min, lowest
average - synonyms: avg, medium
last - synonyms: latest, newest, now, present, most recent, up to date
Your Intents and Entities pages should now look like this:
- Create the Dialog
- Open the Dialog menu, and click Create
- In the Triggered by field, start typing welcome, and select welcome (create new condition) from the list. In the Fulfill with a response field, enter Welcome to the Talk to your Sensor demo. Press Enter.
- Note that a node with the condition anything_else was automatically created in the dialog tree. Click on that node. In the Fulfill with a response field, enter My cognitive capabilities are presently limited. Feel free to boost my brain if you can, but for the time being, I can only answer questions relative to developerWorks, or to the MAX, MIN, AVG, LAST values recorded by a sensor for LIGHT and TEMPERATURE. This node will be called whenever Watson does not recognize the intent of your question.
- Select the + icon on the bottom of the welcome node. In the Triggered by field, start typing #Greetings and select #Greetings (create new condition) from the list. This condition will be triggered by any input that matches the #Greetings intent. In the Fulfill with a response field, enter How can I help?. Press Enter. In the Add a variation to this response field, enter What can I do for you?. Click Set to random next to the Response variations are sequential – that means that Watson will randomly select 1 of the 2 responses when recognizing the #Greetings intent.
- Select the + icon on the bottom of the #Greetings node. In the Triggered by field, start typing #developerWorks and select #developerWorks (create new condition) from the list. As in step 4., add the following random responses: developerWorks is a free web-based professional network and technical resource center from IBM where you can create Recipes and developerWorks provides tutorials, articles, recipes and other technical resources to help you grow your development skills on a wide variety of topics and products.
- Test your Dialog by clicking the Try it out (top right) and entering some questions – start by using those you defined in your #Greetings and #developerWorks intents, then by tweeking them a bit.
- Train Watson ! If you enter e.g. blablabla in the Try it out, Watson will provide you with the answer we defined in the anything_else node. If you want Watson to understand blablabla as a greeting, change the irrelevant intent to #Greetings in the Try it out tool. You will see a message saying that Watson is training on your recent changes. Once Watson is done training, re-enter blablabla and you will see that Watson will now treat this as a greeting.
- We will now create the special #Environment node:
- Select the + icon on the bottom of the #developerWorks node. In the Triggered by field, start typing #Environment and select #Environment (create new condition) from the list. This condition will be triggered by any input that matches the #Environment intent. Do not add anything in the Add response condition.
- Select the + icon on the right side of the just created #Environment node. Start typing @sensordata and select @sensordata (create new condition). Do not add anything in the Add response condition.
- Select the + icon on the bottom of the just created @sensordata node. In the Trigger field, start typing true and select true (create new condition). In the Add response condition field, type: OK, good idea. I understand you want more IoT sensor info. I can provide you with LIGHT or TEMPERATURE info.
- On the #Environment node, click the Jump to … icon, and select the @sensordata node and click Go to condition.
- At this stage, we ensured that the true node will be triggered each time Watson understands you are asking a question with an #Environment intent.
- Select the + icon on the right side of the just created @sensordata node. Start typing @sensordata and select @sensordata:light. In the Add response condition, type: OK, what do you want to know for LIGHT: its MAXIMUM, MINIMUM, AVERAGE or LATEST value ?
- Select the + icon on the bottom of the just created @sensordata:light node. Start typing @sensordata and select @sensordata:temperature. In the Add response condition, type: I understand you want more info on TEMPERATURE. What do you want to know: its MAXIMUM, MINIMUM, AVERAGE or LATEST value ?
- On the #sensordata node, click the Jump to … icon, and select the @sensordata:light node and click Go to condition.
- Select the + icon on the right side @sensordata:light node. Start typing @type and select @type (create new condition). In the Add response condition field, enter: OK, giving you the @type value for LIGHT.
- Select the + icon on the right side @sensordata:temperature node. Start typing @type and select @type (create new condition). In the Add response condition field, enter: OK, giving you the @type value for TEMPERATURE.
- Now test the whole flow. Click the Ask Watson icon (top-right) and Try it out by having a conversation as per the picture below.
- IMPORTANT NOTE: In case the conversation returns a How can I help? or What can I do for you? answer corrresponding to a #Greetings intent when you first enter the values max, min, avg, latest, just change the #Greetings intent to #Environment intent in the Try it out window, and Watson will train on it, and provide the correct answer the next time.
Create the Node-RED flow that glues everything together
We now have all needed elements in place, and can create the Node-RED flow that wil glue everything together.
We have created the Node-RED flow for you. Let’s import it:
- From the GitHub repository you downloaded and unzipped in Section 3 (https://github.com/XopheLucas/TalkToYourSensor), open the TalkToYourSensor-NodeRED.txt, and copy all its content.
- From the Talk to you Sensor app’s Overview page, click on the Talk to you Sensor app’s link, then go to your Node-RED flow editor.
- From the top-right menu, click Import – Clipboard, and paste the TalkToYourSensor-NodeRED.txt code you just copied. The Node-RED flow should look like the picture below.
NOTE: Make sure that you update the following elements in the Node-RED flow, so they map the exact environment that you set up:
- Double-click the Watson Conversation node, and change the Workspace ID field from Your-Watson-Conversation ID to the ID of the workspace you defined in Section 4. To find that ID, open the Watson Conversation service, click on the top-right menu of the Talk to your Sensor workspace, anc click View details – that should look like this: 10789797-c767-443b-9be8-802b4fb479fe
- In the 8 dashDB in nodes (e.g. LAST TEMP, MIN TEMP, LAST LIGHT etc), make sure that (1) in the Service field, dashDB for Analytics is selected (this will automatically be ‘tied’ to the dashDB DB service you created and connected to the Node-RED application in Section 1), (2) the table name (TIDATA) and fields (e.g. AMBIENTTEMP) correspond exactly to the ones you defined in Section 3.
Let’s explain the overall Node-RED flow idea, and detail some of the nodes:
- Setup the Chatbot page: here we use 1 http in node, 1 template node coded in mustache, and 1 http out node. Have a look at the mustache code, and especially see how this invokes Ajax to send the answers to your Chatbot output page.
- Capture the Conversation’s input and output: The Watson Conversation node just returns the output of the conversation as-is. Note in the Post Service node that we set a global variable called theoutput, which we will use later on to construct the final answer.
- Switch depending on the Conversation’s output: This is the key part of the flow. Here we basically determine what as-is answer Watson Conversation has returned. Note in the first 8 values of the switch that we capture whether the answer returned corresponds to the furthest-right nodes of the Dialog we defined in Section 4, i.e. whether the answer is either OK, giving you the @type value for TEMPERATURE or OK, giving you the @type value for LIGHT. In these cases, we will further refine the chatbot answer by doing SQLs. Otherwise (i.e. switch’s value 9), we simply return the answer as-is.
- Do SQLs and format when the Swicth identified the need, else return as-is output: If the switch has determined that the as-is answer returned by Watson Conversation corresponds to e.g. OK, giving you the @type value for TEMPERATURE (or LIGHT), then we: (1) query TIDATA using SQL to find the LAST or MIN or MAX or AVG value of either TEMP or LIGHT, (2) format the answer with the SQL returned value in the next function node (e.g. RETURN LAST TEMP) . Note how we finally construct the answer in the function node, i.e. see: globalvariable.botresponse.messageout.output.text.
WELL DONE !
You can now start playing with the Chatbot:
- Take the URL of your Node-RED flow (e.g.: https://my-talk-to-your-sensor.mybluemix.net/red), remove the /red and replace it by /bot at the end, e.g. https://my-talk-to-your-sensor.mybluemix.net/bot
- Click on that link, and have a conversation as per picture below:
Now that you are able to interrogate your sensor in plain English to ask what last/max/min/avg values it recorded, you might want to consider extending this solution:
- Add the Text to Speech Bluemix service to make Watson’s answers audible. Refer to this Watson Developer Cloud link or this Cognitive application using Watson Text To Speech Service recipe on developerWorks for detailed step-by-step instructions.
- Add the Speech to Text Bluemix service to ask your questions using your voice instead of the keyboard. NOTE: For these 2 services, as well as many other Watson services, Node-RED sample labs (with code) are available on GitHub at Watson NodeRED sample labs – highly recommended for anyone interested in deploying Watson applications on Node-RED !
- On top of just the last/max/min/avg values, consider extending the conversation and associated SQLs to be able to answer more detailed questions like: ‘What was the average value recorded between yesterday and today, by this 1 among many other sensors ?‘
- Use more advanced conversation possibilities, e.g. refer to Pattern for gathering information in IBM Watson Conversations or Recursive questions with IBM Watson Conversations
Where could this be used ?
The use cases that can be thought about using this ability to ‘Talk to your sensor’ are numerous. Here a couple examples:
- At home: ask what the average temperature in a room was during a given day, then adjust the start/end time of your heating or air-co depending on that. This could save costs …
- In any industrial environment (e.g. Mining) where Workers Health & Safety is key: what if in the case of a crisis situation, every member of an Operations Centre had the abilty to ask in plain English: ‘What was this Worker’s IoT helmet latest temperature, movement, CO2 level recorded‘ ? This could save lives …