Overview

Skill Level: Beginner

Simply copy-and-paste and follow some directions, to create your first SMS Chatbot

Want to use the easy Watson Conversation service and share your invention? This recipe will help you connect Watson Conversation with SMS text messaging.

Ingredients

Bluemix account

Step-by-step

  1. Create your Node-RED application

    Your Node-RED application is where you’ll put your application’s logic. It will receive SMS text messages from Twilio, and call the Watson Conversation service. 

    1. Log into Bluemix

    2. Create new application from the Node-RED Starter Boilerplate

    3. (Optional, for security) Update the username and password with Environment Variables

  2. Add the Watson Conversation service

    For reference, we are using this Watson Conversation demo. In this demo, imagine you’re in the driver’s seat and Watson is your co-pilot. Watson can understand your entries and respond accordingly.

    1. Go to your Node-RED application in Bluemix

    2. Add the Conversation service…Bluemix will probably ask you to restage the application. 

    3. Import the Workspace, using these instructions. Make sure to copy your Workspace ID.

    You won’t need the Credentials, because of Watson Conversation integration in Node-RED on Bluemix

  3. Set up your Twilio SMS service

    Twilio provides telephone services over APIs. You’ll use Twilio to send and receive SMS text messages. 

    1. In Bluemix, connect a new Twilio account (or your existing one). Follow directions to connect credentials. 

    2. Get a Twilio phone number. You don’t need to pay for the account yet… you’ll just see that Twilio adds a message into your texts

    3. Set up your Twilio account to forward SMS messages to your Bluemix service /twiliosms

    Twilio-Messaging

    (Hat tip to these instructions)

     

  4. Import the Node-RED flow

    This Node-RED flow is the glue which connects all your services. 

    1. Copy the code below (it’s longer than the box you see)
    2. In Node-RED, open the menu in the upper-right corner
    3. Select Import, then Clipboard. 
    4. Paste in this JSON

    Some people have trouble if the quotation marks are curly, instead of the plain-vanilla straight quotes. Please respond in the comments if you are having trouble with this step after August 29th 2016. 

    This flow is updated October 22nd 2017 to fix issues which caused Twilio to throw error type 11200. 

    [{"id":"337f80d0.e51418","type":"debug","z":"b7e09aa9.bb171","name":"Request Received","active":true,"console":"false","complete":"payload","x":389,"y":73,"wires":[]},{"id":"c04f47b0.a09808","type":"http in","z":"b7e09aa9.bb171","name":"","url":"/twiliosms","method":"get","swaggerDoc":"","x":173,"y":135,"wires":[["337f80d0.e51418","d9fb8a2b.6e66c"]]},{"id":"d9fb8a2b.6e66c","type":"function","z":"b7e09aa9.bb171","name":"ReceivedDate","func":"d = new Date();\n\nmsg.payload.ReceivedDate = d.valueOf();\nmsg.payload.ReceivedDateStr = d.toString();\n\nreturn msg;\n","outputs":1,"noerr":0,"x":379,"y":137,"wires":[["f096212d.cade9","546c66d0.8e6d1","d5b699be.e1907"]]},{"id":"f096212d.cade9","type":"debug","z":"b7e09aa9.bb171","name":"Outbound to Log","active":false,"console":"false","complete":"payload","x":616,"y":93,"wires":[]},{"id":"546c66d0.8e6d1","type":"cloudant out","z":"b7e09aa9.bb171","name":"log-requests","cloudant":"","database":"log-requests","service":"","payonly":true,"operation":"insert","x":595.5,"y":135,"wires":[]},{"id":"d5b699be.e1907","type":"function","z":"b7e09aa9.bb171","name":"Prepare Request for Conversation","func":"msg.params = {};\nmsg.params.context = {}\nif (flow.get(msg.payload.From)){\n msg.params.context = flow.get(msg.payload.From); \n}\n\n\nflow.set ('From', msg.payload.From);\n\nmsg.payload = msg.payload.Body;\n\nreturn msg;","outputs":1,"noerr":0,"x":557,"y":242,"wires":[["9360139d.f1c208","1f130a9c.98cab5"]]},{"id":"9360139d.f1c208","type":"debug","z":"b7e09aa9.bb171","name":"Conversation Request","active":true,"console":"false","complete":"true","x":878,"y":190,"wires":[]},{"id":"1f130a9c.98cab5","type":"watson-conversation-v1","z":"b7e09aa9.bb171","name":"Conversation","workspaceid":"","x":850,"y":243,"wires":[["284242df.e62d46","83563003.39681","6d6e1072.3ad238","167b02cc.c15c0d","306aca55.cf9536"]]},{"id":"284242df.e62d46","type":"debug","z":"b7e09aa9.bb171","name":"Conversation Response","active":true,"console":"false","complete":"payload","x":1142,"y":176,"wires":[]},{"id":"83563003.39681","type":"debug","z":"b7e09aa9.bb171","name":"Conversation Response.output","active":false,"console":"false","complete":"payload.output","x":1162,"y":218,"wires":[]},{"id":"6d6e1072.3ad238","type":"debug","z":"b7e09aa9.bb171","name":"Conversation Response.input","active":false,"console":"false","complete":"payload.input","x":1165,"y":253,"wires":[]},{"id":"167b02cc.c15c0d","type":"function","z":"b7e09aa9.bb171","name":"Output Text and Context","func":"var From = flow.get('From');\nmsg.topic = From;\nflow.set(From, msg.payload.context);\n\nmsg.payload = msg.payload.output.text.join();\n\nif (!msg.payload) {\n msg.payload = \"Something went wrong.\";\n flow.set(From, {});\n}\n\nreturn msg;","outputs":1,"noerr":0,"x":1163,"y":411,"wires":[["998cb989.47508","225ca04c.3984d","194b7b8c.e6b484","f4ec3f28.0b13c","99301ce4.57bc8"]]},{"id":"306aca55.cf9536","type":"debug","z":"b7e09aa9.bb171","name":"Conversation msg","active":false,"console":"false","complete":"true","x":1123.5,"y":296,"wires":[]},{"id":"998cb989.47508","type":"debug","z":"b7e09aa9.bb171","name":"Responded via Twilio","active":false,"console":"false","complete":"true","x":1412,"y":348,"wires":[]},{"id":"225ca04c.3984d","type":"twilio out","z":"b7e09aa9.bb171","service":"Twilio-live","twilio":"","from":"","number":"","name":"","x":1377,"y":452,"wires":[]},{"id":"194b7b8c.e6b484","type":"debug","z":"b7e09aa9.bb171","name":"msg.payload","active":true,"console":"false","complete":"payload","x":1384.5,"y":384,"wires":[]},{"id":"f4ec3f28.0b13c","type":"debug","z":"b7e09aa9.bb171","name":"msg.topic","active":false,"console":"false","complete":"topic","x":1373.5,"y":418,"wires":[]},{"id":"99301ce4.57bc8","type":"http response","z":"b7e09aa9.bb171","name":"Respond to original GET","x":1425,"y":497,"wires":[]}]
  5. Make final tweaks before running

    1. In the log-requests node, change the Service name to your Cloudant service

    2. In the Conversation node, change the Workspace ID to your Workspace ID

    3. In the Twilio-live node

    • Change the Service to your service
    • Change the From to your Twilio phone number. The format must include + and country code. For example, in the US, +12125551212 not (212) 555-1212.
    • Leave the SMS to blank. The code will make sure people get their reply.
  6. Test it out!

    Try sending a text from your cellphone to your Twilio number. You should get back a conversational response from Watson. When you reply, Watson will carry on a conversation!

    If things aren’t working perfectly, turn on the Debug nodes. You can also create a new Inject node to create fake text messages. 

  7. Recommended next steps

    Now that you have a fully working SMS Chatbot, I suggest you try giving it a more interesting brain. Go into your Watson Conversation service and create a new Workspace. Check out how easy it is to train Watson – simple configuration, no coding. Once you’re happy with the new Workspace, just change the Workspace ID in your Node-RED flow, and press Deploy!

32 comments on"Watson Conversation over SMS"

  1. Would you check that JSON text? I couldn’t import it on Node-RED.

  2. This is a great tutorial marred by some confusing instructions. #3 of Step 3 really throws a wrench in the experience. The blog post has very good information but I spent over an hour trying to comprehend it. Ultimately it turned out that I did not have to do anything suggested.

    I was able to import the given JSON (which is really really useful) and just had to use the http://my-node-red-app.mybluemix.net/twiliosms URL on TWILIO.

    Please update the instructions so others don’t go down rabbit holes.

    • danrohtbart October 31, 2016

      Thanks for taking the time to give feedback – this is tremendously helpful feedback. I’ll try to update that section later this week, and I’ll notify you when it’s updated.

  3. This tutorial is amazing, however #3 is extremely confusing. I am having trouble finding my /twiliosms url and where I put it. It appears Twilio has done a huge revamp on their website since the time of this tutorial. #1-2 are fairly self explanitory and I have connected the corrected to my application. However when I send a text message to the Twilio phone number I get their default demo text.

    • Thanks so much for the compliment and feedback. I’ve updated #3 with a screenshot of the new Twilio website. Is this enough help? Or is there something else that could be more helpful?

  4. I do not see “In Node-RED, open the menu in the upper-right corner” . Where do I find the import button for step 4?

  5. YuriNasci May 19, 2017

    Hello, I tried Deploy it, but show de message “Deploy Failed: Not Authorized”.

  6. ChrisBurtonZephyr June 30, 2017

    Hello, I was hoping you could help me out? After configuring my Twilio account SID, phone number etc, I’m getting a couple of errors from within “Prepare Request for Conversation” bubble… they are as follows:
    – Expected an identifier and instead saw “&”
    – Expected an assignment of function call and instead saw an expression
    – Missing “;” before statement
    – Expected an identifier and instead saw “#”
    – Missing semicolon
    – Expected “)” and instead saw “#”
    Too many errors. (100% scanned)

    • ChrisBurtonZephyr June 30, 2017

      This is the block of code that’s in the “Prepare Request for Conversation” bubble which came from this article…
      msg.params = {};\nmsg.params.context = {}\nif (flow.get(msg.payload.From)){\n msg.params.context = flow.get(msg.payload.From); \n}\n\n\nflow.set ('From', msg.payload.From);\n\nmsg.payload = msg.payload.Body;\n\nreturn msg;

  7. ChrisBurtonZephyr June 30, 2017

    Yup sure am thanks to your tutorial I’ve got a chatbot demo through Twilio. The only thing that I now have a problem with is multiple users and the msg.user. Do not know how to implement the the required code to grab this and pass it through to the conversation service… Would you happen to have done this before?

    • danrohtbart July 14, 2017

      Can you describe the problem you’re seeing? This behavior should be working in the existing code. Each SMS user’s context is in the flow object, as a key-value pair, with the SMS number as a key. That context is get/set in the Javascript functions before and after sending to the Conversation service.

  8. RaghavJaggia August 01, 2017

    Hi,
    I have followed all the steps accordingly, but when I send a text to the number, I am not getting a response back.

    When I deploy the application on node_red, I am getting a successful deployment message.

    • danrohtbart August 08, 2017

      Hi @RaghavJaggia, sorry to hear you’re still getting a problem. When you send a text to the number, please check Twilio. Are you seeing evidence that Twilio received the text? Once you’ve confirmed that Twilio is receiving correctly, please double-check the precise URL you are using in the webhook configuration in Twilio /twiliosms matches precisely the endpoint that your node_red is listening on.

  9. Hi, I am using my own custom conversation and I followed all the steps you outlined. Everything works fine and I am getting the correct message response. Except after about 1 min another SMS is automatically sent with the same message. What’s causing this and how can I fix it? Twilio is showing an error 11200 HTTP retrieval failure.

    • I’m seeing this issue as well and have not been able to fix it. Any luck on your side?

      • danrohtbart October 23, 2017

        Great news – an easy fix. In Node-RED, find the Output nodes. Add an HTTP-Response node to the end of your stream. This will successfully respond to the incoming request, which will stop the 11200 error.

  10. danrohtbart October 09, 2017

    A Twilio developer evangelist suggested what’s happening here: When you use webhooks, Twilio expects your page to return TwiML, and in case your page doesn’t return it, you will get intermittent 1200 retrieval errors. https://stackoverflow.com/questions/38469738/twilio-11200-http-retrieval-failure-what-does-this-mean

    I’m getting the error as well, but it doesn’t seem to break the functionality of the chatbot. Does your chatbot still work?

    • danrohtbart October 23, 2017

      Updated Step 4 in the recipe to address Chris@penc and AVargas’s issue.

      • im getting the same error http 11200 from twilio even after updated step 4 in the recipe. The flow who has been updated on October 22nd 2017 does not seem to fix issues which caused Twilio to throw error type 11200.

        • Dan is right, the flow who has baeen updated on October 22nd 2017 fix issues which caused Twilio to throw error type 11200. Don’t forget in your twilio settings to change http POST to http GET. Thanks Dan to help me figured out.

          • danrohtbart October 31, 2017

            I’m so happy that you were able to resolve the issue and get started!

  11. It looks like my text is getting to Twilio but I’m getting an error code from Twilio that says the following:” Schema validation warning. The provided XML does not conform to the Twilio Markup XML schema. Please refer to the specific error and correct the problem. Causes include: misspelled verbs, incorrect case for verbs; TwiML is case-sensitive, so works, but will not misspelled or unknown attributes unknown or unexpected nested elements.”

    Under the Request Tab I see:

    Message Text
    Message-TextShow Raw
    Msg-Content is not allowed in prolog.
    line 1
    sourceComponent-14100
    parserMessage-Content is not allowed in prolog.
    ErrorCode-12200
    EmailNotification-false
    cold-1
    LogLevel-WARN

    My actual message text says: Hello! My name is Biz Bot. I’m an informational chat bot who knows a lot about starting businesses. Here are some questions you can ask me, “How do I file a DBA?”, “What taxes do I collect?”, “What is a corporation?”.

    My last question is do I have to create an application or can I just use what I have already created in Watson Conversation Service?

    Thanks — Elizabeth

    • danrohtbart October 25, 2017

      Hi Elizabeth, happy to help. First, perhaps those quotation marks are causing a problem? Second, what is the entire msg object that you are sending to Twilio? (you can attach a Debug node, and please share the whole msg object, not the default msg.payload). Finally…not sure about your question about creating an application…you do need a Node-RED boilerplate application (Step 1) as well as a Watson Conversation Service (Step 2).

Join The Discussion