Overview

Skill Level: Intermediate

You’ll build a chatbot that gives definitions of not only a given set of entities, but the whole world of Wikipedia. We will make external API callouts from Watson Assistant using Webhooks to get definitions for contextual entities from Wikipedia.

Ingredients

IBM Cloud Account

Watson Assistant

Cloud Functions

Step-by-step

  1. Setup Watson Assistant

    Let’s start with Watson Assistant. It’s going to be a very basic dialog tree like this:

    2

    Let’s do this step by step:

    1.    Go to IBM Cloud Catalog https://cloud.ibm.com/catalog and search for “Watson Assistant”

    2.    Make sure your Assistant is in the same Region as your Cloud Function

    3.    Click on “Create”

    4.    Start by clicking on “Launch tool”

    5.    Set up a new Skill by clicking on “Skills” –>  “Dialog Skill” –> “Create new”

  2. Train Intent

    Now that we have set up the chatbot, we want to teach it to understand our intention.

    1.    Click on “Create intent” and define a name for it (e.g. #tell_me_about). Click on “Create intent”

    2.    Add multiple different user examples to this intent:

    3.     Add user examples:

    • Please define gravity
    • Can you explain something about the International Space Station
    • Tell me something about Star Wars
    • what is love
    • give me the definition of literature
    • etc.
  3. Annotate Entities

    Now, we want to teach the chatbot to extract the correct object of interest. Therefore, we’ll use contextual entities.

    With contextual entities, you can define entities based on their context in the utterance. You do this by annotating the entities in your intent examples. Watson will use these annotations to learn how to identify the entity values and can match new values that have not been explicitly annotated.

    You simply highlight the part of the utterance, enter an entity name (let’s take @object_of_interest) and save.

    3

    Make sure your examples also include entities consisting of several words. Otherwise, the model will learn that the @objects_of_interest are one-word-objects only.

    If you look at the entities page, you can see the annotations, and the intents they were annotated in:

    4

  4. Set Up an IBM Cloud Function

    1. Go to IBM Cloud Catalog https://cloud.ibm.com/catalog
    2. Search “Functions”
    3. Click on Functions service
    4. Click on “Start Creating”
    5. Click on “Create Action”

    1

    6. Type your Action name [action_name], e.g. Wikipedia-Connection
    7. Create a package with any [package_name], e.g. Assistant-Functions
    8. Select your language. In that recipe, we will use Node.js
    9. Create your function
    10. We can now start editing the function!

    Hint:

    –> make sure to work on a Cloud Foundry-based namespace

    –> the Cloud Function and Watson Assistant instance need to be in the same region

  5. Edit Function

    Now, we can start setting up the Wikipedia Connection!

    We will send an API request to https://en.wikipedia.org/w/api.php  and will hand over the parameter ‘object_of_interest’, which is going to be defined in our Watson Assistant Workspace later. As a result, we receive a JSON file with the extract of our desired object.

    let rp = require('request-promise')
    function main(params) {
        const options = {
            uri: "https://en.wikipedia.org/api/rest_v1/page/summary/"+ encodeURIComponent(params.object_of_interest)+ "?redirect=true",
            json: true
        }
        return rp(options)
        .then(res => {
            return { extract : res }
        })
       }

    1. Don’t forget to save your action
    2. By clicking on “Invoke”, you can test your function and see the output in the console. However, if you invoke this action, you will get a result with “undefined” extract. That’s correct, as we haven’t handed over anything yet!

  6. Enable as Web Action

    Now, select Endpoints from the left hand side of the page and check Enable as Web Action and Save your changes.

    Copy down the URL that’s created under Web Actions, you’ll need this later.

  7. Configure Webhook in Watson Assistant

    Now that you’ve created your webhook it’s time to start connecting it to Watson Assistant. Note you’ll need the URL you copied the step before.

    Go to your Watson Assistant, choose Options>Webhooks on the left hand side and paste the URL into the input box.

    webhook_1

    Note that you need to add a “.json” to the URL so that Watson knows you’re returning a json snippet.

  8. Build Dialog

    Now, let’s create the conversational part. Navigate yourself to the Dialog section and click on Create Dialog. You will see how a basic dialog tree builds up.

     

    Create a new dialog node by clicking on Add Node.

     

    Fill in the condition:

    If assistant recognizes #tell_me_about && @object_of_interest

     

    To make use of the WebHook, activate WebHooks (via Customize button) and click Apply.

    webhook_apply

    Next, we need to define the parameter for the WebHook to make the callout:

    Key: object_of_interest

    Value: “@object_of_interest”

    webhook_callout

    As a return variable, we get $webhook_result_1 (as shown in the dialog node).

  9. Define Responses - I

    Below “Assistant responds”, let’s define several answer variations, from specific to general.

    Why? The Wikipedia API sometimes includes images as thumbnails which would be cool to include, but sometimes there’s text only. Moreover, some requests return “standard” responses, some “disambiguation” (when the question term refers to several topics) and some “not found”. In order to handle those, you can define different answer conditions.

    To return both an image and text in the first response alternative, click on the blue gear to customize the response:

    answers_all

  10. Define Responses - II

    First Answer – With picture

    Condition: $webhook_result_1.extract.thumbnail

    Answer (Text): I am defining @object_of_interest : <? $webhook_result_1.extract.extract ?>

    Answer (Image):

    Title: $webhook_result_1.extract.title

    Description: $webhook_result_1.extract.description

    Image source: $webhook_result_1.extract.thumbnail.source

    answer_image

    Hint: You can add the image by clicking on “Add response type” and choosing “Image” instead of “Text”. Define the parameters and don’t forget to hit “save”!

     

    Second Answer – Text only

    Condition: $webhook_result_1.extract.type==”standard”

    Answer: I am defining @object_of_interest : <? $webhook_result_1.extract.extract ?>

     

    Hint: Add more responses by clicking on “Add response” and move them up and down by using the arrows, sorting form specific to general.

     

    Third answer – Anything else

    Condition: anything_else

    Answer: I am not quite sure about the term @object_of_interest . Please try another topic.

     

    Great, let’s check our result!

  11. Check Result

    You can easily try out your assistant by clicking on “Try it” on the right side. Pose different questions and notice the result! Have a look at the structure $webhook_result_1 by clicking on “Manage Context” in the Try Out Pane.

     tryout_iss

    Note the different answer types as you ask for different definition terms – when asking for the ISS, you’ll get a pretty cool response!

     8

    Hint: You might need to add more training data to your intent and contextual entities in order to improve understanding.

  12. That's it - congrats!

    Awesome! You can now extend on this basic functionality or integrate the capability into your existing chatbots and voice bots. Check out further tutorials to learn how to create a voice-enabled bot and let your personal digital assistant give definitions whenever you need them :-)

    I’d be interested in more code patterns of Wikipedia API integrations, feel free to share your developments in the comments!

17 comments on"Connect Watson Assistant with Wikipedia API via Cloud Functions & Webhooks"

  1. sreddy4node June 20, 2019

    Thank you sophierm, for the wonderful article.

    I followed your instructions step by step. While testing the IBM Assisatnt i am getting 401 Error.

    To be Specific, the error is : {“cloud_functions_call_error”:”The supplied authentication is invalid”}

    I have given my valid credentials only, i.e Api key of my CF based name space.

    My CF based space is London.

    Please guide me in fixing this error.

  2. Hi, thanks for your comment! In which space is your Watson Assistant Service? Both WA and CF need to be in the same region + space. Maybe that’s causing the error?

  3. sreddy4node June 25, 2019

    Thank you so much Sophierm. My WA was in Wastington region and my CF was in London. Now after seeing your message i changed my WA to london and its working fine.

    Really it killed all my time with this small mistake.

    I have one more doubt.

    Can i call more than one cloud function in one dialog node?

    My scenario is user text his address and i will send that to my api and after i get the result i will ask him confirmation. When he say yes i need to call another api and no means another api.

    I have already agree intent. I will use this agree intent almost 6 times in my complete chat bot flow. How can I call all separate apis for each time the same intent or node is called. How can i call a CF with different condition in same Node?

  4. Hi sreddy,
    good question! To be honest, I would suggest to create separate nodes for the different API calls – you can define the condition for each child node, letting WA check the conditions of all sub-nodes (in your case, there should be a yes/no condition). It will make it easier to maintain if you design you dialog in a modular way. You can still use the same intent, but different nodes for different scenarios. Does that help?

  5. sreddy4node June 25, 2019

    Thank you sophierm. I will work on the model you said and will check. If i have any doubt again i will post here. If not you can provide any mail to conatct you. As i am new to ibm WA please guide me in fixing all my errors.

  6. raghav123456786 July 11, 2019

    Thank you sophierm, for the wonderful article.

    I followed your instructions step by step. While testing the IBM Assisatnt i am getting Dialog Nod Error.

    To be Specific, the error is : {“SpelEvaluationException when evaluating dialog node ID [node_3_1562873770144]. The syntax of condition [$response.extract.get(0)==”null”] is valid, but cannot be evaluated. Check that objects in expression are not null or out of bounds. SpEL evaluation error: EL1007E: Property or field ‘extract’ cannot be found on null (and there is 1 more error in the log)”}

    Please guide me in fixing this error.

    • raghav123456786 July 11, 2019

      and also while editing json in dialog section for tell_me_about , it shows the following error there as well – “Invalid JSON format, changes will not be permanently saved and propagated to the simple view”

      • Regarding the invalid JSON format, make sure you put all the required brackets (open and close) in order to save the code. If you copied and pasted my code example, make sure you deleted the content that was in there before replacing it with the new one.

    • It seems like Watson Assistant cannot find your object “extract”, which you will only receive if you set up the Cloud Function beforehand. Maybe the WA cannot build up the connection to your Cloud Function? You could edit the child node of your Cloud-Function-Dialog-Node (the one with the credentials, step 13/14) by giving out an answer that says $response.extract – so you see whether you actually get the response object of your Cloud Function.

  7. sreddy4node July 19, 2019

    Hi Sophierm,

    I have successfully built my own WA with cloud functions because of your article. Thank you so much again.

    I had 2 doubts please help me in this.

    1. Direct CloudFunctions call was not successful. Http response code is [400].

    How to handle this in the dialog node of WA. This is coming because my cloud function failed due to wrong input. In that condition how can i handle..

    In my dialog node i have implemented multiple response condition and validating the response from cloud function.

    if response.status== true ( i am building json object with in cf). My success

    anything –> Something went wrong , please try again.

    When ever my Cf is failed due to time out or 400, anything condition is not executing.

    Please help me how to handle this.

    2. Slots validations.

    I have created a slot with pattern for phone number with 10 digit.

    in my slot i implemented multiple conditions check also.

    i. if found and phonenum.length()==10 — success

    ii. if found and phonenum.length()<10 — phone num should be 10 digits.

    my ii condition is not working if i enter less than 10 digits and it is not recognizing the particular node. If it is not found it should display the not found message i kept, it is not showing not found message also.

    Thank you,
    S Reddy.

  8. carlos.gaona August 08, 2019

    Hi sophierm,
    What is the right way to connect Watson with a Cloud Function via IAM Access? I created my own function that was working fine with user/password access in the past.
    Now, I’m migrating to a new account and the Function is now created in a “namespace” that doesn’t allow api_key to be seen directly (user:pswd). What is the right way to do it?
    1. How can I generate the function again to provide the classical “user:password” api_key?
    2. Or, how do I configure Watson’s JSON to get the right credentials?
    Thanks in advance!

  9. Muhammad Usmann640 September 04, 2019

    Hi all I am a Student and working on a chat bot, I just want to know that when we book an appointment with someone how does the chatbot know the requested time is available for appointment or not ? by using some cloud function or by using google calendar?

    Please help me out.
    thanks

  10. I am getting this error in the log whenever I say something belongin to #tell_me_about assistant. please help me resolve the issue.

    SpelEvaluationException
    when evaluating dialog node ID [response_6_1578123616767]. The syntax of condition [$response.extract.get(0)==”null” || $response.extract.get(2).get(0)==””] is valid, but cannot be evaluated. Check that objects in expression are not null or out of bounds. SpEL evaluation error: EL1029E: A problem occurred when trying to execute method ‘get’ on object of type ‘JsonArray’: ‘Invalid index 0 for the JsonArray []’ (and there is 1 more error in the log)

  11. please reply soon

  12. Suraj_Jeswara April 20, 2020

    That was a great tutorial. I could successfully make a chat bot with the Wiki API integration. Thanks!!!

  13. Nicolas G June 07, 2020

    Hi there,

    thanks for the tutorial. So far I was not able to make it work as I get the following error when trying the bot “Exception when evaluating dialog node ID [response_4_1591527601410]. Condition [$webhook_result_1.extract.type==”standard”] evaluated to FALSE. java.lang.IllegalStateException: Cannot handle (8221) ‘”’ “. Any input what could be the issue here?

Join The Discussion