Overview

Skill Level: Any Skill Level

This recipe describes how to teach Watson using a training classification file (on Dropbox), classify and analyze a given text message, then output the results with the highest confidence level.

Ingredients

Prerequisites to do this recipe:

  • Bluemix Account (or sign up on bluemix.net)
  • Dropbox Account (or sign up on dropbox.com)
  • Web Browser - Safari or Firefox recommended

Application Flow to build:

  • Train Watson Natural Language Classifier (NLC)
    • Setup Dropbox access
    • Create a classification file on Dropbox
    • Train Watson using the created classification file
  • Classify Free-Form Text
    • Classify a text message using the trained classifier
    • Further analyze and extract categories out of the same text message
  • Apply Simple Rule
    • Compare results from the classifier vs category
    • Display the highest confidence level

step-0-overview-1

Watson Services to use:

Contributors - from the IBM Dallas Global Solution Center:

 

Important Notes:

  • This recipe shows you how to analyze simple text messages. It is possible to execute the same analysis on text extracted from social media such as billions of tweets. When working with big data sources, an important step is to filter relevant inputs, analyze the language, analyze the location and finally pre-process the data before further analysis. These tasks are out of scope of this recipe, which focuses on the combination of cognitive services, but they can also be implemented using additional services on Bluemix.
  • The IBM Bluemix platform is expanding and improving its functionality very rapidly. As a result, there are occasional changes to the look and feel of the IBM Bluemix website or short interruptions in some of the services. Throughout this recipe, you will be guided by a series of screenshots, but do not be alarmed if your IBM Bluemix session does not look exactly like the screenshot provided.

Step-by-step

  1. Create a Node-RED application

    Log in to Bluemix, select Node-RED Starter from the Catalog > Boilerplates.

    node-red-starter-1

    Specify a unique App name which will automatically populate a URL (Host name + Domain).  Click Create.

    node-red-app-name

    Wait for the app to start then launch the URL from View app (or Routes) dropdown

    node-red-url

    The app will be launched in a new browser tab.  Click the Go to your Node-RED flow editor button to bring up the Node-RED flow editor.  Note: if prompted to secure Node-RED, you may choose to enter the user ID / password to be used to login to Node-RED.

    go-to-node-red

  2. Access a sample NLC training file on Dropbox

    Some 3rd Party functionality such as integration with Dropbox may not be available out of the box on Node-RED.  So we will first add dropbox nodes to your Node-RED palette

    Select Manage palette from a dropdown menu

    manage-palette

    Select Install tab, search for dropbox and click install

    install-dropbox

    If the status of dropbox node does not change to “installed“, restart the app from Bluemix Dashboard then relaunch your Node-RED app.

    Locate the dropbox nodes under storage section, drag the Dropbox input node to the canvas

    dropbox-add-1

     Double click to open the Dropbox node then edit it

    dropbox-config

    Follow instructions in Appendix 1 to configure access and acquire credentials to your Dropbox location (also guided in the yellow section shown below).  Then return here to enter those credentials.

    dropbox-key

    Follow instructions in Appendix 2 to create and upload a train class file to Dropbox.  Then return here to enter your training class file name (e.g. NLCTrainClasses.csv) and give Dropbox node a name (e.g. Class File on Dropbox).

    dropbox-edit-1

    Drag a debug node (located under output section) to the flow and wire the nodes together.

    dropbox-debug

    Drag an inject node to the flow and name it Load Class File.

    inject-add

    Wire the inject node to the rest of the flow and rearrange it.  Then Deploy the changes.

    inject-deploy

    Test the flow by clicking on the button to the left of Load Class File node to trigger the flow to run.  Notice the file content loaded is being displayed in a debug panel.

    load-file-debug

     

     

  3. Create Watson services on Bluemix

    From your Bluemix browser session, navigate to IBM Bluemix Catalog, select Watson Services category and locate Natural Language Classifier and Natural Language Understanding services.

    step-3-1

    Click Natural Language Classifier to bring up a service creation page.  Take all defaults then click Create.

    nlc-create-1

    Repeat the steps similar to above to create Natural Language Understanding service.

    Navigate to the list of service instances view by clicking on the hamburger icon and select Services and Dashboard.

    service-menu

    You should see both Natural Language Classifier and Natural Language Understanding services instance listed.  Note: if there are more than one of each, you may choose to keep all or use the existing one(s) and delete the newly created one(s).  Click on the Natural Language Classifier instance to view detail.

    step-3-0-list

    Click on Service Credentials and View Credentials.  Copy the username and password values for use in the later step.

    nlc-credentials

    Similar to the above steps, view Natural Language Understanding – Service Credentials and copy the username / password values for use in the later step.

    step-3-NLU-credentials

  4. Train Watson NLC

    From Node-RED, locate natural language classifier node under IBM_Watson group:

    nlc-node-red

    Drag and drop the natural language classifier node to the canvas and double click to open and edit it.  Enter NLC: Train: for the Name and make sure Mode is set to Train. Paste the NLC service credentials saved earlier to Username and Password fields.

    nlc-train

    Rename the Load Class File inject node to Train Watson:

    inject-rename

    Update the debug node to output complete msg object:

    debug-msg-obj

    Add a comment node to self-describe the flow (e.g. Train Watson NLC).  Rearrange and rewire the flow to look like the figure below.  Then Deploy the changes and clear debug messages by clicking the trash icon under the debug tab.

    nlc-deploy

    Run the flow by clicking the button to the left of the Train Watson inject node.

    train-watson-1

    Notice the log in debug panel, expand to view detail (e.g. classifier_id and the class file name):

    nlc-debug-detail

    Review the progress of the training process on Bluemix.  Open Watson NLC service instance and click Access the beta toolkit button:

    nlc-bmx 

    If prompted to Login, simply click Sign in with Bluemix.  The Classifiers page will show progress of the run with timestamp.  Upon completion, copy Classifier ID with the latest timestamp (in case there are more than one) for use in the later step.

    classifier-id

     

     

  5. Classify and analyze a text message

    Add a new set of nodes

    Drag and drop the following nodes: comment, inject, natural language classifier, natural language understanding and debug nodes.  Edit node names and wire them together as shown below.  Set the debug node for Natural Language Understanding (NLU) to output  msg.features.

    Step-5-1

    Configure NLC: Classify node

    Make sure Mode is set to Classify.  Enter the Classifier ID from the previous step.

    nlc-classifier-id

    Configure NLU node

    Enter Username and Password from early step, select features to extract including Categories (to categorize content against known taxonomies) and a few others such as Entities and Keywords.  For information about the features, refer to the info tab next to debug.

    Step-5-2

    Open the Text Message inject node, select String from Payload dropdown and enter a short text message to be classified (e.g. I beat 5 pokemons at central park last night or I went to the park at night to battle the gym):

    nlc-input-text

    Deploy the changes and clear debug messages.

    Run the flow by clicking the button left to Text Message node.

     Step-5-3-0-2

    From the output in debug panel, expand and review confidence level / score from NLC and NLU results for the given text message.

    Step-5-3

    Optionally, you may try with a different message or update the class file, retrain Watson, and retry with the same message.

  6. Compare results and take next steps

    Now that you are getting results and confidence levels from both NLC and NLU, there are many additional actions you might take to address your specific needs. For example, if the confidence levels on NLC classification results are consistently low for the texts that you are processing while NLU’s category results offer higher confidence levels, you might decide to retrain your NLC classifier. Alternatively, you might choose to programmatically determine the highest confidence level when comparing NLU results with NLC results. To illustrate this idea, the steps below will show how you could add a simple rule to compare results from NLU and NLC, and output the result that has the highest confidence level.

    Add function node to the flow and change NLU debug node to display msg.payload.

    Step-5-function-nodes

    Edit Highest Confidence Class node to set a global variable and return the class with the highest confidence level (first array) or simply copy and paste the code below:

    global.set(“classifier”, msg.payload);
    var classifierResult = global.get(“classifier”);
    msg.payload = “Class: ” + classifierResult.classes[0].class_name + “, Confidence: ” + classifierResult.classes[0].confidence*100 + “%”;
    return msg;

    Edit Highest Score Categories node to set a global variable and return the taxonomy with the highest score (first array) or simply copy and paste the code below:

    global.set(“features”, msg.features);
    var NLUResult = global.get(“features”);
    msg.payload = “Category: ” + NLUResult.categories[0].label + “, Score: ” + NLUResult.categories[0].score*100 + “%”;
    return msg;

    Deploy the changes and clear debug messages. 

    Run the flow by clicking the button left to Text Message node.  Notice only one value from each with the highest confidence / score is displayed in debug panel.

    Step-5-4-2

    Apply a simple rule to compare the confidence / score values from the global variables.

    Add comment, inject, function, and debug node.  Name them as shown below.

    simple-rule

    Edit Check for Highest function node to compare the values and return the result or just copy and paste the code below:

    var NLUResult = global.get(“features”);
    var classifierResult = global.get(“classifier”);
    if (NLUResult.categories[0].score > classifierResult.classes[0].confidence) {
    msg.payload = “The category (” + NLUResult.categories[0].label + “:”+NLUResult.categories[0].score+”)  has higher confidence than the classifier (“+classifierResult.classes[0].class_name + “:” + classifierResult.classes[0].confidence + “)”;
    } else {
     msg.payload = “The classifier (” + classifierResult.classes[0].class_name + “:”+classifierResult.classes[0].confidence + “) has higher confidence than the category (“+ NLUResult.categories[0].label + “:” + NLUResult.categories[0].score + “)”;
    }

    return msg;

    Deploy the changes and clear debug messages. 

    Run the flow by clicking the button left to Apply Rule node. 

    Step-6-5-0-2

    The correct result should be displayed in debug panel.

    Step-6-5

    Congratulations!  You have just completed this recipe.

  7. Appendix 1: Make Dropbox accessible from Node-RED

    Go to https://www.dropbox.com/developers to build an app on the Dropbox platform.  Sign in using your Dropbox account and click Create your app.

    create-app

    Select Dropbox API, type of access (e.g. App folder), name your app (e.g. to be the same as your Bluemix/Node-RED app – Dropbox will also create a folder with this name under Dropbox > Apps folder), accept terms and conditions, then click Create app.

    create-app-2-1

    The subsequent page contains the App key. Click Show to view the App secret and Generate to produce a suitable access token.

    app-keys

    Take note on these key values (App key, App secret, Generated access token).

    app-keys2-3

    Go to dropbox.com to verify that a new folder is created under Dropbox > Apps folder with the same name as your app.  This is where you will be uploading a file to in order for it to be accessible from Node-RED.

    dropbox-folder

  8. Appendix 2: Create and upload a training class file

    Download a sample NLCTrainClasses.csv file from this link: https://ibm.box.com/s/ywmcbbkf1a1zin0hmiwyc57v3857q1yj

    csv

    In this example, the first column contains sample text messages people might be talking about around the game ‘Pokemon Go” and a class (second column) each sample message should be associated with.

    Feel free to explore or edit the file with more sample messages and / or classes.

    Upload (drag and drop) your version of the NLCTrainClasses.csv file to your Dropbox > Apps > <folder> location.

    drop-a-classfile

    You are now ready to train Watson using this training class file.

     

  9. Appendix 3: Change History

    June 9, 2017

    • Replaced Watson AlchemyAPI service with Watson Natural Language Understanding
    • Minor updates to reflect Bluemix changes, e.g. securing Node-RED, Routes vs View App

2 Comments on "Train Yourself on How to Teach a Cognitive Engine like Watson"

  1. Andrewarren June 02, 2017

    Thanks for this tutorial!

    Is the Beta Toolkit no longer available?

    Cheers,
    Andrew

    • Hi Andrew – the Watson AlchemyAPI service is no longer available from the catalog (will still run if you already have it created). It is being replaced with the new Watson Natural Language Understanding. To reflect this change, we’ve just updated the instructions and screenshot using the new service.

      Thanks,
      Tass

Join The Discussion