Archived content

Archived date: 2019-05-21

This content is no longer being updated or maintained. The content is provided “as is.” Given the rapid evolution of technology, some content, steps, or illustrations may have changed.

Note: Watson Conversation is now Watson Assistant. Although some images and text in this how-to may refer to the service as Watson Conversation, the steps and processes will still work.

In order to develop conversational experiences, it’s important to understand how users are actually using bots. Based on this data, conversational experiences can continuously be improved.

IBM Watson Conversation comes with an improve component that helps analyze and improve bots. Dependent on the service plan, all conversations from the last 7, 30 or 90 days are stored. Via the Watson Conversation web interface, you can get an overview of the interactions between users and bots such as the traffic at a specific date. You can also improve how Watson understands user inputs by adding or changing intents and entities.

Learning objectives

This how-to explains the way to set up and use the community tool Conversation Optimizer for IBM Watson to provide various views and find conversation flows that have potential issues and might require improvements in addition to the Watson Conversation improve component.

The Conversation Optimizer for IBM Watson provides the following views to find potential conversation issues:

  • Conversations with out of scope user input
  • Conversations with unfulfilled intents
  • Conversations with low confidence levels for intents
  • Conversations with dialog errors

Screenshot of the Conversation Optimizer:

Conversation Optimizer for IBM Watson

Check out the screenshots folder for more screenshots.

Prerequisites

In order to use this how-to, you need the following prerequisites:

Estimated time

The set up of the Conversation Optimizer shouldn’t take longer than 10 minutes. Understanding how to use the Optimizer shouldn’t take longer than 10 – 20 minutes.

Steps

Installation of Conversation Optimizer

The easiest way to deploy the Conversation Optimizer to IBM Cloud is to press the Deploy to IBM Cloud button below.

Deploy to IBM Cloud

The initial deployment takes roughly five minutes. You can check the delivery pipeline for status information. After the deployement has been completed, you’ll find a new Cloud Foundry application ‘conversation-optimizer-for-ibm-watson’ in your IBM Cloud dashboard.

Binding of the Conversation service to the deployed Optimizer application

You need to bind a Watson Conversation service to the Cloud Foundry application. You can bind the service to the application either via the IBM Cloud user interface or by using the following commands.

cf login
cf services
cf bind-service conversation-optimizer-for-ibm-watson my-conversation-service-name

After the Cloud Foundry application has been restarted, you can launch the configuration page and choose the Watson Conversation workspace. The URL will be similar to https://conversation-optimizer-for-ibm-watson-xxx.mybluemix.net/home with a unique value for xxx. You can find the URL in your IBM Cloud dashboard.

In order to run the Conversation Optimizer locally, check out the instructions on GitHub.

Usage

After you’ve bound your Watson Conversation service and chosen the workspace you can read the conversation logs from Watson Conversation. Depending on the number of conversations, this can take a couple of seconds or even minutes. After this, you can open the following views with conversations that potentially require improvements.

Out of scope

The following is an example of Out of Scope input:

*Watson:* Hi. It looks like a nice drive today. What would you like me to do?
*User:* Who is Niklas?
*Watson:* I’m not sure about that. You can say things like “Turn on my lights” or “Play some music.”

In order for this filter to work the dialog needs to be changed by adding ”output.outOfScope” : ““ to the nodes where the user input is out of scope. These are typically the nodes with the anything_else condition.

Check out the screenshots to see how this is displayed in the Conversation Optimizer and configured in Watson Conversation:

Unfulfilled intents

The following is an example of Unfulfilled Intents:

*Watson:* Hi. It looks like a nice drive today. What would you like me to do?
*User:* turn on music
*Watson:* Sure thing! Which genre would you prefer? Jazz is my personal favorite.
*User:* Neue Deutsche Welle
*Watson:* Sorry I don’t recognize that genre. Want to try something? Jazz? Rock?

This filter also requires changes to the dialog definition. Developers can mark intents as being fulfilled when certain nodes in the dialogs are reached. In this way, intents that have been triggered but not fulfilled, can be detected. For this feature to work, you need to add information to the JSON of your dialog nodes:

  • Add "output.intentTriggered” : “your-intent-name" to the node where the intent is triggered
  • Add ”output.intentFulfilled” : “your-intent-name” to the node where the intent is fulfilled

Check out the screenshots to view how this is displayed in the Conversation Optimizer and configured in Watson Conversation:

Unconfident intents

The following is an example of Unconfident Intents:

*Watson:* Hi. It looks like a nice drive today. What would you like me to do?
*User:* make it warmer
*Watson:* I understand you want me to turn up something. You can say increase the volume or turn up the AC.

This filter shows all conversations with intents that have less than 50% confidence.

Check out the screenshots to view the Conversation Optimizer display in this case:

Dialog errors

This filter shows conversations with log messages which indicate errors in the dialog definition.

Check out the screenshots to view the Conversation Optimizer display in this case:

Summary

With the community tool, Conversation Optimizer for IBM Watson, you can optimize the user experiences of your bots. The tool provides different views to find conversation flows that have potential issues and might require improvements. The setup of the tool is a easy as pressing a ‘Deploy’ button and binding a Watson Conversation service to the deployed application.