Putting AI to work with Watson

Navigating IBM data and AI products

Many enterprises are now looking at artificial intelligence (AI) and how to use their own data. Enterprise needs vary. Depending on what data is being stored and sent, the data might be required to be kept on-premises. IBM is a great fit for enterprises with strict business needs (like HIPPA or GDPR) because IBM keeps these requirements in mind as the software is written. IBM also provides on-ramps with free-to-use offerings, learnings paths, tutorials, and videos.

In this blog, I’ll look at these sets of data and AI products from a developer’s point of view:

Product families

Before looking at each set of products, let’s take a quick detour and see how they can be used.

Deployment options

Most of the IBM data and AI products can be accessed through:

  1. IBM Cloud: The IBM public cloud offering that offers lite and pay-as-you-go plans
  2. IBM Cloud Pak for Data: A bundle of data and AI add-ons that can be installed on Red Hat OpenShift

The following image shows a representation of each stack for consuming the data and AI offerings on both IBM Cloud Pak for Data and IBM Cloud.

IBM Cloud IBM Cloud Pak for Data
alt alt

NOTE: The Watson Machine Learning Accelerator product suite requires Power hardware.

Watson APIs

These services are the most easily understood, and knowing AI concepts is not required to use them. They each have a REST API that can be called, and are meant to be called at the application level by SDKs. These services use pre-built models and provide a user-friendly way to create user custom models.

alt

Watson APIs:

  • Include services such as Assistant, Discovery, Tone Analyzer, and Visual Recognition
  • Have well-maintained APIs
  • Have SDKs for popular languages (such as the Java language, Node, Swift, and Go)
  • Have generous free tier plans that let you get your hands dirty without excessive costs
  • Are available on the IBM Cloud and as add-ons to IBM Cloud Pak for Data
  • Are truly a great place for developers to start their AI journey

Watson Assistant

The main “chatbot” offering and so much more. Watson Assistant comes pre-trained with industry-relevant content. It has an intuitive editor; provides middleware for integrating with Slack, Facebook, and many other platforms; and can even be integrated with Watson Voice Gateway so that you can talk to your assistant over a phone.

Watson Discovery

The “AI search” offering. Simply put, use the Discovery tool or API to load a bunch of documents (such as .pdf or .doc files) and have Discovery build a model for you. You can query this model with natural language, and Discovery pulls out relevant parts of the documents. You can make Discovery smarter by using “Smart Document Understanding” to ensure documents are read properly or by using “Watson Knowledge Studio” to build a domain or industry-specific model.

Watson Visual Recognition

Explaining this service is pretty simple. Step one is to upload an image and step two is to read the output from the service after it tried to classify the image. That’s it! By default, Watson Visual Recognition comes with two pre-built classifiers (general and food). But where it excels is letting you create your own classifiers. You can use the Watson Visual Recognition tools to upload your own images to serve as “training data,” and Watson Visual Recognition creates the model for you. This model is callable through a REST API or can be exported as a Core ML model for iOS devices.

And more

  • Watson Text to Speech: Text goes in, audio comes out
  • Watson Speech to Text: User’s voice goes in, text comes out
  • Watson Translator: Text goes in, select your source and destination languages, more text comes out
  • Watson Natural Language Understanding: A short string of text goes in, and concepts, entities, keywords, emotions, and sentiment comes out
  • Watson Natural Language Classifier: Build text classifiers based on training data, and then classify text against the models that are built

Tabular representation

Offering On Cloud On Prem Free tier (on Cloud) SDK support
Assistant
Discovery
Visual Recognition
Text to Speech
Speech to Text
Translator
Natural Language Understanding
Natural Language Classifier

Now onto the hard stuff, no more pre-made models. So, let’s move on to Watson Studio.

Watson Studio

Watson Studio gives you an environment and tools to collaboratively work with data. Data can be imported (through connectors), viewed, refined, and analyzed. Models can then be created with the Watson Studio Deep Learning and Machine Learning tools.

Watson Studio image

Watson Studio:

  • Is based on various open source technology such as Jupyter Notebooks and R Studio
  • Provides tools based on popular open source frameworks such as Tensorflow, Keras, and Pytorch
  • Includes 2 GB of free storage on IBM Cloud Object Storage
  • Is available on a public cloud and as add-ons to IBM Cloud Pak for Data
  • Is an excellent platform to begin your data science journey

Jupyter Notebooks

Watson Studio provides support for running Jupyter Notebooks. There are several environments to choose from (Python 3.x, R3.4, and Scala 2.11), and each has the option to add a Spark kernel as well. You can share your Notebook collaboratively, add data from your Object Store as a data frame, publish your Notebook, and use many other of the features you’ve come to expect.

Watson Machine Learning

The Watson Machine Learning service is a major part of Watson Studio. It provides a set of APIs that can be called to interact with a machine learning model. The model can be created using a Jupyter Notebook, SPSS Modeler, or AutoAI, and then deployed to an instance of the Watson Machine Learning service. After it’s deployed, you can score data by using a REST call against an API that the service provides.

OpenScale

Watson OpenScale allows monitoring and management of machine learning models that are built on various platforms: Watson Studio, Amazon Sagemaker, Azure Machine Learning, and other popular open source frameworks.

And more

  • AutoAI: A graphical tool in Watson Studio that automatically analyzes your data and generates candidate model pipelines that are customized for your predictive modeling problem.
  • SPSS Modeler: Watson Studio offers a variety of modeling methods taken from machine learning, artificial intelligence, and statistics.
  • Data visualization: Watson Studio provides the ability to visualize data with Data Refinery.
  • Cognos Dashboards: A dashboard for viewing and analyzing data, instead of using pandas or pixiedust.
  • Connections: Watson Studio provides connections to import data from IBM Services (Db2, Cognos, Cloudant, Cloud Object Storage, and many more) and for third-party services (Amazon S3, Hortonworks HDFS, Microsoft SQL Server, Salesforce, MySQL, Oracle, Google BigQuery, and many more).

Watson Machine Learning Accelerator

IBM Watson Machine Learning Accelerator is geared toward enterprises. It is a software bundle that is optimized to run on Power hardware. It bundles IBM PowerAI, IBM Spectrum Conductor, IBM Spectrum Conductor Deep Learning Impact, and support from IBM for the whole stack including the open source deep learning frameworks. It provides an end-to-end, deep learning platform for data scientists.

Watson Machine Learning Accelerator

PowerAI Vision

IBM PowerAI Vision provides a video and image analysis platform that offers built-in deep learning models that learn to analyze images and video streams for classification and object detection. PowerAI Vision is built on open source frameworks and provides sophisticated methods to create models with an easy to understand user interface.

And more

  • Spectrum Conductor: Deploys modern computing frameworks and services for an enterprise environment, both on-premises and in the cloud.
  • Snap ML: A library developed by IBM Research for training generalized linear models with the intent on removing training time as a bottleneck. Snap ML scales gracefully to data sets with billions, and offers distributed training and GPU acceleration.

Call for Code 2019 Finalist: Healios provides easily accessible and high-quality mental health care

Developer Kevin Kim lived through Hurricane Sandy in 2012 and saw the deep impact the storm had on close friends — not only on physical belongings and health, but on their mental well-being. One of Kim’s friends, for example, had a tree crash through his roof, and though no one was physically hurt, dealing with the insurance and finances after the storm took a heavy toll on his friend.

The experience spurred Kim and his teammates — Christopher McKinney, Sunjae Shim, Tony Park, and Xuelong Mu — to join forces and create Healios, an online platform that uses a conversational interface and AI to help connect those who need mental healthcare with the right case worker. The team’s solution has been named a top-five finalist in the Call for Code 2019 Global Challenge.

Team Healios Pictured: (Bottom row from left) Kevin Kim, Tony Park, Sunjae Shim, (top row from left) Xuelong Mu and Christopher McKinney

“There are a lot of stressors that are being constantly inundated with needs that pop out of nowhere, that they never had to deal with,” Kim said. “With Healios, you can always rely on getting your thoughts through to someone, even if it is a chat bot, eventually a social worker will analyze what you’re saying, and help you get the resources you need. If you need to see a therapist, they’ll hook you up with a therapist nearby.”

Healios aims to bring therapy and guidance in an easy, accessible, and affordable way to anyone who experiences a traumatic event, but doesn’t have the current means or ways to get the help they most certainly need. In her research as a psychology student, Sunjae Shim said she found that almost 30-40 percent of the people who experienced Hurricane Harvey still have re-occurring memories of the traumatic incident, even after time had passed.

The team wanted to bring mental health resources quickly and easily to victims while also using the tech to help de-stigmatize mental health issues and normalize the act of seeking mental health care.

How it works

Healios solution diagram

Healios works through two avenues: a mobile view for the patient and a web view for social workers and medical personnel. The victims communicate via their mobile device to speak with medical professionals with the help of the team’s IBM Watson application. The web view for medical professionals is a dashboard for them to be able to handle all of the cases, and to get an overall birds eye view of a patient’s state and overall mental well-being.

The IBM technology starts working as soon as the user logs in. The initial onboarding session is done with a Watson Assistant chat bot that asks users questions, such as how they’re feeling, what their stress levels are, and what they feel anxious about. The data is processed by natural language understanding and personality insights, which is then parsed to figure out if what a person is experiencing.

After the patient submits responses where the data is parsed, Watson Natural Language Understanding helps sort the emotions the user is experiencing and highlights them for any counselor who’s reviewing the data. From the data, counselors and medical professionals can make a more accurate and more efficient way of counseling and diagnosing and interface back with the user who submitted the questionnaire.

What’s next

By making high-quality mental health care accessible to all, Healios has the potential to profoundly change the way individuals cope with the long-lasting mental health issues following traumatic events — not just natural disasters.

“At the ground level, we really want Healios to be about changing the way people first and foremost perceive mental health care and making it more accessible, making it a lot more easy to participate in,” team member Christopher McKinney says. “But at the same time, expanding it beyond the scope of just natural disaster victims.”

Another goal for the future of Healios is more research on the psychological consequences and outcomes that disaster victims experience after a natural disaster, to gain a better understanding through more data points. Because of the lack of information around mental health and natural disasters, the team wants to ensure they approach it from the right way. They hope to gain information from more studies as these natural disasters happen. For example, the team will look at the research and findings from the recent hurricane in The Bahamas to potentially integrate into their Healios platform.

“We want to create a world where going to take care of your mental health is as normal as going to the gym or going in for a regular checkup,” Xuelong Mu said. “We want to de-stigmatize the associations with finding a therapist or getting help. And eventually we’d like to make our platform accessible to anyone for a variety of conditions to try and get some help and improve their mental health.”

Use Watson APIs on OpenShift

Before we talk about how to use Watson APIs on OpenShift, let’s quickly define what they are.

  • Watson APIs: A set of artificial intelligence (AI) services that are available on IBM Cloud that have a REST API and SDKs for many popular languages. Watson Assistant and Watson Discovery are part of this set to name a few.

  • OpenShift: Red Hat OpenShift is a hybrid-cloud, enterprise Kubernetes application platform. IBM Cloud now offers it as a hosted solution or an on-premises platform as a service (PaaS): Red Hat OpenShift on IBM Cloud. It is built around containers, orchestrated and managed by Kubernetes, on a foundation of Red Hat Enterprise Linux. You can read more about the History of Kubernetes, OpenShift, and IBM in a blog post by Anton McConville and Olaph Wagoner.

Now, let’s talk about how to combine the two. In our opinion, there are really two ways to use Watson APIs in an OpenShift environment.

  1. Containerizing your application with Source-to-Image (S2I) and calling the Watson APIs directly at the application layer
  2. Using Cloud Pak for Data add-ons for specific APIs (more on this option later)

Let’s dig into the first option.

Source-to-Image

What is S2I?

Source-to-Image is a framework for building reproducible container images from source code. S2I produces ready-to-run images by injecting source code into a container image and letting the container prepare that source code for execution. S2I comes with OpenShift but it is also available as a stand-alone tool. Take a look at how simple it is to use S2I through an OpenShift console.

How do I use S2I for my Watson app?

Say you have a Node.js app, and you’d like to deploy it in a container running on OpenShift. Here’s what you do. (Our examples in this section use Red Hat OpenShift on IBM Cloud.)

  1. From the OpenShift catalog, select a runtime (for example, Node.js or Python) and point to a repository.

    add git repo

  2. Add configuration for the application, such as any Watson services API keys, as a Config Map.

    openshift config map

  3. Associate that Config Map with your app.

    openshift add config map to app

And you’re done! The containerized app will be deployed and now can use any existing Watson Service available through a REST API call.

What are the benefits?

  • Minimal refactoring of code
  • Source-to-Image’s ease of use
  • Fastest way to get started

References

We’ve already added OpenShift Source-to-Image instructions for some of our most popular Watson code patterns.

A quick example

We also created a quick video example that demonstrates how to use the approach mentioned above.


Cloud Pak for Data

What is Cloud Pak for Data?

Cloud Pak for Data can be deployed on OpenShift and includes a lot of AI and data products from IBM. These products include, but are not limited to, Watson Studio, Watson Machine Learning, Db2 Warehouse, and Watson Assistant.

How do I use Cloud Pak for Data for my Watson app?

Using our previous example, say that you have a Node.js app running on-premises and behind a firewall. In just a few minutes, you can update the application to call Watson APIs that are running on your Cloud Pak for Data.

  1. (Prerequisite) Install Cloud Pak for Data, on-premises, preferably on OpenShift.

  2. Install the Watson API kit add-on, the Watson Assistant add-on, and the Watson Discovery add-on. The Watson API kit includes Watson Knowledge Studio, Watson Natural Language Understanding, Watson Speech to Text, and Watson Text to Speech.

  3. Launch the Watson API service that you want to use and generate a new API Key.

  4. Update the application to use the new API key and REST endpoint.

What are the benefits?

  • If on-premises, REST calls never hit a public endpoint
  • Some refactoring, mostly at the configuration level

References

We’re still in the process of updating our content to work with Watson APIs on OpenShift, so here are a couple of references instead:

Thanks for reading our blog! Start the journey to containerizing your Watson applications by following our Sample using Watson Assistant or Sample using Watson Discovery. Or, if you’re interested in learning more about Cloud Pak for Data, check out this Overview of Cloud Pak for Data video.

Building a customer-facing chatbot on IBM.com

In this blog, I want to walk you through the process we went through to build chatbots on IBM.com that can answer commonly asked product questions. Each of our chatbots was trained on a specific product or technology. They were trained to answer questions about products or technologies like SPSS® Statistics, slockchain, Cognos® analytics, and Db2® Warehouse. Our scope was to build individual chatbots that could understand the most common customer questions and respond with either relevant information or links and also be able to transfer to a human agent well-versed on that topic. All of our chatbots are built using the Watson™ Assistant API.

Results of chatbot deployments

Higher engagement

We found that customers like talking to our chatbots, and engagement on our pages went up after chatbots were deployed. User testing found that customers were surprised how much our chatbots knew and found that customers are often more comfortable talking to a chatbot initially about their problem versus immediately being transferred to a human agent. Chatbots are also a great way to gain insight into your customers’ journeys (or frustrations) on your website.

Better routing

We have found that chatbots do a great job of understanding a user’s intent and getting them where they need to go. Most of our customers were not wanting to have a long conversation with our chatbot, but they did need help getting to the right resource. Common destinations for our chatbots include:

  • Technical documentation
  • Technical support
  • Sales agents
  • Information for students
  • Job opportunities

Building the bot

To build a chatbot using Watson Assistant, you need three main pieces of data:

  • Intent examples (the how)
  • Entity examples (the what)
  • Dialog tree (the flow)

Examining human chat transcripts for intent and entity examples

We wanted our chatbots to be able to answer the most common types of questions customers ask when they come to product pages. Fortunately, historic human-to-human chat data existed for most of the pages where we wanted to deploy chatbots, so the first thing we did was mine existing human-to-human transcript data for common intents. From those transcripts, we labeled customer questions with different identified customer intents, such as pricing or buy. This can be a cumbersome process, but is critical for a good chatbot. You can speed up this process by letting Watson pre-label your historic human-to-human chat logs (read more).

Here is some example data for our chat data:

Customer utterance Intent label
What is the price? Pricing
How can I buy? Buy
I’m looking for white papers on the product Documentation

From our transcript analysis, we initially identified about 50 common intents that we wanted our chatbot to be able to recognize. We tried to find 10-50 examples from our transcript data for each intent.

We repeated this exercise with entities. We wanted to identify the ways that customers referred to our products in historic human-to-human transcript data.

Customer phrase Entity
Standard version Version:Standard
Std version Version:Standard

By examining historic transcript data, we were able to identify other ways customer might refer to our products, such as saying “Std Version” instead of “Standard Version.”

Why did we do it this way?

We wanted to make sure our chatbots could understand and respond to the most common kinds of customer questions. Thus, we needed to examine historic human-to-human transcript data to make sure our chatbot could understand what types of questions our customers were asking.

Additionally, we discovered in early testing that the way that internal testers knowledgeable with our products phrased their test questions to the chatbot didn’t always match how our customers asked similar questions. Thus, instead of drawing on synthetic utterances to add to our training data, we tried to look for actual customer phrasing verbatim in order to capture the diverse ways our customers might be seeking the same information. This methodology helped ensure that our chatbots could understand our customers in their own words.

Matching entities to intents

After we completed the intent and entity labeled data, we began building a map for which entities were relevant to which intent. We looked at historic transcript data to see which entity types were mentioned in volume for which intents. For example, we might notice that customers asked about the “version:premium” or “version:standard” entities commonly with “Pricing” or “Buy” intents.

Building a dialog tree

After we completed the entity and intent map, we built a dialog tree. We took our prior entity-to-intent mappings and created dialog nodes. Here is an example response for a customer asking a pricing intent where the standard entity is also found:

Condition Example bot response
#Pricing && @version:standard “The price of the standard version is …”

Most of our customers did not want to have a long conversation with our chatbot. Typical conversation length is about six questions. We saw two scenarios in our chatbot data transcript data:

  1. The customer has five or six basic questions about the product.
  2. The customer is looking for a specific resource (for example, a free trial, troubleshooting, downloads, API documentation, requesting a sales agent or support specialist, etc.).

We wanted to make sure our chatbot could achieve both of the above two goals.

There are two main methods to consider when building a dialog tree:

  1. Reflective dialog flow — Get the customer to where they want to go. A reflective dialog flow isn’t looking to guide the user down a particular path. Instead, it provides a set of resources to find out more about a topic.
  2. Proactive dialog flow — This is useful for gathering specific information about a customer, such as contact information or clarifying what type of product he might be interested in. Use cases could include gathering information for a support ticket or proactively asking a number of questions to determine which product best suits the customer’s needs.

A good chatbot should be a combination of both. We wanted to make sure ours could answer basic product questions while at any time being able to guide the customer to the best resource, page, or human agent that would meet his particular needs.

Transferring relevant chats to agents

We needed our chatbot to be able to transfer to human agents mid-conversation. For example, we identified these common scenarios:

  • Transferring unhappy customers to agents
  • Transferring customers with sales intents to sales agents
  • Transferring customers with support intents to support agents

We set up our chatbot to be able to transfer customers with relevant intents to appropriate agents. We also set it up so that the human agents can read through the bot conversation that took place prior to the transfer. Passing the chatbot transcript to the human agent helps prevent the customer from being asked duplicative questions by the human agent after transfer.

Monitoring

Customers may ask your chatbot new or differently phrased questions that were not found in historic human-to-human chats. Therefore, it is important to monitor your chatbot and continue to regularly train it on newly identified customer intents and entities. On a weekly basis, we analyze transcript data to look for scenarios where either the chatbot misunderstood or didn’t have a high-confidence answer, and we regularly retrain our chatbot to ensure that it keeps learning and improving.

Intent data best practices from chatbots deployed on IBM.com

The team I lead has deployed over 20 customer-facing chatbots across IBM.com. We’ve found it much easier to maintain a single set of intent and entity data versus maintaining individual sets of intent and entity data per chatbot. Here are some tips and tricks for scaling and maintaining your intent, entity, and dialog data. These tips are useful whether you are maintaining a single chatbot or scaling your data across tens or hundreds of chatbots.

Intent conflict resolution

A large set of labelled intent data can become hard to maintain over time. Here are two approaches we used to maintain accuracy in our intent data:

  • Leverage the built-in intent conflict resolution built in to Watson™ Assistant. Watson Assistant has a great built-in intent conflict resolution. One flagged example that came up was the question, “What can the product do?” versus “What can you do?” In the former, the customer is asking what the product can do, but, in the latter, he is asking our chatbot what it is capable of doing. While the above examples are valid to have in two different classes, this same script also helped us find mislabelled utterances in our intent training data.

    Conflict resolution

    Read more about conflict resolution.

  • K-Fold Cross-Validation We leveraged K-Fold Cross-Validation to look for classes that had high levels of confusion among them. When these classes were identified, we took deeper dives into those classes to see if they should be combined, further split apart, or just cleaned up. One easy way to jump into K-Fold Cross Validation for your existing Watson Assistant workspace is to plug it into a Jupyter Notebook. Here is one pre-built Jupyter Notebook you can use for K-Fold Cross Validation.

Making intent data reusable across many chatbots

Because building intent data for a single bot requires a lot of labelled training data, we needed to be able to reuse this training data across many chatbots. Maintaining a single set of intent and entity data across many bots comes with the following advantages:

  • Only need to maintain one set of data
  • As one chatbot is improved, the others also improve

However, maintaining a single training data set did require that we normalize our data. For example, the training data utterance, “What is the price of SPSS Statistics?” contains the keyword statistics. That presence of that keyword is not actually relevant to whether the utterance is about pricing. We saw confusion, for example, that the chatbot might think the utterance was about analytics versus primarily pricing. So, to reuse this labelled utterance across all our chatbots, we normalized that sentence in our intent training data to “What is the price of @product?”

Intent entity window

Read more about directly referencing an entity name in an intent example.

Smoke test data

Separate test data should be maintained to test intents, entities, and dialog flows. As part of our continuous delivery pipeline, our chatbots are tested with our test data to ensure that the intent, entity, and dialog data has not regressed. There are many ways to test a dialog tree, but our basic data set looked like this:

Test ID, Seq ID, Utterance, Intent, Entity, Dialog Node 1, 1, What is the price of Watson Studio, Detailed Pricing, Detailed_Pricing-General

We include both a test ID and a sequence ID to test deeper parts of the dialog tree. For example, we might want to test follow-up questions and the different paths that these dialog flows take.

This smoke test data can be incorporated into your CI/CD pipeline to test your workspace every time the dialog, intent, or entity data changes.

Leverage disambiguation

No set of intent data is perfect, and there are many scenarios under which a chatbot can be confused. Disambiguation is a great way to give your customers options to disambiguate their questions. On the plus side, you are also leveraging your customer’s choices to make your chatbot smarter in the future.

Disambiguation is built in to Watson Assistant Plus out of the box.

Disambiguation

You can read more about disambiguation.

Use third-party crowdsourcing websites

To increase the size of our labelled intent data, we sent example utterances to third-party crowdsourcing APIs for similar ways to restate the example utterance.

Use actual customer phrasing verbatim

Using only synthetic entity or intent data limits the accuracy of your chatbot against actual verbatim customer phrasing. Ideally, you should mine customer verbatims to build your intent and entity data. You can also leverage Watson to pre-label your historic human-to-human chat logs to speed up this process.

Read more about getting intent recommendations.