As the cognitive computing era evolves and progresses, several cognitive solutions are being developed to tackle a variety of business challenges across multiple industries (Finance, Healthcare, Retail, etc.) and regions (North America, Europe, Asia Pacific, etc.) Discussing these cognitive solutions with partners and clients, it was clear that there is a sense of excitement and confusion in the market; excitement about the potential of cognitive technology and confusion about how to get started.
Some common questions we heard back from clients and partners:
- “What is cognitive computing?”
- “How is cognitive computing different from analytics?”
- “How difficult is it to build a cognitive solution?”
- “What kind of data do I need for cognitive solutions?”
- “What skills should my team have for building a cognitive solution?”
- “How can I apply cognitive computing to my business problems?”
- “How can I get started with cognitive computing?”
Start with a clear goal
To address these common questions, help reduce the confusion in the market and stimulate excitement, we set out to build the IBM Watson Business Coach, a tool designed to help introduce cognitive computing to users by understanding their needs and accordingly returning most relevant client references with enough details to resonate with the users. It is important to realize that business and technical leaders are always focused on solving their business challenges. While a new technology is always intriguing; business and technical leaders are mostly interested in understanding how this technology can help them achieve the results they desire. Given that the tool was to be launched at World of Watson, we had 10 weeks to build a highly interactive experience, powered by Watson.
With that in mind, we designed the IBM Watson Business Coach to be an interactive tool that interacts naturally with users to understand their needs and priorities. We leveraged the Watson Developer Cloud services, namely Watson Conversation, Natural Language Classifier (NLC), Speech to Text (STT) and Text to Speech (TTS), which are designed to enable natural interaction with users and understand their needs. In essence, Watson Business Coach is not only a tool to educate users about cognitive computing by serving relevant client references, but it also shows by example, as it was designed by following recommended methodology for designing cognitive powered-by-Watson solutions.
Find and prepare the right data
Once the objective is defined, the starting point for any cognitive solution is “data”. Since data is the lifeblood of cognitive solutions, it is critical that the data used to power the solution is accurate, relevant, and well curated. The team focused on collecting a large set of recent cognitive client references and curated that data by reviewing/editing/updating it to make sure it is complete and accurate. By cognitive client references, we mean actual powered-by-Watson solutions that were already successfully built by clients and partners to tackle various business problems across multiple industries and regions. For anyone interested in building a cognitive solution, it is important to realize that (and plan for) data collection and curation is a critical task which, often times, needs to be executed iteratively throughout the development of the cognitive solution and continues even beyond release of the solution.
Reviewing the client reference data, we extracted a number of parameters that can be used to map users’ needs to the relevant references; namely, business need, industry, team skills, data types, region, role, and implementation challenges. Effectively, by understanding the user’s perspective on these parameters, we can identify and deliver the most relevant references which would resonate with the user.
Given these parameters, we defined our conversation flow in the IBM Watson Business Coach to capture as many of these parameters as possible from the interaction with the user. This is a common pattern in building bots, virtual agents, or conversation-based solutions for a variety of problems where the solution consists of two main objectives:
- Understanding user’s needs.
- Delivering good data or executing some action.
To understand the user’s needs, we leveraged the Watson Conversation service and defined a number of intents and entities. The purpose of intents is to map “what the user says” to “what the user means”. Since there is a wide variety of ways a user can express their business needs and implementation challenges, we defined multiple intents to capture common business needs (10 intents for business need) and implementation challenges (10 intents for implementation challenges). Some examples of business need intents include improve_customer_service, reduce_costs, improve_productivity, accelerate_innovation, and expand_into_new_markets. For implementation challenges, we defined intents such as lack_of_stakeholder_support, immature_technology, budget_ constraints and several others.
Entities, on the other hand, are designed to provide specific information to help the bot understand a focus area of interest to the user. We defined entities to capture industry (Finance, Healthcare, Education, …), role (CEO, CTO, CIO, CMO, …), region (North America, Europe, Asia Pacific, …) , team skills (developers, data scientists, computer scientists, subject matter experts, …) , and data types (internal, open source, social media, CRM, …). For each of these entities, we provided a rich set of synonyms to understand users’ input and map it to corresponding entity type.
The second objective, delivering good data, involves providing relevant and good quality data. For Watson Business Coach, we delivered good quality data by manually curating the original set of client references. To deliver relevant data, we designed the dialog in Watson Conversation service as an orchestrator of user’s intents and entities integrated with app logic to identify relevant client references and serve those back to the user.
Train the system
A critical task in building a cognitive solution is to train the system; effectively, teach the system to recognize the nuance of natural language within the context of the application. The Watson services rely on a variety of machine learning algorithms, most of which fall in the supervised machine learning category, which learn the specifics of the problem from example data and accordingly, help make predictions on unseen data. Training a supervised machine learning system involves providing it with representative inputs and desired outputs and the system will learn by example. These pairs of representative inputs/outputs constitute the “ground truth” from which the system learns.
The Watson Conversation service employs “deep learning” technologies to extract intent from a natural language utterance. Training Conversation would require providing a ground truth which includes representative utterances (input) and the corresponding intents (output). Conversation would then learn which utterances map to which intents. Note that it not only will be able to extract intent from utterances it has seen but it can also extract intent from any utterance based on similarity of such an utterance to what is available in the training data.
For initial training of the Watson Business Coach application, we populated synonyms for entities and collected training data for intents by reaching out to end users who provided common entity synonyms and sample utterances for the various intents. Note that training a cognitive solution is an on-going process. Beyond the initial training, we continually collect user feedback and re-train the system to improve its performance.
In conclusion, we hope that we’ve illustrated how we went about designing and implementing the Watson Business Coach application leveraging Watson Developer Cloud services (Conversation, NLC, STT, TTS) and following the recommended approach for building cognitive solutions. Namely, for solutions powered by Watson Conversation, the approach consists of the following steps:
- Define the business problem to solve.
- Once the objective is defined, start with the data.
- Make sure data is accurate and relevant.
- Plan for data curation and ingestion.
- Define intents and entities specific to your application. Train the system
- Capture real end user input for representative training data
- Plan for continuous training and data curation of your cognitive solution beyond application release.