Call centers are a general term for help desks, information lines, and customer service centers that provide customer care and support services. Call center managers are routinely expected to identify opportunities and trends from customer engagement, improve customer experience, drive customer satisfaction, and manage agent performance. Because a call center can receive thousands of calls every day, it can be challenging for call center managers to accomplish their goals effectively and efficiently. How to stay informed of the products and ensure that customers’ questions and problems are resolved properly and quickly in any of them? What are the top concerns that customers have with the products? How to monitor and evaluate agent performance and coach staff members to improve performance?

To answer these questions, call center managers typically rely on quality monitoring questionnaires. However, this approach is not scalable to handle a large number of calls. Call center managers need new analytic tools to prepare summary reports for customer engagement and oversee agent performance while ensuring that service level and quality objectives are achieved.

This tutorial explains a new application that can help with these objectives. The Customer Engagement Analytics app uses Watson Tone Analyzer and Watson Discovery to provide visual insights on chat quality monitoring, customer satisfaction measurement, agent performance evaluation, and customer engagement tracking. The app uses tones provided by Watson Tone Analyzer as metrics to describe the emotional aspects in customer engagement, which can shape customers’ attitudes and drive their decisions. These tones (frustrated, sad, satisfied, excited, polite, impolite and sympathetic) are particularly suitable for a customer engagement scenario. The tones are more fine-grained than sentiments in capturing the precise emotional states of customers and agents. By integrating domain-specific tone analysis with Watson Discovery, which automatically identifies product entities frequently mentioned in these conversations, you can get an effective summary of and satisfaction across various products.

The application can help call center managers address the following challenges in their daily practice:

  1. How satisfied are customers with our service, by product?
  2. What are the top concerns our customers discuss?
  3. How are our agents performing based on customer tones?
  4. What are our customers’ tones in their first interaction with us?

Tone Analyzer plus Discovery

 The input to the system can be any form of conversation texts, ranging from calls, emails, and forums to online chats. The system performs conversation analysis and creates a visual interface for call center managers to explore the conversations.

Within the app, the tone analysis is intentionally performed at the utterance (or statement) level rather than for a whole conversation. This is because a customer’s tones can change dynamically as they talk to a customer service agent in a conversation. The app uses Watson Tone Analyzer for Customer Engagement Endpoint to analyze the tones in each utterance in a conversation corpus. For each utterance, Tone Analyzer produces a confidence score for each of the predicted tones taken from the following set of seven tones: frustration, satisfaction, excitement, politeness, impoliteness, sadness and sympathy. We note that sympathy is only considered for agents’ utterances and satisfaction only for customers’ utterances, as call center managers can be particularly concerned with whether agents are sympathetic and whether customers are satisfied. The tone that receives the highest confidence score is considered to be the dominant tone if multiple tones are detected for an utterance.

The app then uses Watson Discovery to enrich the conversations with semantic information through Entity Extraction, which identifies items such as persons, places, and organizations that are present in the conversation text. It adds semantic knowledge to content to help understand the subject and context of the conversation that is being analyzed. In customer care conversations, the top entities are mostly the products of a given brand as they are frequently mentioned during customer care conversations. The following figure illustrates the system pipeline.

Figure 1. System Overview
Figure 1. System Overview

 

The input conversations are first analyzed using Tone Analyzer. The detected tones, together with the conversation texts, are loaded into Discovery to extract the entities, which are associated with tones. As a result, the conversations loaded in the Discovery cloud contain entity and tone information, which can be retrieved through a user interface for further visual analysis of conversations.

Visual analytics

To provide an effective summary of customer satisfaction, customers’ concerns and agent performance, the app has a visual interface that highlights customers’ tones across different products or agents at different stages of conversations. As shown in the following figure, the interface consists of four parts: (a) Main View, (b) Agent/Customer View, (c) Conversation View, and (d) Search Panel.

Figure 2. Visual Interface: (a) Main View, (b) Agent/Customer View, (c) Conversation View, (d) Search Panel.
Figure 2. Visual Interface: (a) Main View, (b) Agent/Customer View, (c) Conversation View, (d) Search Panel

 To address the previously mentioned challenges, the app employs a multi-tab design (Figure 3), which includes four views of (1) Customer Satisfaction, (2) Customer Concerns, (3) Agent Performance, and (4) Initial Customer Tones. In each tab, a stacked bar chart highlights customers’ dominant tones across different products and agents. Tones are colored in a diverging color scheme to display both negative tones (frustrated, sad, impolite) and positive tones (polite, excited, satisfied).

Figure 3. Visualizing customers’ tones across products
Figure 3. Visualizing customers’ tones across products

Because tone analysis is performed at the utterance level, the app considers customers’ utterances at different stages of conversations when creating the visualizations. For example, in the Customer Satisfaction view and the Agent Performance view, the app uses tones detected from the customer’s last utterance to describe whether the customer is satisfied or still frustrated toward the end of a conversation, which also indicates how agents performed based on their customers’ satisfaction rates. In the Customer Concerns view, we use all of a customer’s utterances because the customer might express their concerns in any stage of a conversation. The app also provides a summary of customers’ tones in their first interaction with agents in the Initial Customer Tones view so that call center managers can compare how conversations start verses how they finish.

In addition to the bar chart visualization, multiple views are included to enable further analysis of conversations. In particular, the Agent/Customer view displays the top agents that provide customer service and the customers with the most number of requests. The Conversation view displays the full conversation texts in a structured manner to help call center managers understand how a customer’s tones evolve during a conversation.

A rich set of interactions are supported for visual exploration of customer care conversations. For instance, users can hover over a bar segment to view the percentage of a particular tone, or select a bar segment to drill down into a subset of conversations where the selected tone is detected at a certain stage (Figure 4). Users can also select a customer or an agent in the Customer/Agent view to refine their selection of conversations, or set a threshold to further filter the conversations by the number of utterances.

Figure 4. Displaying filtered conversations with top agents and customers
Figure 4. Displaying filtered conversations with top agents and customers

Furthermore, users can search any keyword of interest (for example, a product, a customer’s name, or an agent’s name) in the search panel (Figure 5). The system extracts all relevant conversations regarding the search keyword using Discovery in real time, and updates the bar chart visualizations with only the relevant conversations. This provides users with an alternative way to drill down to conversations of interest, after they have an initial understanding of the conversations by visual exploration.

Figure 5. Using the search panel for further exploration
Figure 5. Using the search panel for further exploration

Usage Scenario

To illustrate the capability of Customer Engagement Analytics, I will use a scenario of investigating customer engagement using the application deployed here. Suppose Brian is a call center manager whose job is to monitor chat quality, assess customer satisfaction, and evaluate agent performance. He used Customer Engagement Analytics for this purpose. His data has been processed and analyzed by the Customer Engagement system for a collection of customer care conversations in Twitter.

To start with, Brian viewed the distribution of tones across top products in the Customer Satisfaction view (Figure 3). He observed that the product tabletop received the most number of customer requests, followed by ovx and mtx, which has almost half the number of requests compared to tabletop. He also found that customers’ tones regarding tabletop are somehow polarized: a majority of customers were either frustrated or satisfied toward the end of conversation. To investigate further, Brian clicked the satisfied segment in the visualization and filtered the conversations with ‘tabletop’ and ‘satisfied,’ as shown in Figure 4. He was able to quickly examine the corresponding conversations with this combination of filters, as 10 out of 120 conversations were extracted and displayed.

To dig deeper into the conversation data, Brian searched the product tabletop in the search panel (Figure 5), and obtained a new visualization generated from the conversations with respect to tabletop (Figure 6). As a result, more entities that are related to ‘tabletop’ were identified and the corresponding customers are mostly positive about their service calls.

Figure 6. Drilling down into conversations related the product ‘tabletop’, after performing keyword search
Figure 6. Drilling down into conversations related the product ‘tabletop’, after performing keyword search

Brian would also like to see how his agents handled customers’ questions and problems. His went to the Agent Performance view (Figure 7). He observed that the top three agents (ranked by the number of requests handled), Tamisha Glasgow, Richelle Smyre and Janean Chiang, were more satisfactory when customers ended their calls. Nicolette Avalos did not perform as well as her colleagues because many of her customers were still frustrated at the end of a call. Furthermore, Brian found that while Tamisha Glasgow was the most hard-working agent who dealt with the most number of calls, Janean Chiang actually had a higher satisfaction rate (35%) than Tamisha Glasgow (23%). This new insight encourages Brian to consider assigning more customer calls to Janean Chiang in future.

Figure 7. Viewing Performance of Top Agents
Figure 7. Viewing Performance of Top Agents

Summary

I hope that this tutorial has given you a good demonstration of the capability of bringing Tone Analyzer and Discovery together for customer engagement analytics. With the Customer Engagement Analytics application, call center managers can scale their efforts in preparing summary reports for customer experience and satisfaction, overseeing agent performance, and tracking customers’ emotional engagement. If you want to apply your customer care conversation data to this application, you can follow these instructions.

Note: This article is a collaborative work by the authors Xiaotong Liu and Anbang Xu, Vibha Sinha, Rama Akkiraju.

Learn more about Watson and Cognitive Computing at IBM

8 comments on"Get customer engagement analytics with Watson Tone Analyzer and Discovery"

  1. The “instructions” link redirects to a private repo which is not publically accessible, can you please share it on a public repo or share that application on any other medium.

  2. Isabella Magacho September 18, 2017

    IBM employees can not use for free?

  3. Hi Haider,

    Thanks for your comment. For public repo please refer to https://github.com/fallingstar621/CustomerEngagementAnalytics

    Xiaotong

Join The Discussion

Your email address will not be published. Required fields are marked *