by David Amid and David Boaz
David Amid is an IBM master inventor and the technical lead of the Watson Decision Analytics team based in the IBM Research and Development Lab in Israel.
David Boaz is a Decision Analytics researcher who works out of the IBM Research and Development Lab in Israel.
Watson Tradeoff Analytics has been out for over five months. Since then, we have been asked on numerous occasions: How does Watson Tradeoff Analytics actually help me to make better decisions??
In this blog post we will try to answer this question. To make it clear and enjoyable we decided to follow Explained Visually’s paradigm — making difficult ideas intuitive.
Watson Tradeoff Analytics has three pillars:
- Pillar 1: Finding the top options, out of many possible options
- Pillar 2: Visually exploring these options
- Pillar 3: Simulating human judgment to guide you through the decision-making process
Pillar 1: Finding the top options
In most scenarios we all want to find the top option, not the top options. But, life is not that easy. In every decision we make in life, we have a dilemma around which option to choose, because there is always a tradeoff between competing goals.
And it’s not always about money! For example, “what do I wear today” entails: how appropriate it is vs. how comfortable it is vs. did I wear it last week? “What car do I buy” involves: is it safe? How much does it cost? What is its fuel consumption? Serious decisions are the most difficult, like “what treatment best meets a patient’s specific needs?” Is it the treatment with a high success rate, or the one with low adverse affects, or simply the one they can afford?
If there is no obvious best option, how do we find the option that best meets our preferences and needs? The best approach is to first identify the top options, because they represent the best deal you can get out of the tradeoff.
Filtering versus Tradeoff Analytics
What do people do if they don’t have Watson Tradeoff Analytics? How do they identify the top options? The common approach is to use traditional filtering. This is available on every shopping site. However, Filtering has drawbacks.
In the example below, we have a set of mutual funds, and we want to find the top investment option. Relying solely on the traditional filtering approach, we've begun to narrow down the number of options by setting the Risk threshold to 0.8. Did you notice that when we started to filter by Risk, we missed good opportunities for high Mid Term Value investments? Those opportunities are now grayed out. Even more problematic, we've filtered out quite a lot, but we are still left with many options (blue circles) that aren't actually good options. There are options marked with an orange cross that are better than the blue ones in both Risk and Mid Term Value.
The options marked with an orange cross are Watson Tradeoff Analytics. Those are the top options. In economics, this is called the Pareto Frontier, or Efficient Frontier (watch the movie), in Mathematics it is called Pareto Optimality. For each of the options marked with an orange cross, there is no other option that is better on both Risk and Mid Term Value goals.
Try it out! You can change the goals and see how the graph changes, or you can play with the filters to understand how filtering is misleading.
Identifying top options
OK, so we understand that there is more to it than filtering. But how can one identify those top options? In our example, the set of top options is pretty easy to discern on this (2D) diagram, since they were the ones closest to the "you want to be here" point. However, in practice, this gets complicated, since it is rarely the case that we have only two goals or criteria.
For example, selecting a car requires us to balance between safety, price, environmental, comfort and many other variables. In the Mutual Funds example, an individual making this decision would actually be looking at many other goals besides our two selection criteria, such as maximizing our Short Term Value, Mid Term Value, and Year-to-Date return (YTD) while minimizing our risk.
So can we look at each two goals separately? Unfortunately, the answer is no. If we apply Pareto Optimality multiple times, each time for only two goals, the results between each set of two goals will differ.
On the visual below, hover your mouse on a specfic option. When you hover over an option, that option is highlighted on all of the other views. Notice that all options that are considered "top" (orange cross) in one view, are not usually "top" in at least one other view (If you find one that is "top" on all views, it means that it is best in every respect. If this data was real and up-to-date, we would have told you to drop everything and go buy it!).
Luckily, Pareto Optimality can take into account more than two objectives at a time. That is the not-so-secret sauce behind the first pillar of Watson Tradeoff Analytics. When Tradeoff Analytics gets its inputs (a table of options and a list of the objectives), it first executes a multi-objective Pareto filtering process, which picks for us the top options with respect to all of the objectives in that list. This allows users of Tradeoff Analytics to focus only on the top options, excluding the inferior ones, thus making the decision process much easier. With the top options list in hand, we are now ready to go to the next stage - visually exploring the trade-offs between those options.
Pillar 2: Visually exploring your top options
So now that we have found our top options (and got rid of the inferior ones) - how can we distinguish between them? How can we better explore and understand their trade-offs? Using conventional diagrams, we are limited to considering two objectives at a time. So how about using colors or size for additional attributes on the diagram?
On the visual below, we added to the bubble (or cross), size as an additional visual attribute. Well, it’s a start, but humans tend to associate different importance to different visual gestures such as size, position, and color since it is just easier to discern some visual cues. This may cause unintentional bias towards one goal. Finally, what will we do if we have six objectives? What additional visual attributes can we add?
To overcome this, the Operational Research community has traditionally used a visualization called Parallel Coordinates. Unfortunately, Parallel Coordinates only do a decent job when you have either
But in the Mutual Fund example, where there are 115 funds, and 16 are considered top options (this time, according to 5-objective Pareto optimality), you won’t be able to reveal the tradeoffs, since you have more than a few options, but not nearly enough to look for trends. It’s simply overwhelming (see below)
Once again Watson Tradeoff Analytics can help you. Watson Tradeoff Analytics provides a Client Library with novel visualizations that foster exploration. Consider the map below. The closer an option (a colored bubble) is to one of the corners/vertices (where each represents a goal), the better its value on that specific goal and the greater is the intensity of the sector that correspondes to that goal.
In the Watson Tradeoff Analytics tool this is a interactive visualization where you can explore options and the tradeoffs between them.
Pillar 3: Simulating human judgment to guide you through the decision-making process
So, what do people do when they explore funds? They compare between them, and they employ their judgment in order to decide if the tradeoff between one goal to another is worth it. But we are humans, and in many cases, a computer can employ similar judgment and save us from making mistakes or missing out on more suitable options. That's the purpose of the third pillar of Tradeoff Analytics: simulating human judgment to guide you through the decision-making process.
This example scenario shows how Tradeoff Analytics can simulate human judgement to predict that you may mistakenly consider dropping a mutual fund before you can benefit from its other significant gains: I am a delayed gratification kind of a person, so I am interested in funds that will benefit me in the long run. So I set Long Term to be above 13%. Now, I can obviously select the mutual fund that is highest on Long Term revenue, but I am aware that this is highly risky and if I change my mind in a few years, I won’t have as good of Mid Term gains. So I select the one to its right, marked with a star.
However, Watson Tradeoff Analytics shows me that I can get more out of another option. If I am willing to endure a small, insignificant change in Mid Term Value. I can have large gains on YTD, Short Term Value, Risk, and even some Long Term. What I eventually select is completely up to me, but Tradeoff Analytics helps me to see a more complete picture.
Watson Tradeoff Analytics provides you with a unique way of making informed decisions. Personally, I make all decisions this way (just don’t tell my wife… Oops too late?)..
Watson Tradeoff Analytics has graduated from Beta and is now Generally Availlable.