Pele or Messi? Ali or Sugar Ray Robinson? Froome or Quintana?
Could we combine statistics, cognitive understanding and fan’s opinions to find out what makes great?
Our starting point was to assemble a team of statisticians, tennis analysts, coaches, journalists, and developers. The team discussed the different attributes that might make a great Wimbledon champion (passion, stamina, serve, return of serve, adapt or impose and performing under pressure) and a long list of potential greats, giving us a base for analysis. The team would assess each of these topics using a different approach, whether that be statistical, cognitive or just years of tennis experience, before coming back together to discuss what they’d found.
In this post, we’ll focus on one of those approaches, how we used IBM’s Watson cognitive services to analyse 25 years of unstructured data. The services we used in this project were Watson Discovery Service and Personality Insights from the IBM Watson Developer Cloud.
Given the large set of documents from The Telegraph, wimbledon.com and Wimbledon’s own archives, our first task was to decide how to make best use of the available Watson services. We knew that Watson Discovery Service (WDS) would likely provide part of the answer. It allows you to search and run analytics against a document archive, making use of existing meta data and layering in its own enrichments (document sentiment, entities, concepts and emotions). Its aggregations would allow us to query, grouping by difference facets, such as by player or by year.
We were also interested in looking at how a player’s personality might affect elements of greatness. Some of the archive content was made up of player interviews, giving us scope for analysing what players had said via Personality Insights (PI).
Importing the Archives
WDS was going to be the central hub in the project, so getting data in was the first task. We worked with Wimbledon and The Telegraph to make use of their tournament and newspaper archives. To get the most from WDS it is important to organise and structure the data and then configure a WDS collection to use that structure.
We setup WDS to add in its own enrichments. This is as simple as selecting which enrichments to use (entities, concepts, keywords, sentiment and emotion) and telling WDS which part of the document structure it should analyse to infer these from. Enrichments can be used to build more sophisticated queries, find all the documents talking about the person Andy Murray (entity) and the topic serve (keyword) that are positive (sentiment), for example.
Once WDS was configured we could work on importing the documents. Going back to the 1990s, most of the content was in PDF format, but the text was machine readable. Apache Tika was used to parse the PDF files and extract the raw text.
With WDS you need to decide how to split the documents up. This drives the results you’ll get from queries. We knew that the type of information we’d want to get from the system were counts and short paragraphs of text, we weren’t looking to return long documents so we made the decision to split the content in to lots of short documents, of a couple of paragraphs each. It was important to capture as much meta data about the archives as possible. The directory structure of the documents was important, providing, for example, the publication date.
There were 1000s of these documents to work though, so we created a simple NodeJS project to trawl through the directory structure, parse the documents, capture any meta data and split them up into smaller fragments for upload to WDS using its add document API . A similar process was used for Wimbledon’s own data and the archived versions of wimbledon.com sites. WDS does have its own crawler for simpler scenarios.
Getting the Answers
With a populated collection in WDS the focus turned to querying against each of the topics. This is where the human contribution is really important. A WDS collection can be queried in lots of different ways, but asking the right question is how you get to useful insight. A large part of the project was spent developing these queries.
Many of the queries involve combinations of keywords, entities and sentiment. For most of the topics we were interested in an entity (e.g. a specific player) a keyword (e.g. stamina) and a sentiment (e.g. positive). At a simple level, we could count how many hits WDS gave us for each player and a topic. Or, it could be that we look at the descriptive words used around a keyword to identify how one player’s serve is described differently to another’s. Looking at documents in this way individually might not give you anything significant, but when you look at hundreds or thousands of documents over several decades you can build up a reliable picture.
For other questions, we wanted to look at how a player’s personality would affect their greatness. WDS was invaluable here as with the right query we could distinguish between things said about a player and those said by the player themselves. The Personality Insights service works by analysing content someone has said themselves. Once we had player quotes from WDS we could pass them into Personality Insights for analysis.
We needed to run these queries multiple times to get answers for each player and then organise the results. We wanted to be able to quickly change some of the properties of the queries, to change the keywords or sentiment, for example. The quicker we could rerun the analysis the more questions we’d be able to explore. This meant packaging the queries up in a script that would run the queries, and organise the results into files that could be accessed via Excel, providing an easy way to share and discuss the data with the team, usually opening up a whole new round of questions. Automating this task was an important factor in us having the capacity to explore new ideas and questions.
Example code of how to run Watson Discovery Service and Personality Insight calls in a script can be found here. https://github.com/ibmets/discovery-service-with-personality-insights
Round the Loops
Using WDS is very much an iterative, exploratory process. Coming to each topic we didn’t have an exact plan for how to analyse it. We had some starting thoughts and ideas, ran with those, looked at the results and went around the loop again. The results often prompted us to look for something different, or refine the queries we’d been using. There’s no right answer with doing this so it was difficult to know when to stop, usually carrying on until we came to the editorial meeting.
The editorial meeting was a weekly session with the wider team: statisticians, sports journalists and tennis coaches. As a group, we discussed what each of us had come up with. Sometimes the stats and cognitive results supported each other, but it was more exciting when they did not. That made us ask questions, had we missed something in the data? Was there a difference in how a player was perceived compared to the reality? Or had we been asking the wrong question? It led to more work, but the process was at its best when it made us challenge our preconceptions.
The best example was when Marion Bartoli featured prominently in the WDS results for a positive perception of her serve. The tennis coaches and statistician questioned this, Bartoli is not known for having a good serve, her average stats are weak for one of the top women. That made us look at what WDS was telling us at a deeper level. The positive descriptions were related to people noticing she was working hard on improving her serve and Bartoli herself talking about working to make it better. With this information the statisticians produced a new report, looking at her serve statistics over time. There was a clear upward trend, quite different from the other great players who were more consistent over their careers. The upward trend peaked in 2013, when she won Wimbledon. Even then, her serve wasn’t a strength, but she’d raised it to the point where it was no longer a weakness. It’s not that we couldn’t have worked this out before, it’s that without the cognitive analysis we wouldn’t have known to look.
Open for Debate
We used this process to look at six different attributes that we think make up a great Wimbledon player, but that’s not enough. We still need tennis fans perspectives.
We are going to open this up for discussion, using The Telegraph, Twitter and Facebook to collect fan opinion. We’ll use WDS to analyse the comments and help us pick out some key findings for what makes great in tennis fan’s view. In the same way we’ve used WDS to analyse historical media content about Wimbledon we plan to use it to help us understand what fans think.