AI-generated content at the US Open

This year’s action at the US Open tennis tournament will leverage content and insights that are generated by IBM Watson. Our AI algorithms read, interpret, forecast, and correlate data across 17 courts to produce insightful context for interesting impact players. For this year’s US Open, IBM is introducing two new innovative technology solutions: IBM Power Rankings with Watson and Match Insights with Watson.

  • IBM Power Rankings are AI-powered daily rankings of player performance and punditry insights. The Tennis Tour ranking systems use 52 weeks of historical data to quantify player performance. To complement these, Power Rankings focus on a player’s most recent history, combining advanced statistical analysis, the natural language processing of IBM Watson, and the power of IBM Cloud to analyze the most recent match data and media commentary to direct the attention of fans toward the most compelling matchups.

  • Match Insights with Watson are AI-generated fact sheets that help fans quickly get up to speed ahead of every singles match at the US Open. They use natural language processing, machine learning, and IBM Cloud to mine the most recent player statistics and media commentary for insight, including the latest IBM Power Rankings, relevant quotes from various media sources, and a natural language summary of key performance metrics.

Experience the IBM Power Rankings

The IBM Power Rankings (IPR) is the measure of a player’s strength going into and throughout a tournament. The factors that contribute to IPR provide strong indicators as to who will win a head-to-head match. The IPR is complementary to the traditional tour rank. Over a tennis season, a player’s ATP or WTA ranking is based on the number of points that are accrued over 19 different tournaments within a 52-week rolling window. As its foundation, IPR uses relevant industry punditry observed through hundreds of thousands of news sources combined with player performance to create an index of a player’s game capability. During a grand slam, the most current game play is more important than it is during the lead up to a grand slam. Further, throughout a tournament the crowd becomes more focused on a player with precise language. The shifting of IPR over time creates a dynamic and flexible player ranking to focus on different predictors based on gaming context. This helps both rankings to move toward independence, as shown in Figure 1.

Relationship between IBM Power Ranking and Tour Rank
Figure 1. The relationship between IBM Power Ranking and tour rank

Each day, the IBM Power Rankings are updated and available on a leader board. Figure 2 shows the experience on a mobile device. The board displays each player’s power ranking, power ranking movement, and tour rank. A last updated timestamp provides the context as to which data was used for the current player power rankings. The experience is broken out into men’s and women’s power rankings to track the road to the championship.

Power Rankings leader board
Figure 2. The IBM Power Rankings leader board mobile experience

Here is how the IBM Power Rankings work.

IBM Power Rankings

Over 25 factors contribute to the IPR. Within the player performance dimension, a player’s win velocity, overall win ratio, and projected future win ratio account for win power. Next, the quality of a win, rank difference, injury status, tournament participation boost, round progression award, and win margin boost award players for meaningful play. Within natural language, the crowd’s opinion about a player’s performance and health is a large factor within the IPR. Both content sentiment and normalized volume are forecasted forward a day to provide leading indicators for IPR. At the same time, the overall assessment of the player adapts to the current grand slam with a refocus metric. This enables IPR to rapidly adapt to current play outcomes.

The IPR becomes an insight with the application of a predictive model called “likelihood to win.” The model has 30 features that include comparative elements of IPR. Several additional features that describe the historical head-to-head matchup between two competitors refine the IPR. Every singles match is assessed by the model, and a win probability is assigned to each player. The win probability can shift day by day as the data around punditry and performance changes. Figure 3 shows the overall architecture of the IBM Power Rankings system.

Power Rankings system
Figure 3. Power Rankings system

The core IPR system runs over IBM Functions, a serverless technology that can run code that is bootstrapped by containerized technologies. A series of triggers runs action functions on predefined schedules. The long-running ranking action calls itself as it processes players to maintain processing speed. Statistical data is pulled from SportRadar while punditry is queried through Watson Discovery. The functions code calls Red Hat OpenShift RESTful services that apply natural language processing techniques to the text. The volume and sentiment trends of the queried data are forecasted a few days into the future by Watson Core OneNLP. A spike forecaster using Watson Studio to train AutoAI that is deployed on Watson Machine Learning helps to discover anomalous future situations. The results of the data are stored within Db2.

At the end of each player’s IPR process, a feature vector is posted to a likelihood to win Python application that is running on Red Hat OpenShift. The feature vector is normalized, and missing values are imputed before being posted to the likelihood to win predictive model. The resulting probability of a win for two players within a match are saved to Db2. A Cognos dashboard pulls data from Db2 into data visualizations. In parallel, IBM Code Engine aggregates likelihood to win and IPR data together into a JSON file for upload into an IBM Cloud Object Storage. The IBM Code Engine publisher creates data that feeds into the US Open experience.

Experience the IBM Match Insights with Watson

While the system creates the IBM Power Rankings, the Match Insights system is applying natural language processing, AI, and statistical analysis to tennis-related content. The IBM Power Rankings, likelihood to win, and Match Insights are joined together within a singular experience, as shown in Figure 4. The most meaningful insights provide transparency to both the upcoming tennis match and to the power rankings.

IBM Match Insights experience
Figure 4. The IBM Match Insights experience

In the media section of IBM Match Insights with Watson

We decided to focus on core media outlets to answer key questions. What makes a player interesting? What happened in their career to lead them to play at the US Open? Player Insights with Watson seeks to uncover the answer to these types of questions, along with any other facets of a player’s background that make them stand out from the field.

To achieve this, Watson searches for information on a given player across millions of news articles, blog posts, and other online media, supplemented by deep dives on a targeted selection of tennis sources, such as Watson has a deeper understanding of the editorial content through natural language processing enrichments such as categorizing articles by their prevalent topics. Articles deemed relevant to both the player and the topic domain are then summarized using extractive algorithms.

Having collected relevant articles and extracted salient information, the next stage is to assess each of the sentence’s quality. Two dimensions are used to determine the quality of a given snippet: its grammatical coherence, which is determined by scikit-learn surface form parse rules and decision tree, and a trained machine learning model that measures topic alignment. Sentences that pass a quality threshold are determined to be insightful and are stored in our Cloudant natural language processing store as factoids. The factoids are stored as JSON documents by topic/player and are then sent through our Insights Human Review Tool. Here, human operators review and approve the stored factoids.

Figure 5 depicts the architecture of the factoid system.

Figure 5. Tennis factoids architecture

By the numbers section of IBM Match Insights with Watson

The on-court action at the US Open produces dozens of distinct statistics for fans and tennis experts to analyze. These statistics are particularly useful when previewing an upcoming matchup, as they can indicate the relative strengths and tendencies of each player. Does this player hit many winners from her forehand? Does the player often approach the net? Statistics can answer these questions and many more, giving fans insight into the forthcoming match. A skilled analyst can study data tables and uncover the areas in which each player stands out. Match Insights brings this level of comparative analysis to statisticians and casual fans alike by presenting the data in natural language.

IBM maintains databases that store these statistics and other relevant information using the Db2 on Cloud service. In their raw form, these stats are still difficult to interpret. Comparisons are difficult to make because matches can differ in length, from under 1 hour to over 4 hours. To normalize for this variability, IBM calculates per-point frequencies. Each frequency is then converted to a rank value with respect to that statistic among the entire tournament field of 128 competitors.

The most extreme values are the items that will be most interesting to the tennis audience. Additionally, Match Insights draws contrasts by highlighting the stats with the largest percentile differences between the two players in the matchup. After these key stats are selected, the system converts the stats to natural language. To do this, the system must understand the various components of a statistical highlight. These components include the subject phrase, verb phrase, and contextual phrase. As humans generate natural language using various word choices and syntactical ordering, the AI system also varies these elements to produce human-like language.

The output structure and diction are then selected according to probability. At this level of variety, the natural language generation system, which is powered by open source natural language generation and IBM Research technologies, can produce hundreds of unique texts for each match’s selected stats. Additional processing then confirms grammatical correctness such as pronoun, article, and verb agreement.

The final task of the Literature Generator web service is to persist the texts and corresponding metadata to a Cloudant NoSQL database on IBM Cloud, which feeds the human review UI. After a Match Insights package receives approval, an IBM Code Engine application joins the statistics with corresponding factoids and writes a JSON document to a bucket on IBM Cloud Object Storage. The contents of this bucket are delivered on using the IBM Content Delivery Network. The Content Delivery Network is well equipped to serve the high traffic for these data files to power the Match Insights features on

Natural language generation
Figure 6. Natural language generation for tennis architecture

Fantasy tennis

The US Open Fantasy experience, which is enriched with IPR and Match Insights data, allows tennis fans to join the action. Anyone can join the game at any time before the start of round 3 to play with the help of our insights. Initially, users can consult the IBM insights to help them pick 4 men and 4 women players. This is especially important information because users can only pick one player from the men’s side in the ATP top 16 and one player from the women’s side in the WTA top 16. After those players have started the first round, they are locked onto your roster until the redemption round. Particularly important is the redemption round, which is between rounds 3 and 4. During this time period, the IPR and Match Insights uncover high potential players that can replace 1 male and 1 female on your fantasy roster if they are active within the tournament.

Now the stakes get higher with your additional choices during the redemption round. Here, any eliminated player can be replaced by any active player within any quadrant. The movement and magnitude of the IPR will provide users with objective player potential understanding. In combination with Match Insights, users can select players to optimize their fantasy scoring total with a Watson advantage. Highly trending IPR for a selected player is important because a match win is worth 100 points, serving an ace is 2 points, and breaking a serve is 5 points.

The IBM Power Rankings and Player Insights will be available on a player card within the fantasy tennis experience. The data changes throughout the tournament to be timely, relevant, and accurate. Figure 7 shows a fantasy tennis player card.

Fantasy tennis player card
Figure 7. US Open fantasy tennis player card

Enjoy the tournament

Tennis data that matters the most helps fans enjoy the game and win fantasy tennis. This increases the longevity of fan engagement while preparing them for the next generation of tennis play. The current stars of the US Open are soon to retire after their historically long reign, and fans are in need of their next US Open hero.