Solving Problems with Artificial Intelligence and Machine Learning
On September 3, 2019 I attended a wonderful meetup about the way the USTA (The United States Tennis Association) parses thousands of hours of tennis video footage, looking for the best moments to present to fans. The meetup also explained that similar methods are being used at sporting events in general, and also discussed fantasy football and how participants (acting as virtual team managers) sift through the news, analysis, and statistics generated about players in order to make more informed decisions. Throughout the talk popular open source AI frameworks were discussed, like PyTorch, TensorFlow, and Theano, and how they fit in the various machine learning pipelines that support the tennis tournament. The session also outlined the huge reduction in installing the whole system from 3 days to 1 hour via Docker Compose. The session explained how ethical AI methods are used to avoid bias when reporting on an event. The speakers, IBM Distinguished Engineers Aaron Baughman & Stephen Hammer, were extremely knowledgeable with long careers in using tech in a sports setting. The meetup was well attended and the audience was animated.
Sports Video Highlights
The speakers introduced:
- Narrow AI: Learning from sports data
- Broad AI: Transfer Learning, Learning from Less
- General: Self Learning, Self Organizing, AI at the Edge
They explained that they are working towards general AI for sports – then went on to explain how the Wimbledon Tennis and USTA sports video highlights project. The goal is to automate the speedy generation of highlights from the video produced from filming at the various tennis courts (7 courts at the US Open, and 10 courts at Wimbledon).
- For the US Open – 150 highlights packages were produced from 31,000 video scenes
- For Wimbledon – 375 highlights packages were produced from 66,000 video scenes
The speakers discussed how scene changes are identified in a long video, and the first frames are collected, and appropriate rough cut clips are built from which highlights videos are created- always ending with the succeeding player’s reaction. They covered how Watson Visual Recognition is used to identify the celebratory gestures of a player â€“ an arm raised overhead or a fist pump in celebration. (I wonder how the system deals with Daniil Medvedev who sometimes raises his arms in celebration of being boo-ed!) Watson can detect emotion in a playerâ€™s face, and it assigns each reaction a score.
Detection of Tennis Events from Acoustic Data
The speakers talked about a paper they co-authored and that will be presented at the upcoming ACM multimedia conference in Oct 2019 in Nice https://acmmm.org/. The paper entitled “Detection of Tennis Events from Acoustic Data” https://researcher.watson.ibm.com/researcher/files/us-wangshiq/AB_MMSports2019.pdf is based on work originally conducted in a manufacturing setting. In the paper they present a system that detects events such as ball hits and point boundaries in a tennis match from sound data recorded in the match. They described the sound processing pipeline that includes preprocessing, feature extraction, basic event detection, and point boundary detection. The acoustic processing augments the visual analysis mentioned earlier and enhances the quality of the scenes clipped – because of a better understanding of the events taking place in the video being analyzed.
The final segment of the session covered fantasy football and the use of AI. Fantasy football is a game in which the participants serve as general managers of virtual professional gridiron football teams (from wikipedia ). Fantasy Insights with Watson on ESPN https://www.ibm.com/sports/fantasy/ had recently launched and we heard how unstructured text is analyzed to and a machine learning pipeline is stuctured to advise Fantasy Football managers.
Tweets and Blogs
Thank you New York Developer Advocacy team le by Kelcey Gosserand for producing such a great session – the speakers and content were fantastic.
Blog : The future of the fan experience at the us open https://www.ibm.com/blogs/watson/2019/08/the-future-of-the-fan-experience-at-the-us-open/
— gsteinfeld (@gsteinfeld) September 4, 2019