Implement computer vision for PlayerUnknown’s Battlegrounds live streams  

Build a web app that uses OCR on live-streaming video platform Twitch

Last updated | By Cullen Taylor, Spencer Krum

Description

PUBG Rotisserie features a video stream of a player in the extremely popular multiplayer video game, PlayerUnknown’s Battlegrounds (PUBG). It performs optical character recognition (OCR) on the live streams found on Twitch to identify which streams have only a handful of players left and are close to the final moments of play. In this developer journey, learn one of the many ways to implement computer vision to derive data from a video.

Overview

On Twitch and other websites, professional and regular computer players live-stream their games. At any given time, thousands of people are streaming on Twitch, and hundreds of thousands of viewers are watching those streams.

PUBG is a first-person or third-person shooter game. One hundred players spawn on an island, searching for loot and killing other players’ characters. There are no re-spawns. Only one player or team can win. Think Hunger Games meets Battlefield. The problem is that part of this game is relatively boring.

Watching a streamer when close to winning (25 people or fewer left alive) is very exciting. But watching a streamer when there are 90 or even 50 people still alive is much less exciting. This developer journey shows you one of the many ways to implement computer vision to derive data from a video. You will learn how to set up this application locally or in containers. You will also dive into deployment in Kubernetes, a container orchestration platform.

PUBG Rotisserie watches all the streams available on Twitch and switches between streams, showing the streams that are the most exciting. This can be left on a second monitor or on a TV in a lobby or sports bar.

Using computers to watch the entire streaming space for a game is a new thing only PUBG Rotisserie does right now. Using algorithms to switch between the streams like a virtual director is another innovative step.

Flow

  1. The main application uses the Twitch API to fetch the list of PUBG streams. Livestreamer, an open source project, takes a short recording of each of those streams.
  2. FFmpeg is used to take a single frame out of that short recording.
  3. That frame is then cropped with ImageMagick so the image only contains the number of players alive.
  4. The final image is pushed to the OCR service that uses Tesseract, an open source OCR engine. This service reads the number on image.
  5. The translated data will be in a JSON block received by the main app. The app fetches another list of streams every 15 seconds.
  6. The user will see the stream with the fewest remaining people alive. The front end is served by the main Node.js app. The user’s client will check every 15 seconds and switch if another stream is found to have even fewer remaining players.

Related Blogs

Jax 2018 – Just An Awesome Experience

What a week! From 23rd to 27th April our Berlin team attended the Jax conference in Mainz, Germany. We had such a great time sharing our fresh perspectives, in the form of a rousing keynote and two informative sessions. The concept of this annual event with over 2,000 participants, revolves around innovating with Java, architecture,...

Continue reading Jax 2018 – Just An Awesome Experience

CloudNativeCon and KubeCon are coming to Copenhagen!

With May just around the corner, mark your calendars for an exciting event, CloudNativeCon/KubeCon, in Denmark’s capital city of Copenhagen. Many of us in the Cloud Native community already visited this beautiful city for DockerCon EU last year and we’re excited to be able to take in all of the wonderful sites again this year....

Continue reading CloudNativeCon and KubeCon are coming to Copenhagen!

Live analytics with an event store fed from Java and analyzed in Jupyter Notebook

Event-driven analytics requires a data management system that can scale to allow a high rate of incoming events while optimizing to allow immediate analytics. IBM Db2 Event Store extends Apache Spark to provide accelerated queries and lightning fast inserts. This code pattern is a simple introduction to get you started with event-driven analytics. You can...

Continue reading Live analytics with an event store fed from Java and analyzed in Jupyter Notebook

Related Links

Tesseract OCR

Learn more about the Tesseract open source OCR engine.

PUBG Rotisserie

The web app made by IBM developer advocates shows the most exciting PUBG live Twitch streams.