Cloud9 Video Review Tool

It began in 2019, when Cloud9 and Microsoft tried an experiment together by running a hackathon with members from C9’s professional esports teams, their Lead Data Scientist, and passionate Microsoft gaming enthusiasts. The hackathon was a special way for C9 and Microsoft to collaborate on new ideas and tools to improve player and team performance. “We got a huge response from employees who wanted to help,” recalls Thomas Labuzienski, Brand Partnerships Manager, on the Microsoft team working in the esports space. “We have cultivated great partnerships where we empower organizations to do interesting things with Microsoft technologies like improvements to player performance and fan engagement.” The first mini-hackathon was such a success that C9 participated again in the larger, company-wide Microsoft Global Hackathon in 2020.

During the 2020 Hackathon, the group behind Watch For internally shared an experimental computer vision toolset for employees to hack on and create their own amazing projects, training custom computer vision skills. C9 sponsored a challenge asking Microsoft employees to work with C9 data scientists to come up with ideas for team performance improvements. Many of the ideas leveraged the toolset, including the sponsored hack challenge winning project: the Video Review Tool (VRT), created specifically for analyzing Valorant gameplay. The hackathon project team was a small group of international employees who were complete strangers that rallied around esports, something they were all passionate about.

Valorant is a 5v5 character-based tactical FPS hero game, and C9 is home to two competitive all-star squads, the Cloud9 and the Cloud9 White team. The hack project was a proof-of-concept for a front-end, user-experience that enabled managers, coaches, and players to watch their videos and jump to key moments in a game, with the help of the computer vision trained models to identify these moments, like spikes planted, ultimates used, and players defeated.

“It started with passion and a simple idea. I used to record matches on a VCR to help analyze my own gameplay and see how I could improve. This clearly wouldn’t be efficient in today’s world, but a lot of teams, players, and coaches still resort to reviewing full videos-on-demand. Shortly after we kicked off our Hackathon project we quickly realized there was a lot more that could be offered with machine vision learning. While this is still just scratching the surface, I couldn’t be more excited for the team and final product!” said Alexander Castro, Program Manager at Microsoft and part of the original hack project team.

The Game Library page of the Video Review Tool, showing the recently uploaded matches available for review

After the 2020 Hackathon, Halee Mason, Lead Data Scientist at Cloud9, worked with Microsoft esports group to identify a collaborator that could help take the prototype and build it into a working tool capable of handling a robust data pipeline on the backend. Halee and C9’s in-house developers worked closely with a partner company, Southworks, to build the latest iteration of the tool that’s now being used by C9’s Valorant squads and coaches. The original hackathon team was there to assist and excited to help usher in the next phase of the tool.

“The VRT allows the players and coaches to review in-game recordings of their matches (both practice scrims and professional matches) and analyze specific scenarios or certain rounds. This tool has enabled the coaches to conduct a deeper analysis of the players performance and quickly jump to key moments of interest such as kills, spike plants, or ultimates used in the game,” said Halee.

She explained the steps needed to make the prototype a reality. “In order to productionize, it needed a data pipeline and processing component, a database, and the models needed to run in production on Youtube or Twitch videos, for example. All of it is stored in Azure to be called upon and served back to the front-end tool.”

Though Halee and her small team of developers plans to continue making improvements and new features to the tool, it is stable and in production for C9 squads to use. Teams are already experiencing the benefits.

Both the Cloud9 teams are using the VRT to discuss and analyze team performance. Coaches and players can log in and see their recorded games and scrimmages, and easily jump to automatically detected key events in the video. “For Valorant specifically this tool really served a need in the spectator realm that didn’t exist,” said Halee. In addition, the tool is built to easily integrate new skills (identifying and classifying more events), and it can be applied to other game titles, with a few tweaks.

The hackathon helped connect employees around the world to C9’s data analysts, coaches, and players, hacking across different game titles like League of Legends to Valorant to Super Smash Melee. Though C9 gained new insights and ideas from the many hack projects, the Video Review Tool proved especially advantageous to build out and met a significant need.

“Our goal is to integrate technology in an impactful way for partners like Cloud9,” said Thomas. “I’m happy we helped their teams and that we’re able to tell that story.”

At the 2021 Valorant Champions tournament, the C9 team was the only North American team to reach the playoffs. As of writing, they are the #1 team in North America according to vlr.gg rankings (Valorant Competitive Team Rankings: North America | VLR.gg), and consistently rank in the top 3. In addition, Cloud9 White, the all-women’s team, won all three VCT 2021 Game Changers events.

Learn more: Cloud9 takes esports to the next level – Microsoft In Culture