Three years ago, a team of Microsoft employees joined together on a quest to make gaming more fun and accessible to people who are blind or have low vision. After years of hacking on machine learning algorithms in their spare time, talking with customers in the blind and low vision community, and partnering with gaming experts, the team releases their first title: Ear Hockey, a Microsoft Garage project. With in-game audio cues that enable gamers to play with or without for visuals, Ear Hockey was crafted using inclusive design to promote accessibility in game development.
When spatial audio meets accessible game development
Ear Hockey is a Pong-style, first-person paddle game, where players use their keyboard or controller to hit the ball back and forth with a partner or computer player—but with a twist on the arcade classic air hockey: it uses unique spatial audio cues, powered by Microsoft’s HRTF spatial audio engine for Unity, to signal to the player where they are in the play-space and where the ball is traveling. The team was inspired by and worked with Daniel Kish, an expert in echolocation and President of World Access for the Blind, to craft special sounds for ball movement and contact with the paddle or other play space surfaces to help orient players. For example, a fine-tuned combination of low-frequency humming and high-frequency ticking, along with other contextual sound effects, makes the ball easier to locate with audio alone.
As they developed Ear Hockey, the team also consulted with Senior Program Manager Brannon Zahad, who has over 16 years of experience in the gaming space and works on Accessibility R&D. “One of the biggest challenges we have in the industry right now is: when a developer wants to make a game, it’s really overwhelming to think about all the different things they need to do to make their game accessible,” describes Brannon. “For the longest time, there’s been a perception that it would be impossible to build a first-person perspective game accessible to people who are blind, for example. The work this team is doing to evolve those perceptions and build technology that empowers game developers is, frankly, incredible.”
The grassroots team, which goes by Audio Augmented Reality Gaming, or AARG, originally set out to tackle making games more accessible for people who are blind or have low vision. After winning the Ability Hack category at Microsoft’s annual Hackathon in 2017, they met with several people working towards similar goals across Microsoft, including the Microsoft Researchers who developed Project Triton. The spatial audio solution creates realistic reverb effects based on objects in a video game’s map using sophisticated machine learning algorithms. The AARG team recognized the potential for this kind of audio capability for accessible game development, and has since been partnering closely to explore how this could improve not only the gamer’s experience, but also the developer’s experience in building a more inclusive game. The game studio who developed Gears of War 4, The Coalition, used Project Triton to bring next-level sound to life, featuring the possibilities of advanced reverb effects; the AARG team quickly realized they could help inspire even more innovation in this area by building their own title.
Inclusive gaming
AARG is comprised of a passionate set of team members from every corner of the company, many of whom have never formally worked on game development. To ensure Ear Hockey was a fun experience for both people who are blind or low vision, as well as people who are sighted, the team attended one of the world’s largest accessibility summits, the CSUN Assistive Technology Conference, where they showcased a few experimental games to members of the blind and low vision community. Players of all experience types provided feedback that helped shape the game. “For me, I realized the importance of orienting the player. We’d taken for granted how critical it is to provide that information at the beginning of the experience,” shared Patrick Farley, a Content Developer in the Cloud and AI group and one of the core architects of Ear Hockey. “Since then, we’ve enhanced our tutorial with a lot more detail.” For Boris Baracaldo, a Software Engineer and the other primary game developer, he noticed there was room for more customization with respect to ambient noise. Ear Hockey gamers can now adjust the volume of background music because of the invaluable customer feedback the team received.
In addition to soliciting feedback from customers and employees from the blind and low vision community, the team also designed Ear Hockey with a visual experience, subtitles, and support for the Xbox Adaptive Controller, another Hackathon project, in an effort to make the game more inclusive for a wide range of players. This has influenced how the team approaches developing for accessibility in their day-to-day activities. “I’m a Software Developer in Dynamics and this has completely shifted my approach to making products accessible,” continued Boris, who has found new ways to apply inclusive design to the products he works on in his full-time role.
Try it Out
With Ear Hockey now available worldwide, the team is eager to get feedback from players and drive a larger conversation around accessible gaming and game development. When asked about the team’s journey over the past 3 years, original organizer and Senior Mechanical Engineer Brandon Arteaga spoke of their excitement to offer this fun experience and help promote awareness about the power of echolocation. “We support Daniel Kish’s mission to teach echolocation as a skillset which can empower members of the blind and low-vision community to explore the world more freely. Anything we can do to shine a light on his work would be a dream come true.” Wilson Dreewes, a Senior Software Developer in AARG added, “Our goal is to showcase feature sets and what can be done in this space. We hope that the games we’re releasing now would be fun in and of themselves, but also that they inspire more innovation in game development.” So, have fun playing! Please try out Ear Hockey and share any feedback to the team on the game or accessible game development via UserVoice. You can also read about the full game features on the Garage Workbench.
