There is healthy debate about the productivity of multi-tasking. Is it possible to take excellent notes while also being fully present in a meeting? Now, you don’t have to choose between focus and productivity with our latest experiment, Group Transcribe, a Microsoft Garage project. This app uses a multi-device approach to provide real-time, high quality transcription and translation, so users can be more present and productive during in-person meetings and conversations. Group Transcribe also joins a strong research tradition at Microsoft, finding new ways to improve upon speech and language AI. The app is now available worldwide on iOS.
Multi-device transcription and translation
To use the app, conversation participants start a shared session and each leverage their phone’s microphone to capture a highly accurate transcript, showing who said what in real-time. With confidence in the high-quality record of the conversation, users can skip note-taking and focus their attention on the conversation itself. Transcripts are easy to share or relocate after the fact.
Users can also leverage the real-time transcription to speak across different languages and make meetings more inclusive. The app supports several languages in over 80 locales and enables people who are deaf, hard of hearing, or non-native speakers to participate more fully by following along with the live transcript.
The Group Transcribe team collectively speaks over a dozen languages and dialects, and team members are passionate about connecting people across different cultures. “This can be a fantastic tool for communication. What I would love to see is for this to break down barriers for people speaking across multiple languages,” shares Franklin Munoz Principal Development Lead and one of the lead engineers who built the project.
Key features
Group Transcribe provides high-quality, real-time transcription and translation.
- Start a conversation from your phone and easily invite others to join*
- Stay focused without taking notes or pushing-to-talk
- Follow along with the conversation in real-time in your preferred language
- Automatically save the transcript after each session
- Browse and view previous transcripts from your home screen
- Easily share transcripts with others
- Supports languages in 80+ locales
*Please note: each user will need to use their own phone within arm’s reach for the highest quality experience.
Group Transcribe is powered by cutting edge speech and language technology that is years in the making. The project is designed around in-person meetings. It can be used during socially distanced conversations or in closer proximity once safe to connect in person per local guidelines.
Group Transcribe supports languages spoken in 80+ locales including: Arabic, Bulgarian, Catalan, Cantonese, Chinese (simplified), Chinese (traditional), Croatian, Czech, Danish, Dutch, English, Estonian, Finnish, French, German, Greek, Hindi, Irish, Italian, Japanese, Korean, Lithuanian, Latvian, Maltese, Norwegian, Polish, Portuguese, Romanian, Russian, Spanish, Slovak, Slovenian, Swedish, Thai, Turkish, and more.
Contribute to speech and language AI
In recent years, Microsoft has reached several speech and language AI milestones being the first to achieve human parity in a number of widely-used benchmark tasks. The Cognitive Services team continues to enhance the quality of speech and language models across a variety of research projects. Group Transcribe will allow Microsoft researchers to improve accuracy, latency, and speaker attribution.
“Microsoft has been leading the way in speech and language AI innovation with our research on distributed and dynamic microphone arrays as pioneered via Project Denmark,” shares Xuedong Huang, Technical Fellow and CTO of Cognitive Services. “Group Transcribe allows us to address an authentic, ad-hoc, interactive mobile scenario to help people communicate without language barriers. Improving these models with usage data will ensure that our customers will have the best experience.”
User privacy and the principles of responsible AI are first and foremost in Microsoft’s approach to research and product development. “Today, Microsoft Teams facilitates billions of conversation minutes each day, but because Microsoft goes above and beyond to ensure the privacy of our customers, our research does not leverage a single minute of this audio.” shares Pavan Li, the Principal Program Manager of the project. “In order for us to build quality AI products that are accessible to everyone, it is really crucial that we’re training AI models with highly relevant data.”
Inspired to provide a high-quality transcript experience accessible to the everyday customer and continuously improve speech and language models, the team found a way to achieve both goals with a combined solution. Pavan continues, “We came up with this concept of designing a mobile app that would not only help users experience more focused and productive conversations but would also open an opportunity for data donation. We hope consumers are willing to help us and join us on this journey.”
While aiding users with quality, real-time transcription and translation, Group Transcribe will help make AI more accurate and inclusive by enabling an option to contribute conversation data.
Prioritizing privacy through granular data controls
Microsoft is committed to transparency and choice with privacy, and in designing a mechanism to accept conversation donations, the Group Transcribe team allows users to determine what they share. “We’ve taken care to make data and privacy controls as granular as possible,” explains Senior Program Manager Masaki Itagaki who focuses on speech privacy. “I hope users leverage these to take the sensitivity of a conversation into account and withhold sharing those conversations on a case-by-case basis.”
The team has also compiled a detailed FAQ regarding the Group Transcribe project and privacy for any users who have questions about how Group Transcribe collects, manages, and uses data responsibly.
“We believe strongly in the importance of transparency in order to develop AI responsibly,” adds Xuedong. “Thank you in advance to our users whose choose to contribute conversation data—you will be helping us reduce bias, increase inclusion, and accelerate innovation in our quest to build AI that empowers everyone on the planet to do more.”
Try it out
Group Transcribe is now available on iOS worldwide. Users can share their feedback to the team via the app.
