Google has introduced two new AI-powered features into Google Translate. According to Google, people can translate around 1 trillion words across Google Translate, Search, and in visual translations in Lens and Circle to Search. The new features aim to help users with live conversations and language learning and are backed by the advanced reasoning and multimodal capabilities of Gemini models.
To help users connect with people from across cultures in a meaningful way, Google has introduced the ability to have a back-and-forth conversation in real time with audio and on-screen translations through the Translate app. Google said that it has built upon existing live conversation experience.
Google said it heard from users that the toughest skill to master was conversation — ie, learning to listen and speak with confidence. To that end, it’s piloting a new language practice feature (on iOS or Android) targeted toward an individual’s specific needs.
To create tailored listening and practicing sessions, the new learning tool posts a couple of questions. It first requests which language you want to learn (like Spanish) and your your current level, then asks “What’s motivating you to learn Spanish?” From there, it will generate customized scenarios that allow you to either listen to conversations or practice speaking, with helpful hints available when needed.
The app was “developed with learning experts based on the latest studies in language acquisition,” Google explained in a blog post. To that end, it can track your daily progress to help build your language skills, possibly as an aid to Duolingo and other dedicated language learning apps. “We see what we’re doing right now as really complementary to other things out there,” Google product manager Matt Sheets said in a media roundtable. “So whether you’re taking classes in a formal educational setting or doing immersion experiences, we think this is something that can work alongside of those.”
Following early testing, language learning is rolling out more widely as a beta experience for English speakers practicing Spanish and French, as well as Spanish, French and Portugese speakers working on English.
Google also introduced AI-powered live conversations, a feature that builds on Translate’s existing conversation feature by making it more seamless and adding more powerful models. It offers the ability to have a back-and-forth conversation in real time with audio and on-screen translations right inside the Translate app.
It’s available in the Translate app for Android or iOS when tapping on “Live Translate.” First select the languages you want to converse in and being speaking. You’ll hear the translation out loud and written in both languages on your devices. It will switch between the two languages spoken by you and the other party, identifying pauses, accents and intonations so you can have a natural conversation.