Google has launched a new AI-powered feature that translates American Sign Language (ASL) into written English on its latest Pixel smartphones.
Announced by Google DeepMind on August 12, the Sign Language-to-Text (SL2T) model powers new sign-to-text functions in Gboard and Live Transcribe on the Pixel 11.
The technology allows users to sign towards their phone’s camera, with their signing translated into text. Google says the system uses body, hand and facial landmark data rather than sending the user’s raw camera footage to its servers for translation.
Google says the model was trained using more than 100,000 hours of data spanning more than 50 sign languages. The company developed the technology with input from Deaf people and established an AI Sign Language Advisory Committee involving Deaf organisations and experts.
For now, the consumer feature translates ASL into English. Google says it is working to expand the technology to additional sign languages.
The announcement raises an obvious question for Australian users – when, or whether, Auslan will be supported. Google has not announced Auslan support in its current release.
Google’s Pixel 11 Pro Fold announcement says users can sign ASL to the phone’s front camera and see the translation appear as text on the outer display.
To read the full announcement, visit Google DeepMind’s article on the launch of its Sign Language-to-Text technology.