Sign In

Communications of the ACM

ACM TechNews

Lip-Reading AI Could Help the Deaf—or Spies


View as: Print Mobile App Share: Send by email Share on reddit Share on StumbleUpon Share on Hacker News Share on Tweeter Share on Facebook
Researchers taught artificial intelligence software to read lips through the use of neural networks.

U.K.-based Deepmind has created artificial intelligence software that can read lips.

Credit: Jake Olimb/istock.com

An artificial intelligence (AI) program from DeepMind can read lips better than professional lip readers after reviewing thousands of hours of YouTube videos along with transcripts via machine learning.

The researchers tested the program on 37 minutes of video it had not previously viewed, and it misidentified only 41% of the words. In comparison, the best previous computer method, which focuses on individual letters instead of phonemes, had a 77% word error rate, while professional lip readers had a 93% error rate in the same test, which lacked context or body language.

Columbia University's Hassan Akbari says the AI, if incorporated into a phone, would enable hearing-impaired users to have a "translator" with them wherever they go.

Helen Bear at Queen Mary University of London in the U.K. envisions applications for the program that include analyzing security video, interpreting historical footage, and understanding what a Skype partner is saying when the audio goes dead.

From Science
View Full Article

 

Abstracts Copyright © 2018 Information Inc., Bethesda, Maryland, USA


 

No entries found