University of Washington (UW) researchers have developed a system that can take audio clips from one speech and sync them with video clips from another speech, creating realistic-looking lip-synced videos.
The researchers say the technology could be used to apply a moving face to historic audio recordings, or to improve videoconferencing.
In a demonstration of the technology, the researchers used video of former U.S. President Barack Obama from a range of appearances and combined it with audio spoken in separate instances.
A recurrent neural network, trained on many hours of Obama's weekly address footage, learned the mapping from raw audio features to mouth shapes.
"These type of results have never been shown before," says UW professor Ira Kemelmacher-Shlizerman.
The researchers note the technology also could lead to better video chat performance.
The system will be presented at the upcoming ACM SIGGRAPH 2017 conference in Los Angeles, CA.
View Full Article
Abstracts Copyright © 2017 Information Inc., Bethesda, Maryland, USA
No entries found