Sign In

Communications of the ACM

ACM TechNews

Virtual Learning: Using AI, Immersion to Teach Chinese


A student practicing tai chi with an avatar.

As part of the "Mandarin Project," a joint venture of Rensselaer Polytechnic Institute and IBM, researchers in the university's Cognitive and Immersive Systems Laboratory are developing a smart room that can understand students' words, answer their questions, and perceive their gestures.

Credit: Michael Hill/AP

Researchers at Rensselaer Polytechnic Institute (RPI) and IBM have developed an artificial intelligence system that relies on immersive scenes to teach Mandarin.

The researchers created a smart room that can understand students' words, answer their questions, and perceive their gestures. Lessons in Mandarin are presented as games or tasks, such as ordering a meal.

RPI researcher Hui Su says, "Our plan is to complete several scenes of real life in China, to let the student be able to have a virtual trip over there."

As part of the Mandarin Project, computers simultaneously interpret speech and gestures to keep a dialogue going and provide immediate feedback. This type of technology could be applied in other settings such as corporate boardrooms, where it could seamlessly provide relevant information during discussions, according to RPI's Shirley Ann Jackson.

From Associated Press
View Full Article

 

Abstracts Copyright © 2018 Information Inc., Bethesda, Maryland, USA


 

No entries found