Sign In

Communications of the ACM

ACM TechNews

Microsoft Machine Learning Tech Adds Captions to Images


View as: Print Mobile App Share: Send by email Share on reddit Share on StumbleUpon Share on Hacker News Share on Tweeter Share on Facebook
Microsoft is developing software that automatically captions images.

A look at how Microsoft's new software creates captions for images.

Credit: Microsoft Research

Microsoft is now leveraging its machine learning research to build software that automatically captions images.

Trained with machine learning, the software has bested human-generated image captioning in Bilingual Evaluation Understudy (BLUE) testing. However, it was somewhat below human performance with regard to testing on the Metric for Evaluation of Translation With Explicit Ordering, according to Microsoft Research's John Platt.

The researchers used Amazon's Mechanical Turk service, which pays "workers" to complete Human Intelligence Tasks online, but many people preferred the captions that were generated by Microsoft's BLUE technology. "People thought that the system caption was the same or better than a human caption," Platt notes.

He and his multidisciplinary team of experts launched the project this summer. Platt notes machine learning is nearly pervasive throughout Microsoft's products, including the Bing search engine and the Kinect motion sensor. "The only way you can answer the billions of questions Bing answers is to have something that operates autonomously," he says.

"The Kinect was also trained with machine learning. The fact that it can see you in the room even though it's poor lighting and you can wave your arms and it can track you--that's all done with a piece of software that was trained with machine learning."

From eWeek
View Full Article

 

Abstracts Copyright © 2014 Information Inc., Bethesda, Maryland, USA


 

No entries found