Sign In

Communications of the ACM

ACM TechNews

Microsoft Machine Learning Tech Adds Captions to Images

View as: Print Mobile App Share:
Microsoft is developing software that automatically captions images.

A look at how Microsoft's new software creates captions for images.

Credit: Microsoft Research

Microsoft is now leveraging its machine learning research to build software that automatically captions images.

Trained with machine learning, the software has bested human-generated image captioning in Bilingual Evaluation Understudy (BLUE) testing. However, it was somewhat below human performance with regard to testing on the Metric for Evaluation of Translation With Explicit Ordering, according to Microsoft Research's John Platt.

The researchers used Amazon's Mechanical Turk service, which pays "workers" to complete Human Intelligence Tasks online, but many people preferred the captions that were generated by Microsoft's BLUE technology. "People thought that the system caption was the same or better than a human caption," Platt notes.

He and his multidisciplinary team of experts launched the project this summer. Platt notes machine learning is nearly pervasive throughout Microsoft's products, including the Bing search engine and the Kinect motion sensor. "The only way you can answer the billions of questions Bing answers is to have something that operates autonomously," he says.

"The Kinect was also trained with machine learning. The fact that it can see you in the room even though it's poor lighting and you can wave your arms and it can track you--that's all done with a piece of software that was trained with machine learning."

From eWeek
View Full Article


Abstracts Copyright © 2014 Information Inc., Bethesda, Maryland, USA


No entries found

Sign In for Full Access
» Forgot Password? » Create an ACM Web Account