How Human is Your Robot?

Have you ever done a double take after being told a certain robot is "humanoid" or "humanlike"? Presented with a squat, caterpillar-tracked, one-eyed droid—or perhaps a robot with a conical body sporting fluttering eyelids and animated lips—it can be difficult to take claims of a robot's human similarities seriously. Yet roboticists do apply such labels if their creation has just a couple of humanlike characteristics, such as friendly hand gestures or expressive eye movements. 

As robots increasingly enter human environments like homes, shops, and hospitals, some researchers say it is becoming more important that such human-like claims are accurate. People engage more with truly human-like robots, researchers at Aachen University in Germany have found, while psychologists at the University of Auckland in New Zealand have found patients prefer to interact with healthcare droids with human faces rather than robots that lack them. Carnegie Mellon University researchers have found that people are much more likely to take advice from human-like robots with whom they feel they can empathize.

Until now, there has been no way to objectively assess the human similarity of different robot designs, so roboticists have been free to make extravagant claims. That changed recently, when Brown University social robotics researchers Xuan Zhao, Elizabeth Phillips, Daniel Ullman, and Bertram Malle suggested a way to gauge a robot's similarity to a human objectively.

Speaking at the 13th Annual ACM/IEEE International Conference on Human Robot Interaction (HRI 2018) in Chicago, the team revealed a way to 'score' a robot on a scale of 0 to 100 for its human similarity, with zero representing "not at all human-like," and 100, says Zhao, being akin to something indistinguishable from humans, like the replicants in the Blade Runner movies.

What the Brown researchers have developed is an online human likeness estimator, which they are calling the Anthropomorphic Robot Database, or ABOT for short. To assign a human similarity score to a new robot design, users log on to ABOT and answer 18 questions about the robot's physical appearance; an algorithm then calculates the robot's human-similarity score.

"Our study is the first to empirically identify the specific physical features that make up the overall perception of a robot's human-like appearance," says Zhao.

Designing that score-calculating algorithm was far from simple. The algorithm has to identify the physical robot features that people on the street will agree lends a robot a human-like appearance. That being the case, the public's opinions on robots had to be considered.

To accomplish this, the team obtained photographs of 200 types of robot that are on the market now or in research labs around the world, and tasked 98 people on Amazon's Mechanical Turk online crowdsourcing system with rating each robot for how human-like they thought each to be.

After that, the researchers tasked more than 1,000 people, also on the Amazon Mechanical Turk platform, with identifying which of 18 physical features each rated robot had, including eyes, eyebrows, torso, head, indication of gender, hair, arms, hands, fingers, grippers, legs, wheels, and caterpillar tracks. That allowed Zhao and colleagues to create an algorithm that correlated the perceived human likeness ratings to the presence of those physical features.

Are you wondering how some popular robot models scored? Surprisingly, the doll-sized, bipedal Nao robot, made by Softbank of Japan, which is often cited as humanoid, scored below 50 (45.9 on the ABOT scale), while its larger wheeled sister robot, Pepper, used as a customer helper in the retail industry, scored 42.2. At the low end of the spectrum, the dancing tabletop digital assistant Jibo scored 1.4, while Hiroshi Ishiguro's robotic android doppelgänger Geminoid scored highest with a 92.6.

"We don't consider any of the scores good or bad.  They simply refer to where people perceive the robot on a broad scale," says Zhao. She explains, "Nao and Pepper are in the mid-range because they have some of the basic characteristic features of human likeness, such as body manipulators, a head and a face, but not some of the more fine-grained ones, like skin and detailed eye features such as lashes or brows."

Those lashes and brows had Softbank taking issue with ABOT's methodology. "We are not surprised by this grading of our robots because most of the criteria, like hair and eyebrows, are not specific to humanoid robots but rather to androids," says Rodolphe Gelin, chief scientific officer at SoftBank Robotics Europe in Paris, France. Based on those criteria, he says, it is not surprising that Ishiguro's androids do well.

"We do not try to make robots look precisely like humans, and do not want people to wonder whether they are interacting with a machine or a human," Gelin says. "We believe that the behavior of the robot, its body gestures, attitude and speech, plus the way people interact with it, are more of a factor in defining its humanoid side and promoting interactions."

The Jibo result is not too surprising, says Zhao. "This low degree of human-likeness in its physical appearance might be intended and even desired from the perspective of the designers," she says.

Whatever their ABOT score is, designers can use it to scale a robot's features up or down to improve or degrade its humanness level as they see fit. A robot with a head, face, nose, and eyes will garner a 23.7 score, for instance, but adding eyebrows can boost that to 30.1.

Roboticists have given a qualified welcome to the ABOT scale, seeing it as a promising start towards grading human-likeness for robots, but one that needs to incorporate data on how the machines move and interact with people, in addition to their physical appearance.

"There is value to what they've done. I'm going to use their scores to check whether a more human-like robot has more impact on how much people learn from a robot," says Tony Belpaeme, professor of cognitive systems and robotics at the University of Plymouth, U.K.  However, Belpaeme has concerns, which he says are shared by other roboticists, over the fact that the ABOT scoring technique is based on crowdsourced views of static photos.

"People got to see carefully edited photos of the robot cut out and presented against a white background. There are two problems with that: first, it's a picture, so anything that makes a robot a robot, like motion, is missing, and scale is lost in the pictures. Are the robots large or small? There's no way of telling."

Meeting an actual robot, Belpaeme says, "is a visceral experience, one that leaves you with a different impression to the kind you get from seeing a photo."

Rich Walker, managing director of London-based Shadow Robot, a maker of dexterous human-like robot hands, is impressed with ABOT.  "It's really useful to have a way to understand how robots present to humans. As part of the design of a system, you might want to make your robot very natural or very unnatural, and that might become a really important way for us to interact with robots," Walker says.  "You could imagine having the social model that humanlike robots are safe to be around, for instance, whereas 'inhuman' robots are hard-working busy machines that don't spend their time being soft and safe—like tractors or excavators."

At the Intelligent Robotics Laboratory of the Department of Systems Innovation in the Graduate School of Engineering Science at Osaka University in Japan, Hiroshi Ishiguro, the maker of eerily realistic androids (like Erica, with an ABOT score of 89.6), says such a scale needs to take robot-human communications into account as well.

"Appearance is important but it is not all," Ishiguro says. "We need to evaluate the humanity or human-likeness of robots through their interactions with us."

Paul Marks is a technology journalist, writer, and editor based in London, U.K.