Head-Mounted Displays to Aid the Disabled

Consumer electronics make many activities easier; it is not uncommon to make a dinner reservation online, get directions to the restaurant, and pay the check all by using a smartphone. If head-mounted displays (HMDs) like Google Glass gain enough traction with the public, that combination of mobile computing and applications might also make it easier for people with disabilities to navigate the world.

The availability and affordability of Google Glass made life easier for researchers who consider how head-mounted displays might help people with disabilities such as tremors and paralysis, speech problems, or hearing and visual impairments, says Leah Findlater, an assistant professor in the University of Maryland’s Inclusive Design Lab.  

Findlater and her colleagues presented some of their research at the CHI 2015 conference in Seoul, South Korea in April. One project uses a head-mounted display to help people with aphasia, a condition in which sufferers have difficulty summoning the words they want to say. Aphasia is often the result of some brain trauma, such as a stroke. The display offers users visual and auditory prompts.

Imagine, for instance, that someone goes to a grocery store around Thanksgiving, and needs to ask where to find the ingredients for a pumpkin pie. He can scroll through a pre-loaded grocery list, and the device will display both the word "pumpkin" and a picture, and speak the word to him through his head-set.

Similar apps already exist for smartphones, but the head-mounted display provides more privacy and an increased sense of autonomy, Findlater says. If the smartphone is displaying and speaking the word in question, the person the user is talking to can see and hear it and react to the handset, perhaps making the user feel left out of the conversation.

Another project seeks to help the deaf and hard-of-hearing, many of whom not only lip-read, but also draw cues from the facial expressions and movements of the person to whom they are speaking. That may be easy enough to do when talking to one person, but in a group conversation when speakers suddenly switch, the deaf person may not know where to look next. The Maryland researchers are developing a program that displays visual cues to a person wearing a head-mounted device. It might, for instance, display an arrow pointing forward when the speaker is straight ahead, and another pointing to the left when someone out of the user’s field of view starts speaking. If multiple people are talking at the same time, the system could display thicker arrows to represent who is speaking loudest; it might also display the name of an out-of-sight speaker, or distinguish speech from other sounds, such as laughter.

A device like Google Glass might even be useful for blind people, says Shaun Kane, director of the Superhuman Computing Lab at the University of Colorado, Boulder, where he studies accessibility and wearable computing. There are apps that use cameras to help identify objects, such as Vizwiz, which allows a blind person to take a picture and then ask sighted users a question about it. Kane is studying whether a head-mounted device can help orient the camera better, though he has not yet published any results.

Head-mounted displays might also substitute for smartphone displays for people with upper body motor impairments that make it difficult to operate a touch screen, or possibly even to get a smartphone handset out of one’s pocket. However, those same problems can make an HMD difficult to operate, or even to reach the controls mounted on the side of the display. Findlater found that people with such impairments were much happier controlling the device with separate touch-pads that they could mount in spots that were comfortable and convenient.

One advantage that could arise if head-mounted displays come into widespread use is that private apps layered onto consumer devices would render the assistive technology less visible to the outside world. If everybody’s wearing headsets, that could lessen the social stigma that might discourage people from using it to aid with a disability. "Providing assistive support with a mainstream technology is an advantage, because you don’t stand out as different," Findlater says.

Of course, the market did not latched onto Google Glass as a must-have gadget, but head-mounted displays may yet catch on. At the Consumer Electronics Show in January, Intel CEO Brian Krzanich announced his company was partnering with sunglasses maker Oakley to develop smart glasses for athletes.

Making accessibility a part of technology aimed at a wider audience brings down costs and makes it more likely a device will stay on the market and have technical support, says Kane, benefitting the disabled. "When you can make something accessible through an app, that just fundamentally changes the opportunities that are available," he says.

Neil Savage is a science and technology writer based in Lowell, MA.