Technologies Support Dementia Patients

Swedish computer gamer and technologist John Nilsson did not intend for a project he originally developed to improve the user experience for games to become a globally recognized platform for people who may never have used a computer in their lives.

Nilsson's technology, the jDome, is an immersive domed screen with a 125-degree field of view that creates a virtual reality (VR) experience without the need for a VR headset. Through some serendipitous marketing opportunities and conference introductions, Nilsson's domed screen has become the centerpiece of BikeAround, a device that allows patients with dementia (a wide range of symptoms associated with a decline in memory or other thinking skills severe enough to reduce a person's ability to perform everyday activities) to "ride" a bicycle anywhere they wish, particularly through the neighborhoods of their past.

The BikeAround has become such a hallmark of the assistive technology ecosystem that it has become a highly visible promotional vehicle for Google—the system uses a form of Google Street View, modified by one of Nilsson's software engineers into StreetViz, as its visual interface. As early as 2013, Google executive chairman Eric Schmidt was talking about the technology at conferences such as Zeitgeist Europe; and in September 2017, Google released an updated video of the BikeAround in action, featuring narration by Anne-Christine Hertz, a biomechanical engineer and project coordinator at The Center for Health Technology in Halland County, Sweden, and Sweden's Halmstad University, who adapted Nilsson's immersive game screen to assist dementia patients.

"We started out around 2010 with the idea of the BikeAround," Hertz said in a recent conference call with Nilsson and representatives from the company that now owns the jDome and BikeAround technology, the Swedish firm Camanio Care. "We wanted a screen that had a three-dimensional experience, that you could feel like you were there. Somebody in our network had been to a technology fair in Stockholm who had seen John's screen and was really excited about it, so we bought one to use as our prototype.

"A few months later," Hertz told Nilsson, "you and I spoke on the phone and you asked me 'What are you doing with my screen?' and when I said we were doing a dementia project, you said 'What?'"

Nilsson's initial surprise quickly turned to enthusiasm, Hertz said.

"It was really cool;  John was so surprised and into gaming. And we all realized this is actually perfect for the target group we were looking at, and we started to collaborate. It was really easy."

New tech for aging populations

The BikeAround may be one of the most visible new technologies aimed at aging people with cognitive impairment, but it is certainly not alone. Some of the latest research in user experience design, in fact, is being done by computer scientists who face the unique challenges of serving that demographic.

For example, even a task considered simple by most people—such as washing one's hands—calls for extensive consideration of what sort of assistive technology will work best for a dementia patient.

Jesse Hoey, leader of the Computational Health Informatics Lab at the University of Waterloo, in Ontario, Canada, directs a team working on an emotionally intelligent cognitive assistant to help older adults with Alzheimer's disease complete activities of daily living more independently. The project, called ACT@HOME, combines two research streams: the development of cognitive assistants with artificially intelligent controllers using partially observable Markov decision processes, and a model of the dynamics of emotion and identity called Affect Control Theory that arises from the sociological literature on culturally shared sentiments.

One of the key elements of developing assistive technology for dementia patients is an understanding of what can be a very changeable emotional state. Hoey said he gained a valuable lesson while watching a participant in one of the studies for the hand-washing assistant.

"One gentleman would get stuck, and then the reminder voice which was male and matter-of-fact imperative in tone would begin, and he would get upset and hit the mirror," Hoey said. "He had the same reaction the second day and was removed from the trial. Later on, talking to his caregivers, we discovered he was a veteran and thought the voice was like a command in an army camp; that was what would make him very upset.

"That example really triggered something for me,  and I said, 'Wow, that system may have worked perfectly for this guy if it had been a female voice, in a softer tone, maybe more suggestive than authoritative.' It made me realize that where the rubber hits the road is the moment when the prompt is placed with the person; they form a concept about what just happened to them and it is related to how they feel about themselves at that moment. So I started work on building a model of the emotional state of the person and the emotional state of the system, and how these two pieces will align in a way that the person will actually follow the prompt and feel good about it."

Using the partially observable Markov process as a building block of his model, Hoey said, allows the developer to represent the uncertainty inherent in the interaction and then layer onto that model different aspects of emotions and identity. For example, he said, "One thing you can pick up on fairly easily is energy levels; is the subject moving fast or slow? We can look for things like head position, facial expression, and if we hear him speaking we can analyze his tone of voice. All of those things basically push you into determining what mode he is in. Then, based on that, the system can adjust its prompting style to suit whatever mode he is in."

Quickly to market

Assistive technologies such as the BikeAround and ACT@HOME face fewer barriers to market entry than technology deemed as strictly medical in use: there is no need for clinical trials or approval from the U.S. Food and Drug Administration, though adequate testing is certainly something prospective users demand. Also, while both the BikeAround and ACT@HOME both feature some custom code, they also use well-tested commercially viable technologies as inherent components. Hoey said he thinks the ACT@HOME technology, including the emotionally-aware component, might be ready for market in about four years.

Camanio Care's BikeAround product manager, Jasmine Dahlberg, said more than 80 BikeArounds are now in use in Sweden, and the product is slowly making inroads elsewhere in Europe as training modules and software for caregivers are improved. The company's first BikeAround in the U.S. is being used "off label" to help rehabilitate patients of brain injury and stroke.

"I think it's really interesting," Dahlberg said. "We don't sell to rehabilitation facilities because at the moment we don't think it's developed enough to use in rehabilitation, but these facilities are requesting it. They need the gamification aspect of it. They have so many people they can't motivate."

Wendy Rogers, director of the Human Factors and Aging Laboratory at the University of Illinois, said developers of mainstream technology intended for use by older adults could take lessons from the painstaking iterations researchers such as Hoey and the BikeAround team have used in creating products for dementia patients.

"We are just starting to do some studies with older adults using the (Amazon) Echo and Alexa, and they are missing the mark on how to design for those individuals," Rogers said. "Older adults do have changes in perception and cognition and motor control we need to consider when we are designing technologies for them. We've found in our early interviews, when we ask them what they would like Alexa to do or wish it could do, the things they come up with are things it can already do. Engineers have designed it to do those tasks, but older adults don't know how to get it to do them. So it's a mismatch between capabilities and then instruction and support.

"One of the things that people who don't really study aging think is that most older adults either have or are going to have dementia. It's really a very small segment of the population. So if you are thinking of designing technology for older adults, the vast majority do not have dementia until you get into very old age, and the vast majority live independently in the community. That's a huge market opportunity."

Gregory Goth is an Oakville, CT-based writer who specializes in science and technology.