News
Computing Applications

What’s That Smell?

Digital scent technologies bring aromas to virtual reality.

Posted
user wearing a headset that incorporates OVR Technology's ion scent detector technology

The fashion designer Donna Karan once said, “Smell is the primordial sense, more powerful, more primitive, more intimately tied to our memories and emotions than any other. A scent can trigger spiritual, emotional, or physical peace and stimulate healing and wellness.”

Researchers who endeavor to create digital scents wholeheartedly share this belief and are working to make this possible in a virtual context.

“Smell is a very under-researched and a lesser-understood sense that we and many other creatures in this world share,” observes Heather Kelley, an associate professor at Carnegie Mellon University’s Entertainment Technology Center, and a game designer.

For people who like virtual reality experiences, “I think including senses other than vision and sound are important” for making those experiences richer, Kelley says.

Use cases for digital scents

In the gaming realm, a research team from Sweden’s Stockholm and Malmö universities have developed a scent machine that can be controlled by a virtual wine cellar gaming system. The player moves around the wine cellar, selects virtual wine glasses containing different types of wine, and guesses the aromas. A small scent machine is attached to the virtual reality (VR) system controller, and a scent is released when the player lifts the glass.

“The possibility to move on from a passive to a more active sense of smell in the game world paves the way for the development of completely new smell-based game mechanics based on the players’ movements and judgments,” explained Simon Niedenthal, an interaction and game researcher at Malmö University, in an article on the university’s website.

Besides gaming, digital scents can be ideal for marketing, entertainment, and training, Kelley says. In a marketing context, they could be used to evoke some sort of emotion based on a smell emanating from, for example, a restaurant or shopping mall, she says. In the entertainment realm, digital scents could be used to make movies and video games more immersive.

Virtual scents also could be used to train someone who works in a factory or in the medical or military fields to detect the smell of certain chemicals or, Kelley says, “If you’re trying to train someone to do something in real life, having the smell might help acclimate them.”

One method for creating digital scents is to use a VR olfactory apparatus that is comprised of an odorant saturation chamber with an inlet and an outlet, according to inventors at Northwestern University. Part of the odorant saturation chamber includes beads and a liquid that has an odorant concentrate.

The inlet extends into the concentrated part of the odorant saturation chamber. The apparatus also includes a mass flow controller that generates airflow to the inlet. When the airflow passes through that portion of the odorant saturation chamber, it forms an odorant. The airflow passes the odorant through the outlet of the chamber. There is also a nose chamber on the VR apparatus that is connected to the outlet and configured to receive the odorant, the inventors wrote in a 2018 patent application describing the device.

Promoting better health and well-being

Judith Amores, a senior researcher at Microsoft Research and a research affiliate at the Massachusetts Institute of Technology’s MIT Media Lab, is focused on digital scents for people’s well-being. Digital scents are “not only good for memories but our emotions, compared to other senses,” she says.

Amores wrote her dissertation on olfactory interfaces; specifically, on their use in wearable devices. She did a pilot study at MIT comparing the brain wave activity of 12 people wearing a virtual reality headset and an olfactory necklace she developed that released a burst of scent, to the brain wave activity of people who did not wear the necklace.

She found that relaxation increased by 26% among those wearing the headset and necklace, and the physiological response also increased by 25%. The physiological response was measured using an electrocephalogram (EEG) signal, and relaxation scores were computed from EEG frequency bands associated with a relaxed mental state using an entropy-based signal processing approach, Amores wrote in an IEEE paper titled “Promoting relaxing using virtual reality, olfactory interfaces and wearable EEG” that discussed the study. She also noted that this was the first VR therapy system using scent in a wearable device, which “proves its effectiveness to increase relaxation in everyday life situations.”

The challenges of digitizing scents

Amores has been conducting a study at Massachusetts General Hospital for the last year or so ago on patients undergoing cardiothoracic surgery, in which she is attempting to help reduce their anxiety both before and after surgery. Study participants are given a VR headset that monitors their brain activity, and a necklace that releases a customized burst of smell from an atomizer.

However, there have been challenges and a lot of logistics to deal with, Amores says. For example, devices get lost because patients often start off in one room prior to surgery and are taken to a different room post-surgery.

A survey also revealed that people don’t like to use wearable devices overnight, she adds. As an alternative, the atomizer can be attached to a gooseneck fixture (something usually curved like the neck of a goose, or U-shaped), close to the nose. Another issue is that many patients share rooms and some have allergies, so the scent has to be very localized. In that instance, people prefer to have an olfactory device attached to their glasses, Amores says.

There are many ways to deliver digital scents, depending on the application. In Amores’ case, she likes using an atomizer, given her interest in applications related to health, and “ideally, you want to keep compounds as natural as possible.” The atomizer is basically a disc that vibrates at a high frequency and when it comes in contact with a textile, it releases a small droplet, similar to a humidifier.

In the future, Amores sees the potential for embedding such an atomizer in pajamas and other clothing.

Yet another challenge is the difficulty of replicating the enormous discriminatory capacity of natural olfactory systems, observes Vanessa Ruta, a professor at Rockefeller University, who does not work on generating virtual scents, but is interested in the challenge of odor detection via e-noses of the olfactory system. She explains, “The diversity of chemicals is enormous, and their physicochemical properties are not directly related to any perceptual quality in the same way that, for example, the wavelength of light relates to our perception of color,” she says.

Consequently, while there are three photoreceptors in the eyes that are sufficient to sense the entire spectrum of visible light, Ruta says, “large repertoires of olfactory receptors appear to be necessary to detect and discriminate amongst the diversity of chemicals in the environment.”

Tying virtual olfaction to memory

Jas Brooks, a Ph.D. student at the University of Chicago, is conducting historical research on digital scent technology, specifically with regard to the trigeminal nerve, which runs through the face and responds to chemicals, touch, and temperature.

“For me personally, the important aspect of digital scent is its connection to memory and enjoyment,” Brooks says. They add that digital scent “has the potential of maybe being able to change how we experience memories of each other.”

In a virtual reality context, smell is “very grounding,” because in real life, people may not think much about smells, which have to compete with visuals and sounds, they say. “But when an odor hits you, it grabs all your attention and it’s so engaging, and that quality can be brought to virtual reality.”

Brooks is working on reproducing scents using electrical stimulation. “I’m curious about how you digitize smell on the output side,” they said.

Like a printer, you need a cartridge of basic smells that can be combined, and finding those basic smells is an open challenge in the field. “One of the things I’ve been exploring is, can you actually digitize the stimulation side without chemicals and produce the sense digitally,” Brooks says. They are interested in reproducing a limited palate of digital scents–things like the sharpness of vinegar, the bubbliness of champagne, and the refreshing scent of peppermint.

“You have the technology getting better and … developing very rapidly, and at the same time, a more fundamental understanding of how smell works, and how we interact with odors, whether it’s odors in real life, or how we use smell to potentially ignite our memory,” they said.

Other work being done in this area includes a wireless, skin-interfaced olfactory feedback system that was recently unveiled by a team of researchers at City University of Hong Kong (CityU), which releases different smells using miniaturized odor generators. The idea is to integrate the odors into VR and augmented reality systems to provide users with a more immersive experience. This can be applied to 4D movie watching (the ability to access the 3D dimension without any spatial restrictions), medical treatment, and online teaching.

In terms of the most popular digital scents, while Amores says it depends on demographics and culture, her own research has shown that vanilla and lavender are popular with most people. “There seems to be something intrinsic in human nature-—probably related to olfactory receptors—on what we perceive as pleasant,” she says.

Echoing Brooks, Amores says that “Olfaction is so good for memory.” Creating digital scents can be useful for people suffering from dementi, a and those who lost their sense of smell as a result of being infected with COVID-19. “Smells are very connected to memory; you remember particular smells from your past.”

We don’t usually consider the full potential of the sense of smell, and there is so much that can be done with digital scents, Amores says.

Creating smells in virtual reality “is socially interesting,” she adds. “It grounds either the environment you’re in or the objects you’re interacting with.”

    • Wireless olfactory interface brings scent to VR experiences, Global Times, May 15, 2023, https://www.globaltimes.cn/page/202305/1290728.shtml
    • Panagiotakopoulos, D., Marentakis, G., Metzitakos, R., Deliyannis, I., and Dedes, F. Digital Scent Technology: Toward the Internet of Senses and the Metaverse, IT Professional, Volume 24 Issue 3, May-June 2022 pp 52–59, https://bit.ly/3PbfJe8
    • Ko, LW., Su, CH., Yang, MH., et al. A pilot study on essential oil aroma stimulation for enhancing slow-wave EEG in sleeping brain, 2021. Sci Rep 11, 1078. 10.1038/s41598-020-80171-x
    • Flavian, C., Ibanez-Sanchez, S., and Orus, C. The influence of scent on virtual reality experiences: The role of aroma-content congruence, ScienceDirect, Journal of Business Research, Vol 123, Feb. 2021. https://www.sciencedirect.com/science/article/pii/S0148296320306093#f0010
    • Amores, JF., Nirmita, M., Rasch, B., and Maes, P. Olfactory Wearables for Mobile Targeted Memory Reactivation, ACM Digital Library, April 2023, 10.1145/3544548.3580892
    • Fernandez, J. A. Olfactory interfaces: toward implicit human-computer interaction across the consciousness continuum, 2020. DSpace@MIT, April 2020. https://dspace.mit.edu/handle/1721.1/129318
    • Wang, Y., Amores, J., and Maes, P. On-Face Olfactory Interfaces, 2020. ACM Digital Library, 10.1145/3313831.3376737
    • Brooks, J., Nagels, S., and Lopes, P. Trigeminal-based Temperature Illusions. 2020. University of Chicago. https://lab.plopes.org/published/2020-CHI-trigeminal.pdf
    • Smelling in VR environment possible with new gaming technology, Stockholm University News, October 12, 2022, https://bit.ly/3PbZuxk
    • Wireless olfactory feedback system to let users smell the VR world, Science Daily, June 6, 2023, https://www.sciencedaily.com/releases/2023/06/230606111647.htm

Join the Discussion (0)

Become a Member or Sign In to Post a Comment

The Latest from CACM

Shape the Future of Computing

ACM encourages its members to take a direct hand in shaping the future of the association. There are more ways than ever to get involved.

Get Involved

Communications of the ACM (CACM) is now a fully Open Access publication.

By opening CACM to the world, we hope to increase engagement among the broader computer science community and encourage non-members to discover the rich resources ACM has to offer.

Learn More
News
Architecture and Hardware

What’s That Smell?

Posted
A special chip can mimic the brain’s olfactory senses.
The new system, which runs artificial intelligence software on Intel's Loihi neuromorphic chip, is essentially an "electronic nose" that can learn the scent of a chemical from a single exposure.

Researchers at Cornell University and Intel have developed artificial intelligence (AI) software that can learn the scent of a chemical with just one exposure, and then remember that scent forever.

The software, which is designed to run most efficiently on an experimental chip from Intel known as Loihi, is so precise, it can even detect a scent that's masked by a number of other scents, according to researchers.

Ultimately, the researchers hope to produce a market-ready solution that can detect hazardous substances in the air, sniff out dangerous drugs, discover hidden explosives, and assist with medical diagnoses.

"Low-energy modules built around Loihi, running our algorithm, and hooked-up to diverse sensor arrays could be built into robots, medical analysis devices; for example,  blood composition, hyperspectral processors, air quality sensors, food processing pipelines, you name it," says Thomas A. Cleland, a member of the research team and associate chair and professor of psychology at Cornell University.

The system works by processing an input signal pattern for a scent drawn from an array of sensors, then recording that signal pattern in the AI software as a recognizable scent for future use.

The beauty of the system is that a scent detected by the software only needs to be 'learned' once, and the system will always be able to identify that pattern, according to Cleland.

Another advantage of the software is that all learning regarding a scent is 'local.' Essentially, when the software discerns a scent, only a few cyber-synapses (the AI equivalent of human synapses) are encoded. "All learning is local to particular synapses and derived only from things that happen at that synapse," Cleland says.

That approach differs markedly from conventional deep learning, which would require the software be exposed to hundreds or even thousands of samples of the same scent before it could discern a common pattern it could use as a reference for that scent.

The system's AI also differs from traditional deep learning in that the software can add new scents to its repertoire without losing the memory of a previous scent. This technique also offers a marked advantage over conventional deep learning systems, which generally need to 'forget' a previously learned scent when prompted to learn a new scent, according to Cleland.

Under the hood, the system relies on data generated by 72 chemosensors, which send a fingerprint signal for each of 10 scents (including ammonia and carbon) to the system's software. Once detected by the AI, the fingerprint signal is processed, recorded, and stored, enabling the system to instantly recognize that same scent the next time it is sensed.

Processing is greatly aided by Intel's experimental Loihi neuromorphic chip system, a network of 64 chips offering the power of 8 million neurons, which Cleland describes as "ridiculously energy-conservative."

Tests showed that even when a sought-after scent was overpowered by a number of other scentswhat Cleland calls 'noise'the software was still able to ferret-out the scent sought.

"This classification under noise is really our killer app," Cleland says. "The visual analogy is that you can't see something if there is an obstruction in the way, but maybe you can recognize your friend just from a corner of his head and one ear poking out from around the side of this visual obstruction, because you know what your friend's head looks like, even if most of his head is not able to be sensed directly because of that obstruction."

Cleland says development of the AI software was initiated at his lab, with the help of Nabil Imam (while Imam was a Ph.D. student at Cornell), along with help from current Cornell Ph.D. student Ayon Borthakur. "Nabil got hired by Intel a few years after his Ph.D. defense, after some time at IBM, and joined the Intel team that developed that new Loihi chip," Cleland recalls, "so we ported the algorithm to the Loihi chip for its debut publication."

It was a win-win collaboration, according to Cleland: "Extra buzz for the algorithm, a cool application for the Loihi chip; everybody is happy."

Even so, the system needs additional work before it can be commercialized, including studying how the AI software performs 'in the wild' under completely unpredictable conditions.

Says Santiago Marco, group leader of the Artificial Olfaction Lab at Spain's Institute for Bioengineering of Catalonia, who specializes in signal and data processing for sensing systems, "I would like to see how this research can be extended to do the odor classification tasks in real time and with a larger diversity of conditions."

Timothée Masquelier, a researcher specializing in bio-inspired AI at the French National Centre for Scientific Research, agrees on the need for further research. "The proof of concept is convincing, but the problem they tackle is somewhat simple: identification of a previously learned odor, despite noise.

"Most real-world problems will involve recognition of broader odor categories; for example, flowers, food, animals, etc. It remains to be proven if their system can handle this sort of categorization."

Adds Brice Bathellier, an experimental and theoretical neuroscientist at the French National Centre for Scientific Research, "It is not ready to be a real flexible nose, simply because it has been tested with too few odors, in very controlled conditions. But it seems to work to identify some odors among variable mixtures. Overall, I think this is a nice algorithm for the problem of odor recognition with this type of chemical chip."

Enhancing scent detection and performing experiments in the wild will hinge on linking the system to a greater number of sensors, Cleland says. "We really need collaborators with larger sensor arrays to hook up to our" network to move forward with research.

Fortunately, a wide variety of sensors will work for further development. "A great advantage of this algorithm, as inspired by the history of work in the artificial olfaction community, is that it doesn't need very specialized sensors to be developed to detect this or that particular signal or compound," Cleland says. The system works "on any sensor array that can be processed as a list, not just a chemosensor array."

"On the hardware side, we're looking to Intel," Cleland says.  "We can run this on the Pi (computer). But the Loihi chip is intended for exactly this sort of purpose."

Meanwhile, Imam, now a senior research scientist at Intel, also is looking to repurpose the research other types of analysis. "My next step is to generalize this approach to a wider range of problems, from sensory scene analysisunderstanding the relationships between objects you observeto abstract problems like planning and decision-making.

"Understanding how the brain's neural circuits solve these complex computational problems will provide important clues for designing efficient and robust machine intelligence."

Joe Dysart is an Internet speaker and business consultant based in Manhattan, NY, USA.

Join the Discussion (0)

Become a Member or Sign In to Post a Comment

The Latest from CACM

Shape the Future of Computing

ACM encourages its members to take a direct hand in shaping the future of the association. There are more ways than ever to get involved.

Get Involved

Communications of the ACM (CACM) is now a fully Open Access publication.

By opening CACM to the world, we hope to increase engagement among the broader computer science community and encourage non-members to discover the rich resources ACM has to offer.

Learn More