If “the most profound technologies are those that disappear,” as Xerox PARC manager Mark Weiser wrote in 1991, then most human-computer interfaces are still relatively shallow. While the touchscreen-enabled pocket device has “disappeared” the computer in many contexts, a truly vanished device would never have to leave your pocket, or the hem of your clothes, or its spot embedded in your jewelry: Its interface would be your complex touch on the world’s existing surfaces.
Scientists at Disney Research in Pittsburgh have advanced this vision with their Touché sensor project, which uses a new Swept Frequency Capacitive Sensing technique to distinguish several kinds of touch on any of five types of surface. These sensor-enabled surfaces could be used to control any kind of electronic device. Perhaps Touché’s most interesting mode turns one’s own body into a controller: In a demonstration video, a user controls the music player in his pocket by touching his hands and arms to play a song, change volume, and switch tracks.
The Disney paper “Touché: Enhancing Touch Interaction on Humans, Screens, Liquids, and Everyday Objects” has sparked enthusiasm among researchers and analysts, partly because it could transcend traditional computing devices. Engineer and human-computer interaction researcher Darren Leigh believes Touché’s design is best suited for “interaction that’s much more subtle, and perhaps much more natural” than existing solutions. “Previous touch technologies say where you were touching a device,” he says. “Touché is different because it can tell how you’re touching it.”
But the project raises as many questions as it answers. It is unclear how much user training would be needed for acceptable results; radio interference caused by the frequency sweep could incite regulatory agencies to act; and the project’s paper refers several times to the need for “future work” in various areas. Further, the market is glutted with more than a dozen touch technologies including those based on resistive, optical, and other types of capacitive phenomena.
Using Capacitive Profiles
Technically, Touché is unusual in that it measures capacitance in a range of frequencies—from 1KHz to 3.5MHz, cycled 33 times a second—rather than relying on a single frequency, as is most common. This in itself is not new: Antenna builders commonly use a network analyzer in this way to optimize signal and minimize interference. But Touché goes further by taking the resulting curve, which its researchers call a “capacitive profile,” and comparing it to others using machine learning, thereby distinguishing among touches.
The results are impressive, if imperfect. Touché was able to distinguish among five types of doorknob touch—one-finger, pinch, in a circle, grasping, and absent—at nearly 97% accuracy. Dropping the most trouble-some one, the circle touch, raised accuracy to 99% with per-user training and 96% without. Similar results came from the other four modes studied, with users touching a table; a handheld device; their own bodies, as in the music-player example; and water in an aquarium.
Rather than try to predict the capacitive profile that would result from each touch, Touché’s researchers fed their experimental data curves into a machine learning system. One of the paper’s authors, Disney Research’s Senior Research Scientist Ivan Poupyrev, described their approach as “brute force.” “We just took all our data and fed it into a machine-learning system,” he says. “We did identify some of the distinctive features, such as peaks and slopes. But we didn’t try to understand the exact reasons why data looked that way or do much fine tuning.” Fortunately, their research methods yielded an enormous amount of data to work with, and they have already started secondary analysis. “We didn’t just take the curve, but also multiple derivatives,” says Poupyrev. “Now it’s time to go back and look at those curves and understand why they work this way. Then we can design specific algorithms to build something better.” Such optimization could improve “walk-up” accuracy significantly.
Jonathan Westhues, a self-employed engineer and designer of capacitive systems, points to other ways of extracting information from the project’s data. “It would be cool to see how much that additional info [Touché’s Swept Frequency Capacitive Sensing system] buys you over single-frequency sensing,” Westhues says. “You could train a classifier using just one scalar capacitance, then train a classifier using the full set of data over frequency. If you compared those, and saw that the recognizer trained with the full set of data performed better, you’d have mathematical proof that there’s extra information in there. The researchers are in an excellent position to test it, possibly using their existing data.”
Even without follow-up optimization and analysis, Poupyrev believes the current results could be good enough for some purposes. He points out that “the sweet spot for accuracy depends on the application. For example, when you’re buying something from a vending machine, you don’t want to learn how to use it. You just want the stuff, so it has to be 99.9% accurate right away. At the other extreme are home controls—your TV or home stereo and such. People have incentive to do it right, and therefore they can quickly learn the exact gestures that would get them what they want.”
Market Possibilities
The Touché paper is short on specific applications, focusing instead on general categories of interaction. But the payoff could be high for creating touch interface applications that can compete with existing ones. Jennifer Colegrove, vice president of emerging display technologies at DisplaySearch, says the market for touchscreens has been on the upswing since 2009 when she sized it at $4 billion. It jumped to $7 billion in 2010, then to $12 billion in 2011; Colegrove predicts it will exceed $20 billion in 2017.
Rather than try to predict the capacitive profile that would result from each touch, Touché’s researchers fed their experimental data curves into a machine learning system.
Although Touché would not be appropriate for all types of applications in the touchscreen space, it could capture some parts of the market, particularly those that require instrumenting nontraditional surfaces such as medical devices and tabletops.
But substantial roadblocks stand in the way. According to Colegrove, competing touch technologies are cheaper, easier to get, and backed by known manufacturers. “The current touchscreen market is about price competition,” she says, “and I don’t see Touché as a threat in that market within three years. Also, the market depends on its suppliers, and Touché is still in the early stage, with only one institute doing it. Other touch technologies have 20 or 80 suppliers. Among them, Intel is trying to grab market share with projected capacitive [technology].”
Another barrier could come in the form of government regulation. “Because Touché radiates radio frequencies, anyone who tries to commercialize it will have problems with Federal Communications Commission [FCC] certification,” says Leigh. “It sweeps from audio frequencies [starting at 1kHz] through the AM radio frequencies, and then into the shortwave bands [up to 3.5MHz]. The FCC controls everything over 9kHz. The AM band is already a very noisy part of the spectrum, and AM radios are incredibly sensitive devices.”
But if industry can work out those issues, a new world of as-yet unimagined applications could appear. So says Brandon Taylor, an engineer at Samsung Electronics who developed a “graspable” user interface that could, for example, let players vary pitch selections in a baseball video game based on how they grasp the ball. “In my work, I asked, In what types of interactions is fine manipulation really important?” he says. “If Touché produces a rich set of data about how a user’s hand is actually manipulating an object, you could start to understand some of the differences between a skilled artist in a field and a beginner. That could provide a different insight into how people learn these skills and how they’re really manipulating things. But that means going from ‘We can determine there are two fingers in water’ to ‘We can understand why someone is a virtuoso on the piano.'”
Poupyrev believes Touché could also be broadly valuable for increasing accessibility. “We don’t talk about this in the paper, but users can define their own gestures,” he says. “That’s very interesting, because we design our world assuming that people all have five fingers and can use them. But there are a lot of people with injuries or disorders who are often left out. So we usually have to design something special for them that often could be ugly, expensive, heavy, and sets them apart from everybody else. But people who are in these situations don’t want to look disabled or wear funny devices—they want the same ones as everyone else. With our technology we might make devices easily adaptable to any special needs.”
The Touché paper is short on specific applications, focusing instead on general categories of interaction, but the payoff could be high for creating touch interface applications that can compete with existing ones.
Touché is an example of research that combines existing technologies—capacitive sensing, network analysis, and machine learning—rather than introducing truly new ones. Taylor again recommended that follow-on research look beyond traditional computer applications.
“The real root of this sort of research isn’t in designing another way to right-click on a mobile phone,” says Taylor. “Instead it’s ‘If we broaden our scope of thinking, what does this allow us to do that isn’t already being done in another way?’ Maybe these sensors could figure out what you’re doing and react appropriately. Are you busy writing something? Should I route things to voicemail because you’re engaged in a behavior that I can recognize as being occupied? That’s where we can see a real breakthrough with these types of interactions.”
Further Reading
Poupyrev, I., Yeo, Z., Griffin, J.D., and Hudson, S.
Sensing human activities with resonant tuning, Proceedings of the 28th International Conference on Human Factors in Computing Systems, Atlanta, GA, April 1015, 2010.
Sato, M., Poupyrev, I., and Harrison, C.
Touché: enhancing touch interaction on humans, screens, liquids, and everyday objects, Proceedings of the 2012 ACM Annual Conference on Human Factors in Computing Systems, Austin, TX, May 510, 2012.
Taylor, B.T., and Bove, V.
Graspables: Grasp-recognition as a user interface. Proceedings of the 27th International Conference on Human Factors in Computing Systems, Boston, MA, April 49, 2009.
Touché
http://www.touche-sensor.com
Zimmerman, T.G., Smith, J.R., Paradiso, J.A., Allport, D., and Gershenfeld, N.
Applying electric field sensing to human-computer interfaces, Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Denver, CO, May 711, 1995.
Join the Discussion (0)
Become a Member or Sign In to Post a Comment