Two critical goals for mobile devices seem intrinsically in conflict. For carrying, the smaller the better. Yet for interacting, more real estate is generally better. This tension makes for great sketch comedy: absurdly tiny phones that are impossible to interact with, or giant touch screens that are back-breaking to carry.
Chris Harrison and colleagues may have the last laugh. What if the body itself could be an input surface? The average body surface area of an adult (1.73 m2, according to Wikipedia) is 400 times greater than a touch-screen phone (0.004 m2, by my estimate). Sailors and tattoo parlors have long seen opportunities for the body as a display. Skinput adds interactivity via a pico-projector and vibration sensing: tap an image projected on your arm, and the resulting arm vibrations control an application.
How is this a harbinger of a fundamental change, and what makes its appeal more than…skin deep? One powerful contribution of the graphical interface is input on output: direct manipulation. In the coming years, pervasive direct manipulation—where Skinput is an early foray—will likely mature and become a major force. Every surface is a potential site for both projection and input, breaking the picture frame of the desktop interface. Phenomenologically, the change induced by ubiquitous projection is that the computer disappears by seamlessly weaving computing into the physical world. Skinput showcases three key tools for building disappearing computers: rich sensing, machine learning, and flexible projection. Systems like Skinput that flexibly sense body pose, movements, and gestures illustrate how interaction design benefits from innovating both software and hardware.
Does Skinput spell doom for touchscreens? Maybe not. The discourse around interactive systems often frames technical evolution in terms of "generations" of interfaces. That there were punch cards. Then the terminal. Then the mouse and graphical interface. Each supplanting the previous one. On this view, the logical question to ask is: "What’s next?" With input, this is often phrased as: "What will replace the keyboard and mouse?" Of course, different paradigms are good for different tasks. While new tools reshape the landscape and supplant some old tools, people benefit from a diverse interface ecosystem. Today, one’s computing likely spans direct manipulation, gestures, keyboard commands, and search. The screwdriver does not obviate the value of a hammer. In some cases, ubiquitous projection and sensing will enable fluid interactive experiences. In other cases, like text messaging, technologies can become powerful and pervasive even though the interface itself is quite primitive.
Isn’t an interactive forearm a little ridiculous? ("Come on! People won’t really interact this way.") Watch the video (http://research.microsoft.com/cue/skinput); it’s amazing. Also, Skinput is an early prototype in two important ways. First, it’s a sketch of a possible future: suggestive rather than complete. The viewer’s imagination is key to filling in the details. Menu selection is just one of many things this approach enables. Second, it instantiates a time-honored computer science research strategy: Build the bulky, expensive thing now to understand what it’s like to live in a world with that technology; future revisions will get smaller and cheaper. It pays to be broad when prototyping the future. Explore 10 future realities, and if any come to pass, that’s a win. Furthermore, research can succeed by inspirational value beyond its direct utility. Expanding the input repertoire will pay broad dividends.
With the forearm as the input surface, Skinput is very literally embodied interaction. Embodied interactions can offer incredible power by leveraging the amazing implicit intelligence of the human perceptuo-motor system. At the same time, bodies have clear physical limitations; you get tired holding your arm still. Unless the goal is to get into better shape, such mundane factors impose real constraints on what interfaces you’re likely to actually adopt.
One enabling insight that can’t be ignored: the tap sensing is really creative. (By which I mean, "I wish I’d thought of that.") Tapping on skin yields both transverse waves (ripples) and longitudinal waves (bone vibration). These subtle waves generally elude people’s notice, but high-frequency sensors can track them reliably. (So can high-speed cameras—another reason to watch the video.) The authors use piezoelectric sensors to measure the deformation. Today, such sensors are commonly used as guitar pick-ups. Increasingly diverse—and cheap—sensing technologies make this a really exciting time for inventing new interactive systems.
Research probes like Skinput currently require building bespoke systems. The next step is to flesh out the design space of alternatives, understand their trade-offs, and build theories. This exploration will require tools (and curricula) for rapidly and flexibly creating interfaces with rich sensing and machine learning. The DIY and research communities have made great strides here, and much exciting work remains.
Interactive tattoos?
That remains future work.
Join the Discussion (0)
Become a Member or Sign In to Post a Comment