Technical Perspective: Achieve Big with Devices that Track Small Things

When we have important matters to discuss with others, we may choose phone/video calls over text/email messages. We may even prefer face-to-face meetings if it regards something very important.

What do phone/video calls and in-person meetings have that are missing in plain texts?

The subtle gestures we use, such as the nuanced signals in our tone, facial expressions, and hand gestures are essential capabilities that have been honed over millions of years of evolution. These small details shape our communications by filling the context and helping to convey our message more precisely and naturally, beyond simply passing the information along.

The power of human communication is through such subtleties. A key to success in our communications with computers lies in powerful interfaces being able to sense our subtleties. This is what I discovered about the AuraRing system as detailed in the following paper—its remarkable ability to sense user interactions at high granularities.

AuraRing tracks the 5-DoF movement of the finger while wearing a smart band that emits alternating electromagnetic fields with a resolution of 0.1mm and a dynamic accuracy of 4.4mm. The system's high-sensing accuracy, which is the precise tracking and measurement of the finger's relative movement to the wrist, allows users to perform more natural in-air gestures to interact with computers.

Interacting with computing devices with in-air gestures is not new. However, this work revealed its unique ability to sense minute movements relative to the wrist. This feature leverages the dexterity and strength of users' hands. As the authors reveal, there is a noticeable difference between considering only wrist positions and considering both wrist positions and finger movements. This likely leads to unique interactions that were not possible before. How come? One example we have seen can be drawn from touchscreens. The scrolling gestures we use on touchscreens to browse a Web page or a list of contacts are only possible when electrodes on our touchscreens are dense and sensitive enough to detect the delicate contacts of the fingers. Additionally, to exit a touch, users do not have to raise their fingers above the touchscreen surface more than a few millimeters. Imagine how awkward interaction would be without high-granularity touchscreen sensing.

The significance of this research, as I have highlighted here, illustrates finding the ideal target. Additionally, this work has demonstrated excellence from the technical aspect. Let me give some examples:

The authors found one promising technical approach to start with: Electromagnetic tracking has shown promise in tracking the human body since the magnetic field does not receive interference from the human body. They choose AC over DC tracking to enable synchronous detection, and this further enhances the signal-to-noise ratio, making it more resilient to ambient noise like EMI from artificial lighting and motors on appliances.

This paper also tackled an undesirable but often unavoidable dependency—calibration. The authors ground the calibration model with physics to pinpoint parameters to calibrate for. They found 23 parameters—14 from the coordinate system offset and nine from the analog signal chain. This effort accounts for the physics system imperfection that is inevitable due to fabrication defects in coils. The consideration of fabrication defects along with other efforts in this work such as adoptions of common wearable form factors and locations, and optimizations for short-range tracking and power improve this work's practicality.

What I discovered about the AuraRing system is its remarkable ability to sense user interactions at high granularities.

Mark Weiser, a pioneer in ubiquitous computing, painted a future where computing resources could be accessed from anywhere and anytime. In his famous quote, "The most profound technologies are those that disappear. They weave themselves into the fabric of everyday life until they are indistinguishable from it," Weiser envisioned that computing experiences will be embedded in the environment so that users can access them without even realizing it. One way to achieve this vision is to embed "smarts" into environments. Another approach, as is shown in this paper, is to embed "smarts" into the user's body. This offers several advantages due to wearable devices' proximity to the user's hands, easily "seeing" and "listening" to what the users are doing, making them the ideal gateway to interactions with computing resources. The high-granularity sensing and the practicality of AuraRing demonstrates will further lower users' interaction overhead with wearable devices—making them "disappear."

Future research in this field might track not only explicit interactions but also implicit movements from users' hands handling everyday objects for inferring user context, which is useful for making truly smart devices in a wide spectrum of ubiquitous computing applications.