Technical Perspective: The Technologies that Disappear

More than two decades ago, Mark Weiser laid out his vision for the computer of the 21^st century: “The most profound technologies are those that disappear. They weave themselves into the fabric of everyday life until they are indistinguishable from it.”^a This vision gave rise to a vibrant research area of ubiquitous computing, where extensive research efforts are dedicated to developing sensing and computing technologies that are seamlessly integrated with ambient environments and daily objects so we are less aware of their presence. Existing research has leveraged sensors on smartphones or wearable devices, or ambient signals emitted by existing infrastructure (for example, Wi-Fi, lighting infrastructure) for inferring fine-grained human physical behaviors, physiological states, and contextual information.

Among various research efforts, a particularly appealing direction is to weave sensory technologies into textiles/fabrics. This is catalyzed by the evolution of computing technologies and advances in material sciences. Fabrics as sensors present numerous advantages. Not only are they essential materials for garments worn by us, but they are also common materials used in everyday objects and appliances. Empowering fabrics with sensing and computing capabilities truly allows us to approach the vision of ubiquitous sensing. More importantly, unlike traditional electrical sensors that are rigid, the soft and flexible nature of textiles makes them far more suitable for continuous monitoring applications that require long-time wear. It also makes the technologies accessible to population groups with sensitive skin who cannot handle rigid sensors. Additionally, the flexible nature of the fabric material offers boundless possibilities for sensor customization and optimization to suit the needs of a specific application scenario.

Active research has studied the potential of fabric sensing in a wide range of applications, spanning from human physical and physiological sensing to enabling interactive and natural interactions with everyday objects. Physical sensing typically infers physical motion based on changes of fabric properties, such as resistance, capacitance, inductance, and electrostatic states. Physiological sensing, on the other hand, uses fabrics as sensing electrodes to sense the bi-potential signals on the human body and further infer physiological states, such as electrocardiogram, electromyography, and more. While some works study fabrication of specialized textile materials to optimize sensing performance, others aim to leverage commodity, off-the-shelf fabrics to build fabric sensors, which has a broader appeal given its material accessibility. Commodity, off-the-shelf fabrics, however, are not designed for sensing. Their inherent properties can lead to sensing instability and offer limited sensitivity. As such, these inherent issues have often been addressed through hardware/sensor design innovations and/or sophisticated computational methods.

In “FabToys: Plush Toys with Large Arrays of Fabric-based Pressure Sensors to Enable Fine-Grained Interaction Detection,” Huang et al. present an interesting example of fabric-based physical sensing. It augments soft stuffed toys with fine-grained sensing ability to enable interactivity. In particular, it uses fabric’s resistance change for pressure sensing. This basic sensing rationale has been studied by prior works. However, most prior systems offer limited sensing resolution due to the use of off-the-shelf fabric and are unable to sense pressure caused by subtle motions. This paper presents a holistic framework with both hardware and software solutions to advance the sensitivity of fabric-based pressure sensing. On the hardware side, a fabric coating is considered to lower the fabric resistance by orders of magnitude and significantly boost its sensitivity. Furthermore, the split of fabric sensors into amplified and unamplified sensor streams is a clever trick, which allows differential treatment of subtle motions (for example, tickling) and large motions (for example, squeezing and swiping).

On the software side, the main challenge centers on the presence of a large array of fabric sensors, which sets a key departure of this paper from the literature. It not only entails increased computational and energy overhead to process all the sensor data but also results in crosstalk across sensors where interaction with one sensor triggers pressure applied to adjacent sensors. This paper makes important contributions in addressing these issues. The proposed use of neural networks for mitigating the crosstalk effect offers a refreshing angle. Furthermore, the paper introduces an early-exit strategy to customize the neural network model for resource-constrained computing platforms and lower the overall power consumption.

While targeting plush toys, the general system designs and proposed computational methods transcend the particular problem setting and can potentially be integrated into other fabric surfaces that humans interact with to improve the interactivity with fine-grained touch sensing and localization.