Computing and Assistive Technology Solutions for the Visually Impaired

The idea of “reinventing the wheel” is very often looked down upon in research. But many devices and solutions in the assistive technology (AT) space have been available for nearly half a century and still have not reached most users in low-income countries. Two such examples are refreshable Braille displays, which make digital data accessible in Braille through touch rather than audio, and tactile diagrams, which are critical to helping visually impaired people to pursue subjects, such as science, where diagrams are crucial to understanding the concepts. While accessibility normally refers only to the modality for making information accessible, in the Indian context, it is tightly tied to affordability.

No market exists in the AT space in low-income countries, though the need is very high, because the user’s ability to pay is either low or non-existent. Thus, there has never been an incentive for technology hubs—which are traditionally located in wealthier countries—to pursue alternative designs or innovate to develop affordable solutions. At the same time, a major advantage with new devices and solutions based on electronics and software is that they scale very well—that is, higher volumes translate to lower costs. This feature, therefore, gives us hope that affordable AT solutions will soon reach users in lower-income countries.

Assistech, a laboratory founded circa 2007 at IIT Delhi, is unique in that it focuses on the user rather than on any specific technology. Assistech’s research and development (R&D) activities address challenges related to safe mobility and educating visually impaired people in low-income countries. Located in the Amar Nath and Shashi Khosla School of Information Technology, the laboratory comprises a highly interdisciplinary group of researchers and students. In addition to computer scientists and electrical engineers, the team has always had a strong presence of mechanical engineers, designers, and even visually impaired users. In this article, we describe some of the Assistech lab products that have either reached the users or are ready for production and scaling. We also underline the need for user-focused research in this domain.

Ultrasonic ranging device for safe mobility. SmartCane³ (shown in Figure 1), one of the most successful products to emerge from the laboratory, is used by more than 100,000 people. SmartCane is an obstacle detection device that uses ultrasonic sensing to alert users via vibrations. It can detect obstacles within 3m, while someone using a traditional white cane can detect objects through contact within 1m and only up to knee level. Users needed a device capable of above-knee obstacle detection as well as non-contact detection. These are both safety issues, especially in public spaces in countries such as India. Objects that do not have a footprint but overhang in a walking path, such as tree branches on streets/footpaths or air conditioners jutting out of windows in corridors, are examples of obstacles that impede safe mobility. Protruding objects above knee level can cause severe upper-body injuries, and such repeated injuries can result in loss of confidence in independent mobility. Further, it is also not uncommon to find stray animals such as dogs and cows sharing street space with people. Like inanimate objects, if these stray animals can only be detected through contact (as with the traditional cane), they will pose significant safety hazards. As solutions to such challenges are created and designed, it is critical to involve actual users along with determining initial functional requirements.

Figure. A SmartCane instructor training a user.

Figure 1. SmartCane, an affordable obstacle-detection system mounted on a white cane.

We based SmartCane’s initial design focus on our expertise in hardware and software design, including the use of low-power techniques to increase battery life. Even when the prototypes were being tested, we faced two key challenges related to the handle design. The device was conceived as an add-on handle for existing white canes. Women with smaller hands found the handle circumference in the prototypes too large while users who were already comfortable with a given grip found the device difficult to adapt to as they needed to change their grips. Eventually, the handle was designed to support multiple grips; that and the resultant packaging of electronics were SmartCane’s most innovative components. This was achieved only because of user involvement and the team’s interdisciplinary nature.

RF transponder device for boarding public buses. OnBoard² (shown in Figure 2) is a system that facilitates the independent boarding of public buses. It addresses universal challenges faced by visually impaired people to find the route number and locate the entry door when a bus arrives at a stop. For a variety of reasons, in high-density urban centers in countries such as India, buses do not always stop in the bay to pick up passengers. Our initial studies showed that passengers had to walk up to 20m if 95% of the passengers boarding had to be covered. This resulted in a globally unique solution using two major components: a RF device the size of a small appliance remote control that stays with the user and a controller with an attached speaker that is installed on the bus and faces out toward the bus stop. The user device sends out a query and collects and announces the route numbers of all buses in the vicinity using a simple, slotted network protocol employing two channels. When the user selects the desired route number, it is announced through the speaker, providing the user with an auditory cue for locating the entry door.

Figure 2. The OnBoard bus identification system.

Clearly, user needs were unique and were addressed by the device; at the same time, the other stakeholder—the bus operator—also provided exclusive insights. After we successfully conducted trials on 25 buses operating on two routes in Mumbai, an official from Brihanmumbai Electric Supply & Transport (BEST) responsible for bus transportation pointed out a limitation of OnBoard, as well as an interesting use case outside the scope of our project which we had not thought of. On narrow roads, buses travelling in both directions are within the system’s range. During the trials, sometimes by mistake, users triggered the system in the unwanted direction. This confused the driver, as no visually impaired passenger entered the bus despite the bus waiting for the rider to board. This led to changing the bus antenna design so that it now only picks up signals from one direction. The BEST official also pointed out that although the solution was designed for visually impaired users, it could also be very effective for wheelchair users, pregnant women, and older people with limited mobility—anyone, in fact, with special needs who requires the bus to stop for a longer period so they can board. The same audio output that announces the route number alerts the bus driver that someone with special needs wants to board the bus.

---

Computing clearly has a huge potential in assistive technology solutions because of its cost-effective scalability.

---

Affordable, refreshable Braille display. Braille cells made from PZT-based actuators have been commercially available for many decades. However, they were completely unaffordable and had no penetration in low-income countries. Typically, a connected device with a QWERTY keyboard would cost upward of USD 80 per cell while a simple reader may cost upward of USD 40 per cell. With DotBook⁴ (depicted in Figure 3), a refreshable Braille display device with laptop-type functionality, we put most of the effort into developing reliable and affordable proprietary Braille cells using a shape-memory alloy.

Figure 3. DotBook is a refreshable Braille display device with laptop-type functionality.

Tactile graphics for science, technology, engineering, and mathematics education. Diagrams, charts, graphs, and the like are critical to understanding concepts in science, technology, engineering, and mathematics (STEM). Tactile diagrams have been integral to textbooks for the visually impaired in high-income countries. But with typical production costs upward of USD 2 per page, visually impaired students in low-income countries had little or no access to tactile diagrams. Visually impaired students in high school and beyond could only opt for certain subjects, such as history, languages, and so on, where books were mainly text oriented. By creating a software flow and adopting 3D printing to create molds for tactile diagrams, Assisitech solutions cost at least five times less than comparable solutions. The know-how developed in the laboratory has been scaled up through the Raised Lines Foundation (https://raisedlines.org/), a non-profit incubated at IIT Delhi. We continue to explore the availability of low-cost materials to produce tactile diagrams even after scaling up production and serving thousands of visually impaired students across the country.

Back to Top

Work in Progress

Assistech is engaged in three major R&D projects, again in the space of education and mobility. Mobility Assistant for Visually Impaired (MAVI)¹ uses the latest machine learning (ML)-based object detection and classification algorithms to support the independent mobility and outdoor navigation of visually impaired people. Detection of stray animals with high accuracy and detection and text recognition of multilingual signs and billboards (often handpainted) are again unique challenges found in India’s public spaces.

Reading Assistant for Visually Impaired (RAVI)⁵ uses a variety of techniques to make PDF documents accessible to the screen reader. The challenge here is to make equations, tables, and diagrams accessible while identifying and removing unnecessary elements such as headers, footers, watermark, and so on, which render documents incomprehensible when accessed through a screen reader. The approach combines rule-based and ML-based object-detection techniques which, from our observations, work very well for detecting equations and tables.

IncluNav is a project aimed at providing independent mobility to visually impaired people in indoor spaces. Consisting primarily of a smartphone app with support from Bluetooth beacons installed in the building, IncluNav performs localization and supports navigation. It is an end-to-end solution and includes a simple mechanism for creating indoor maps. This is especially relevant, as one cannot assume systematic preservation of original building drawings. Also, substantial renovations may have occurred. We believe that with the rather weak signage in most public buildings, the app would find wider acceptability even among the sighted, making it a commercially viable product.

Computing clearly has a huge potential in AT solutions because of its very cost-effective scalability when affordability remains a key challenge. On the other hand, Assistech’s experience emphasizes the importance of both interdisciplinary research groups as well as strong user connections for success in defining and building AT solutions. It is also clear that all developmental costs up to the stage of making production-ready AT products and solutions must be either publicly financed or available through charitable funding. Only with such unencumbered funding can one break the cost barriers necessary to reach AT users in low-income countries.

Acknowledgments. Thanks to PVM Rao (co-founder of Assistech), Dipendra Manocha, Rohan Paul, Piyush Chanana, Chetan Arora, Volker Sorge, Pulkit Sapra, Vikas Upadhyay, and numerous other students and research staff for the success Assistech has achieved in making a difference in the lives of the visually impaired in India.