acm-header
Sign In

Communications of the ACM

Viewpoint

We Are Not Users: Gaining Control Over New Technologies


View as: Print Mobile App ACM Digital Library Full Text (PDF) In the Digital Edition Share: Send by email Share on reddit Share on StumbleUpon Share on Hacker News Share on Tweeter Share on Facebook
head of statue from three perspectives, illustration

Credit: Ded Pixto

On August 27, 2020, Amazon introduced its Amazon Halo: a technology comprised of AI software and a wristband that monitors body indicators including voice to detect problems, suggests a behavioral change, or other actions to potentially improve our health.a One day later, Elon Musk and his team presented their Neuralink technology—AI software and a skull chip implant that receives and sends signals to our brain to compensate for brain malfunctioning, aiming to solve various brain-related health problems.

These announcements seem like great news amid the health crisis that engulfs many of us, with technology coming to our rescue to confront some of the most critical diseases of humankind. Yet risks remain, and once the genie is out of the bottle, they are often difficult to manage and contain—they range from unintended consequences and side effects to threats to privacy and loss or misdirection of control.

Endless devices surrounding us include processors that compute and monitor our abundant but wasteful lifestyle, with generations of products getting faster, cheaper, and "better." We cannot envision the world today without them. Further, it seems there is no escape from this trajectory, especially with the visions of smart homes, smart cities, and the like. Even without implants or providing detailed personal data to a third party, we will be constantly watched, sampled, and analyzed.

Back to Top

Reflections on Technology

"…'modern improvements'; there is an illusion about them; there is not always a positive advance….. Our inventions are wont to be pretty toys, which distract us from serious things. They are an improved means to an unimproved end."15

Technology has been long known to have positive and negative effects on society. Technology critic Neil Postman10 made the case that technology not only changes language but also our perception of ourselves. This is the nature of technology since the days of writing and then the printing press. Gutenberg's printing press led to the revolution that eroded the central authority of the Catholic Church. Cars redefined the idea of individual freedom to move while creating suburbs, changing the identity of the collective, along with pollution and isolation that affect our health. Television, and then streaming services, created shared and then individual experiences of entertainment and information.


Are we progressively being made "dumb?"


This continuing trajectory of technology changes us from a collective with multiple perspectives to fragmented, less diverse "communities," which divide communication across perspectives. Lanier5 raised concerns about software technology developments leading to loss of personal identity and creativity by emphasizing the crowd, deterioration of financial soundness, and loss of the core of human free will to create, including a sense of spirituality. More recent work of Hwang4 emphasizes the "Subprime attention economy" run by opaque algorithms, with its ad-driven consumption increasing the risk of a subprime mortgage-like crisis. Perrow9 warned us that even the best-designed complex systems, such as nuclear or financial, will fail as unanticipated failures are embedded in them.

Thoreau's proposition15 has come to haunt us again. Both the potential risks and opportunities lie all around us. Zuboff18 tells us we are being surveilled all the time using the "pretty toys," not by the state but by capitalism, which uses our data to capture our attention all the time. While Orwell7 was concerned about central authoritarian structures controlling our lives, Huxley3 warned us there may not be any need for a central authority if we all are made into dumb citizens. Are we progressively being made "dumb"?

Back to Top

We Are Not "Dumb" Users, Are We?

Some groups have decided to tackle this head-on. Wetmore17 noted the Amish are not against technology, but they decide which technology to adopt based on certain social objectives of community and cooperation. Ostrom's8 work documented how communities manage their common-pool resources better locally than through central authority. Can we generalize from these examples to make communities control their destiny rather than rely on the ethics and social responsibility of professionals and companies while being just users?

We must ask ourselves what we cherish: What are the technologies that make our lives better, while minimizing the risks we can anticipate? We cannot, as Thoreau15 said, celebrate inventions that want to be playthings alongside inventions that matter to us as communities and individual citizens. These more useful innovations tend not to travel at warp speed, as speed is the enemy of good thinking. The way to get out of this trap is to educate each citizen to be a designer of his or her life with an understanding that each community is the place for defining what the "common good" is. Ideally, technology will develop to meet the needs of the community and that its citizens can manage, maintain, and control. It will allow time to explore and understand it while allowing to stop using and contain it.

Communities are losing the local newspapers that informed the citizens, and their leaders, about the issues of the community. They must be reconstructed and the trajectory of technology brought back to the need of the people and not the designers and their corporate sponsors. Locally sponsored community technology providers that bring the community together provide a possible solution; Townsend16 recounted the need for community-controlled Internet experiments and new civics in the creation of smart cities. But the fundamental set of design questions that arise independently of technologies and context require an answer. The answer cannot be a product—yet another app—but requires a process.

Participatory processes are around in civil society, policy, and education.11 To illustrate one implementation of participation that spread around the world, consider participatory budgeting, originally developed in Porto Alegre, Brazil, and its success has expanded to other policy areas, and inclusion of simulations as they have become more accessible.1,2,6

In order to facilitate successful participatory processes, we must create open spaces where diverse perspectives of all affected by a technology have a voice. The debate about a new technology should occur before it is developed and deployed, without those providing comments being accused of impeding innovation and progress. Designers as a profession will be needed due to their artistic and technical skills, but they will cease to control the design process; they will be part of teams of professionals and intended users working together. This would require that we make everybody conscious of what design is—a true liberal art. Every citizen should be taught to understand they can contribute to design, as they would be educated in the process of design from their daily experiences, and all involved in the process would acknowledge their insights and importance in creating useful products, policies, and services.

Beyond spaces conducive to expanded participation, the design challenges we face require addressing several fundamental questions.13,14 "What" and "Why" demand a careful articulation of the need that poses the challenge and the reasons underlying it. "Who" determines all those affected by a challenge and its solution, to be part of the process with those who have the skills and knowledge for addressing it. "How" addresses the means, organization, relationships, processes, or culture that will govern the process. One has to seek answers to all these questions that align together for the community and nationally. A complex challenge requires many skills and different perspectives, working together with a shared vision to create a well-rounded solution. They must be more deliberate and more informed. Paraphrasing Thomas Jefferson: "An educated citizenry is a vital requisite for our survival as a free people." This education should include design as a liberal art to all as that will prepare us to ask the right questions that can be asked about what we need rather than our wants imagined by others. These questions are relevant for policing methods, dealing with the pandemics, or the control of our daily rituals and habits by our new wearable device. Our challenge is to ensure new technologies support us as humans, not ceding our control to them, or letting them make us dumber.

The role and understanding of design in society require constant reflection12 and dialogues with those affected by the designs; it is the way forward to a cleaner, healthier, sustainable, safer, and saner world. We practice and teach these ideas because making them a reality requires participation and inclusion of diverse perspectives and their study and improvement. Fields of View (FOV)—a small non-profit organization—uses games and computer simulations to raise awareness of important issues through dialogue and participation in policy debates from various stakeholders including illiterate and semiliterate citizen (see https://fieldsofview.in/). The games FOV develops are played with stakeholders to address specific policy and operational questions, such as land use planning for a city, or solid waste collection and management, to expose the decision-making process and surface knowledge often unrevealed. The games are subsequently translated to computational simulations with data for exploration of the consequences of scenarios to enhance participation in decision making.


Our challenge is to ensure new technologies support us as humans, not ceding our control to them, or letting them make us dumber.


Another way to develop citizenry skilled in design for future effective participation is design education: we teach these ideas in design courses to hundreds of students from diverse disciplines such as engineering, architecture, stage design, social work, occupational therapy, and management, working on open-ended social and environmental challenges through the participation of and dialogue and reflection with problem owners. The idea is to create a modular design curriculum that will cater to all students, allowing all, especially those in non-design disciplines, to develop their design skills. These implementations of spaces for participation and the design skills to join them effectively further shape our ideas and their introduction to the general public.

Back to Top

References

1. Abers, R. et al. Porto Alegre: Participatory budgeting and the challenge of sustaining transformative change. 2018.

2. Coghlan, D. and Brydon-Miller, M. The SAGE Encyclopedia of Action Research 1–2. SAGE Publications, 2014.

3. Huxley, A. Brave New World. Harper Collins, 1994.

4. Hwang, T. Subprime Attention Crisis: Advertising and the Time Bomb at the Heart of Internet. FSG originals (2020).

5. Lanier, J. You Are Not a Gadget: A Manifesto. Alfred Knopf, 2000.

6. Mougiakou, E. et al. Participatory urban planning through online webGIS platform: Operations and tools. In Proceedings of the 13th International Conference on Theory and Practice of Electronic Governance (Sept. 2020), 831–834.

7. Orwell, G. Nineteen Eighty-Four. Secker & Warburg, 1949.

8. Ostrom, E. Understanding Institutional Diversity. Princeton University Press, 2009.

9. Perrow, C. Normal Accidents: Living With High Risk Technologies—Updated Edition. Princeton University Press, 2011.

10. Postman, N. Technopoly: Surrender of Culture to Technology. Vintage, 1993.

11. Reich, Y. et al. Varieties and issues of participation and design. Design Studies 17, 2 (1996), 165–180.

12. Reich, Y. The principle of reflexive practice. Design Science 3. 2017.

13. Reich, Y. and Subrahmanian, E. The PSI framework and theory of design. IEEE Transactions on Engineering Management. 2020.

14. Subrahmanian, E., Reich, Y., and Krishnan, S. We Are Not Users: Dialogues, Diversity, and Design. MIT Press, 2020.

15. Thoreau, H.D. Walden. Long River Press, Secaucus, NJ, 1976.

16. Townsend, A. Smart Cities: Big Data, Civic Hackers and the Quest for New Utopia. W.W. Norton, 2014.

17. Wetmore, J.M. Amish technology: Reinforcing values and building community. IEEE Technology and Society Magazine 26, 2 (2007), 10–21.

18. Zuboff, S. The Age of Surveillance Capitalism: The Fight for Human Future at the Frontier of Power. Public Affairs, 2019.

Back to Top

Authors

Yoram Reich (yoramr@tauex.tau.ac.il) is a professor at the School of Mechanical Engineering and Systems Engineering Research Initiative, Tel Aviv University, Tel Aviv, Israel.

Eswaran Subrahmanian (sub@cs.cmu.edu) is a research professor at the Engineering Research Accelerator and Engineering and Public Policy, Carnegie Mellon University, Pittsburgh, PA, USA.

Back to Top

Footnotes

a. See https://bit.ly/3gap70P

The authors thank Robin King and the anonymous reviewer for their comments that improved the exposition of the ideas presented in this Viewpoint. This Viewpoint is derived from a discussion between the authors and Communications Senior Editor Moshe Vardi, whose work spans many issues discussed here. Vardi worked to address unintended consequences of computing with ideas such as algorithmic verification, discussed the need to introduce laws and regulations to regulate technology, and recently, initiated a university-wide program on technology, culture, and society to address the negative impacts of technology. The ideas in this Viewpoint draw from our recent book, We Are Not Users: Dialogues, Diversity, and Design. MIT Press, 2020.


Copyright held by authors.
Request permission to (re)publish from the owner/author

The Digital Library is published by the Association for Computing Machinery. Copyright © 2021 ACM, Inc.


 

No entries found