The field of human-computer interaction (HCI) came into being more than 25 years ago with the mission of understanding the relationship between humans and computers, often with an eye toward improving the technology’s design. but that relationship has since been altered so radically—changes in the sociotechnical landscape have been so great—that many in the community of HCI researchers and practitioners are questioning where the field is headed. Computer systems now intrude on our lives as well as disappear into the world around us, they monitor as well as guide us, and they coerce as well as aid us. Thus there are debates about such fundamentals as what HCI’s goals should be, how it should do its work, and whether its methods remain relevant.
In March 2007, academic and industrial researchers from many different countries and diverse backgrounds, including computing, social science, and design, met in Seville, Spain, for a two-day workshop entitled “HCI in 2020.” The event, sponsored by Microsoft Research Cambridge, U.K., was a chance to air views, reflect, and discuss the future of HCI as well as issues of central importance to the field. Needless to say, participants expressed a wide range of opinions, but they were virtually unanimous that the field of HCI must change its scope and methods if it is to remain relevant in the 21st century.
While the researchers agreed as well on the need to keep human values at HCI’s core, they highlighted the fact that our changing relationship with computers means that determining what these values might be and coming to understand them require greater finesse than ever before. If in the past HCI was in the business of understanding how people could become more efficient through the use of computers, the challenge confronting the field now is to deal with issues that are much more complex and subtle. Here we summarize these issues, basing our discussion on the workshop’s report Being Human: Human-Computer Interaction in the Year 2020.1
A Brief Look Back
When the field of HCI was in its infancy, a common activity was to model a user’s interaction with a desktop computer so that the interface between person and machine could be optimized. HCI was mainly a scientific and engineering endeavor, using techniques derived from cognitive psychology and human-factors engineering.8 What went on “inside the head” of a user was specified by observing behavior under controlled conditions, inferring what kinds of perceptual, cognitive, and motor processes were involved, and developing pertinent theories.2 Methods for optimizing “usability” were devised, and iterative testing with real users was seen as prerequisite to introducing any new software or hardware product.
During the 1990s, the objectives of HCI began changing along with the growth of communication networks that link computers. Researchers started asking how users, with the aid of computers, might interact with each other.13 Researchers with backgrounds in more socially oriented sciences, such as sociology and anthropology, began to engage with HCI. These disciplines emphasized not only the effects of computing on groups of users but also how those very same groups appropriated computers, interpreted them, and socially and emotionally experienced their relationships with the technology. Several of the approaches of these disciplines were added to the mix with ethnographic approaches being especially visible.
Values are not something that can be catalogued like books in a library but are bound to each other in complex weaves that, when tugged in one place, pull values elsewhere out of shape.
The practical result of these developments is that HCI has become an academic discipline in its own right, with conferences dedicated to the subject as well as departments and courses offering HCI as a speciality, and it has also become an integral part of the design processes—typically, user-centered—for nearly all technology companies.14 Moreover, an understanding of HCI (if not its details or techniques) has seeped into the broader consciousness, as the common use of terms such as “user-friendliness” and “user experience” in the news media and everyday conversation attest. Such awareness, among practitioners and users alike, has encompassed computers not only in the conventional sense of, say, desktop systems but also as they are manifested in cars, airplanes, mobile phones, and a broad array of other products.
In parallel, important changes in research objectives have also taken place within the field. The HCI of today is exploring diverse new areas beyond the workplace, including the role of technology in home life and education and even delving into such diverse areas as play, spirituality, and sexuality. HCI is now more multidisciplinary than ever, with a significant percentage of the community coming from the design world. This shift has caused the field’s practitioners to think more broadly about their design goals, taking into account not just how technology might be functional or useful but also how it might provoke, engage, disturb, or delight.
Transformations in Interaction
Despite the progress, gradual but now very visible transformations in our relationship to computers are leading many in HCI—including participants in the Seville workshop—to urge a radical rethinking of the underpinnings of HCI: its mission, goals, and philosophical approach, both for research and practice. In essence, the claim is that the interaction between values and technology needs to be much more carefully navigated than before. This is not a simple choice between designing for what is desirable as opposed to what is reprehensible; HCI specialists also need to be astutely aware of how one set of design choices might highlight certain values at the expense of others. In other words, values are not something that can be catalogued like books in a library but are bound to each other in complex weaves that, when tugged in one place, pull values elsewhere out of shape. Further, now more than ever, the diversity, scope, and complexity of the technologies that HCI deals with make tradeoffs between values a conundrum, not a platitude.
The reasons for this new complexity can be attributed in large part to the major transformations that have redefined our relationship with technology. Here we characterize five such transformations, each of which continues to alter the ways in which humans coexist with computers, interact with them, decide what problems to focus on, and pursue solutions.
The first transformation—the end of interface stability—has to do with how computers can no longer be defined by reference to a single interface but rather by many different interfaces or, alternatively, none at all. For example, some computers encroach ever more deeply into our own personal spaces: we carry them, wear them, and may even have them implanted within us. Other forms of computers are disappearing into the richness and complexity of the world around us. They are increasingly embedded in everyday objects; not just toys, home appliances, and cars but also books, clothing, and furniture. And they are increasingly part of our environments, in public spaces such as airports, garages, and shopping malls as well as in the private spaces of homes and offices. In each case, where the interface might be, or even if there is an interface at all, is an open question. All of this has consequences for HCI. After all, the assumption that the locus of human-machine interaction is obvious (and hence can be observed, researched, and designed for) has been at the core of HCI since its foundation. If this is no longer the case, then what an interface might be, where it is, what it allows a user to do, and even whether there is one at all are now the issues that a future-looking HCI must address.
A second transformation, the growth of techno-dependency, refers to the fact that changes in how we live with and use technology have resulted in our becoming ever more reliant on it. There is of course no news in saying that society and individuals alike depend on a technological infrastructure. But what is different about this transformation is that computational dependence is more complex, fraught with more snag points, and vulnerable to more forms of attack. It is not simply that we are increasingly using computers in routine but selected activities, such as to write reports or do our tax returns. Computing now underpins almost every aspect of our lives, from shopping to travel, from work to medicine. At the same time, computers are becoming ever more sophisticated and autonomous. As a result, not only is our reliance on them growing but computers themselves are increasingly reliant on each other. The extent of our need for computers—characterized by a wide diversity of technologies, an “always-on” infrastructure, and an interconnected web of systems—creates new concerns, new design opportunities, and new research topics that specialists in HCI are obliged to address.
A third transformation is the growth in hyperconnectivity, the influential role of communication technologies in tying us together in ways that were unimaginable even as recently as 10 years ago. Despite the ability of such new tools to improve efficiency and save us time, such “digital presence” increasingly consumes our time rather than saves it. Communication devices are now filling our lives up instead of releasing us from burden. Yet hyperconnectivity also has the power to mobilize us, as citizens and members of global communities; we are now in touch in more ways, and with more people, than ever. What these changes mean, how one designs for them, and how one judges value within the myriad forms of being in touch are all substantive issues for HCI.
Fourth, our heightened ability to be in touch is equalled by a passion to capture more and more information about people’s lives and actions—information that hitherto would have been discarded or forgotten. This trend is reflecting as well as driving the massive gains in computer networks’ capacity. What it means to record, why we record, and what we do with the collected material is changing hand-in-hand with the systems we use to capture, manage, share, and archive these burgeoning stores of personal data. Each of us is developing an ever-increasing “digital footprint”—sometimes in ways we desire, sometimes not, and often in ways we know little about—not only on a personal level but also within the databases of government agencies and other public, as well as private, institutions. We call this transformation the end of the ephemeral.
Finally, the proliferation of new kinds of digital tools (exemplified by Web 2.0) and their appropriation by people from all walks of life are enabling us to work, play, and express ourselves in new ways. Computers were once limited to the automation and mechanization of routine aspects of work or problem-solving. Now, more than ever, they are also instruments for creativity. This trend is manifested not only in the explosion of computer tools for play and self-expression; it also propels more “serious” pursuits. For example, computational tools are enabling advances in the world of science and medicine as they assist researchers in discerning, analyzing, and solving problems. This fifth transformation—the growth of creative engagement—underscores the fact that flexible computer tools, which can be assembled and appropriated in new ways, allows us to see the world in wholly new ways too. Computer-enabled creativity means we can all become our own producers, programmers, and publishers, whether in our personal or professional lives, with potentially far-reaching consequences.
New Questions for a Future-Looking HCI
The five transformations are provoking questions that HCI has not had to address before, as they concern issues that simply did not arise in a world where using a computer essentially meant a person sitting in front of a desktop machine doing email, writing a document, or working on a spreadsheet. Because our relationship with computing is now far more extensive and complex, these new questions deal with how we design for the emerging interaction paradigms.
For example, the end of interface stability raises questions such as:
- What interaction techniques are appropriate if devices embedded within us have no explicit or recognizable “interface?”
- Should new interaction techniques build on the skills we have already acquired for dealing with far less complicated systems? And if so, how?
- How do we enable people to understand the complexity of new ecosystems of technologies, and the results of interacting with them, so as to proceed most effectively?
Our growing dependency on computing provokes a different set of questions, including:
- How do we design computer systems to help people cope when infrastructures break down or when devices malfunction or are lost?
- What will be the taken-for-granted technologies of the future and how might they alter the skill sets of the people for whom we must design?
- With computers becoming increasingly autonomous, seemingly able to make their own decisions, what will be an appropriate style of human-computer interaction?
The end of the ephemeral leads us to consider what is being recorded, stored, and analyzed regarding our beliefs, preferences, and everyday actions—and interactions. Questions include:
- What computer technologies are needed to effectively manage vast quantities of personal data?
- How do people learn about their digital footprint as well as the tools that can help them interrogate the systems involved and analyze the data?
- How do we design computer systems so as to give people feedback about, and control over, information-capturing processes?
- How can the capture of information and the need for privacy be balanced through design?
Taken together, these and other transformation-related questions point to a very different kind of agenda, for researchers, practitioners, and technology designers alike, from the one that was appropriate for HCI in the 1980s and 1990s.
But in addition to new questions about interaction and design, many of the issues these transformations raise are much more far-reaching. They include how society should react to the changes that computer systems engender—how their impact will be dealt with in different situations, places, and cultures—and a range of moral concerns. The sidebar here—”Questions of Broader Impact”—posits some of these changes, followed by examples of the new kinds of ethical questions they raise.
Human Values in the Face of Change
Should the HCI community be addressing these more far-reaching kinds of questions? And if so, is it equipped to take on the task? The participants at the Seville workshop agreed that it should—and also that a quite different mind-set is required.1
To begin with, researchers and practitioners in HCI need to analyze the wider set of issues that are now in play. Central to the new agenda is recognizing what it means to be human in a digital future. Human values, in all their diversity, should be charted in relation to how they are supported, augmented, or constrained by technological developments. In many ways, this is arguing for a strengthening of what has always been important to HCI: a focus on human-centered design, keeping firmly in mind what users—people—need and want from technology. The trouble is that the values that systems often impinge on are not the kind that can be easily inventoried. For instance, values related to technologies that capture our digital footprint may support our recollection of the past and influence ideas of selfhood just as much as they might imply more measurable ideals related to bureaucratic efficiency (for example, keeping good records). Computational technology affects both, though the audit of one is considerably more difficult than that of the other.
Making judgments about new computer technologies, and how they will affect us and the social fabric of which we are a part, is not straightforward. Research methods must capture how the use of technologies may unfold over time and in different situations.
It follows that the field of HCI needs to extend its approach in order to encompass the often complex and diverse patterns of human interests and aspirations. This means that the methods of HCI, and the disciplines it engages with, will have to change.
Important steps have already been taken in this direction—in the concept of “use,” for example. A growing number of researchers and practitioners have begun explicating the nature of use as a question of “experience” and how it unfolds over time. This has largely involved the definition of subjective qualities. Analysts have used concepts like pleasure, aesthetics, fun, and flow, on the one hand, and boredom, annoyance, and intrusiveness, on the other, to describe the multifaceted nature of “felt” experiences.10 In addition, HCI specialists such as Norman11 have modeled how we respond to technology at a visceral or emotional level as well as at a deliberate and reflective one. They have also described a more comprehensive life cycle of our response to technology, from when it first grabs our attention and entices us, through our ongoing relationship with that technology, and finally to when it is eclipsed by other technologies and we abandon it. These ways of conceptualizing users’ experience have opened up many new possibilities for research and design.
An emphasis on the individual and the phenomenology of his or her experiences is a natural consequence of HCI’s traditional starting point: the user. But it should be obvious that as HCI moves forward and seeks to address the changes cited previously, the user, however well understood, is only part of a larger system—or set of systems. Much effort also needs to be expended on determining what is desirable within a place, an institution, or a society. Values such as personal privacy, health, ownership, fair play, and security are obvious candidates for analysis, but so too are public, institutional, and civic identities. The values treasured by the individual are not always in harmony with those of institutions or the society; nor, on the other hand, are they always inimical to one another. Here specialists in HCI can learn a great deal from disciplines, such as sociology and anthropology, that focus on organizations and cultures. The bottom line is that the field of HCI needs to take into account the broader context within which human values are expressed.
Some HCI researchers are indeed beginning to emphasize human values as central to research and design,3,5,6,13 while others have been attempting to define a “third paradigm”9 that draws on ideas of embodiment4 such as, taking into account the interactions and conversations that happen in our physical and social worlds that provide meaning. These alternative approaches stress that a deep understanding of our interactions with technology cannot be divorced from their contexts. The meaning of technology is created within Specific situations, and not just by individuals but often by many stakeholders.
Yet making judgments about new computer technologies, and how they will affect us and the social fabric of which we are a part, is not straightforward. Research methods must capture how the use of technologies may unfold over time and in different situations. Consider that computers can help connect us to others, but by the same token it is important that they sometimes allow us to be isolated. Likewise, computers can support our industriousness but at other times we may want to “switch off.”
Moreover, such choices are not always ours alone to make; it is not simply users and their own particular aspirations that are involved. For example, workplaces reserve the right to summon their staff to be industrious. In other words, sometimes communications are meant to be heard even when the audience does not especially want to listen. As Peters notes in Speaking into the Air,12 communications can be about communion as well as about information exchange. So design tradeoffs need to be considered not just in terms of our local interaction with a technology but also in terms of weighing the various moral, personal, and institutional consequences.
In a world where people’s movements and transactions can be tracked—where individuals trigger nondeliberate events just by being in a certain place, physical or virtual, at a certain time—the notion of interaction itself is being fundamentally altered.
A New Approach for HCI
We propose, then, that a broader approach is needed for tackling the new kinds of questions that the transformations are raising. But what are the practical implications of such an avenue? What does it mean for the field of HCI?
Folding human values into the research and design cycle. Our first suggestion, described more fully in the Seville workshop’s Being Human report, is to extend the ways in which user-centered research and design are conducted by explicitly addressing human values.
A simplified but helpful model of current practice is that projects typically follow an iterative cycle, comprised of four fundamental stages, in which HCI specialists sequentially study, design, build, and evaluate technology with users. The goal, for example, may be to design a particular computing technology in order to improve upon a given experience. Initial research involves finding out about people’s current practices, for which ethnographic studies, logging of user interactions, and surveys are commonly employed. Based on the information gathered, the specialists begin to focus on the why, what, and how of designing something better. To aid in the process, usability and user-experience goals are identified and conceptual models developed. Prototypes are built, evaluated, and iterated on until it is determined whether the new technology can meet the user goals and whether the new user experience is judged by the target group to be valuable and enjoyable.
The Being Human report proposes that a new agenda for HCI should enhance this model by adding another stage—an initial stage, called understand—which aims to pinpoint the human values that the technology in question will be designed to serve. Depending on the values of interest, this analysis might need to draw on disciplines as diverse as philosophy, psychology, art, sociology, cultural studies, and architecture, for example. It might also mean collaborating with the stakeholders behind the technology to ascertain what kinds of enduring values they expect their users to derive from the product.
Consider, for example, that there might be an interest in developing new interactive tabletop applications for working with digital photos. The understand stage of the work would involve clarifying what kinds of human values might be made possible through such interactions. Is it about supporting social connectivity around photographs? About play and creativity with digital images? About archiving photographs and other materials in order to preserve and honor family history? Or is it about allowing individuals to reflect on their personal past through images? The list could go on.
Ultimately, this stage is about making basic choices. It requires specifying up front the kinds of users targeted, and in which domains of activity, environments, or cultures. In other words, the stage involves choosing the values being designed for. Its investigations will then point to some fundamental research that needs to be conducted, relevant research that has already been carried out, or some combination of the two. The stage may equally well involve experts from diverse disciplines, such as social historians, game designers, or specialists in the psychology of memory, to cite but a few.
Further, the extended approach to HCI is intended to enable human values to be folded into the mix not just at the understand stage but the other four stages as well. In the report, we give fuller examples of how choices made about the human values of interest can provide guidance in the study, design, build, and evaluate phases. Key here is that the analysis should not just take into account people’s interactions with computer technology but also with the environment, with everyday objects, with other human beings, and with the changing landscape that the “new tech” brings to their world.
Forming new partnerships. Aside from changes in methodology, HCI also needs to develop partnerships with other disciplines that traditionally have not been part of the field. One reason has been outlined here—that different human values, as expressed in diverse contexts, point to the need for all kinds of expertise to deeply understand and creatively design for the relationships between those values and technology.
But other reasons have to do with questions that are even more difficult for the field of HCI alone to address. As we have outlined, new computer technologies and the transformations they are bringing about raise issues with much broader societal, moral, and ethical implications than HCI has had to deal with in the past. It is not clear that all of these concerns are within the scope of the field, but certainly HCI needs to be part of a wider interdisciplinary exchange. Technologies that store personal data, that take on new roles and responsibilities in our lives, that alter our behavior in public places, and that track our movements and activities are as much sociological as architectural and as much about politics as cognitive reasoning. Given the scope and complexity of these issues, HCI professionals need to engage in discourses that may at one time have seemed distant, if not entirely alien to them.
Redefining the H, C, and I. It is with these concerns in mind that the report suggests Redefining the three elements of HCI—human, computer, and interaction.
The “H,” representing the “user,” clearly needs revision, especially given that people nowadays are as much consumers, creators, and producers as they are users of computers, and they often employ computers just for the fun of it. Conceptualizing the emotional aspects of experiencing technologies is already starting to happen. Words like magic, enchantment, pleasure, wonder, excitement, and surprise have begun to creep into the vocabulary when researchers and designers discuss the value of technology to people. But HCI specialists also need to ask what these terms really mean and how technologies may engender such experiences. The aesthetics of computational products has also gained importance in helping to define users’ relationships to technology. Therefore new models would provide a better understanding of how the emotional aspects of computing relate to human values.
A new conception of the “C” in HCI is also needed so that we may better understand how the embedding of digital technologies in everyday objects, in the built structures around us, and in the natural landscape is transforming our surrounding environment into a physical-digital ecosystem. Thus we need to address not just the design of artifacts per se but also the spaces within which they reside. And the design has to deal with deeper and more systemic issues. As the computer becomes increasingly reliant on a larger world, and in particular as the connection to a network becomes an essential part of the computer’s operation, the opportunity for improving the user experience simply through a better interface is rapidly disappearing. HCI needs concepts, frameworks, and methods that will enable it to consider people and computers as part of a messy world full of social, physical, technological, and physiological limitations and opportunities.
It follows that the “I” in HCI will also need to be understood at many different levels. As Greenfield7 has so elegantly described, we will have to consider different sites of interaction—for example, interactions on and in the body, interactions between bodies, interactions between bodies and objects (properties such as graspable, pushable, and other human-centered descriptors may be important here), and interactions at the scale of kiosks, rooms, buildings, streets, and other public spaces. All these levels of interaction offer different physical and social “affordances”—readily perceivable action possibilities—that technologies can potentially change.
In redefining H, C, and I, and in extending what the field of HCI may achieve, we will need to develop a lingua franca that expresses not only new metaphors but also new principles. Such a common language will enable the diverse parties to better understand each other, to talk in detail about the emergent transformations, and to productively explore how to steer them in human directions.
In a world where people’s movements and transactions can be tracked—where individuals trigger non-deliberate events just by being in a certain place, physical or virtual, at a certain time—the notion of interaction itself is being fundamentally altered. As the conception of technology use as a conscious act becomes difficult to sustain, other models of interaction and communication will have to be developed. At the other extreme, digital technologies will continue to be used in more deliberate and engaged ways as media for self-expression, community-building, identity-construction, self-presentation, and interpersonal relations. HCI professionals must understand the complexity of the new forms of social relations and interactions if they are to help develop technology that enables people’s effective engagement.
The fact that we now live with technology and not just use it means that HCI must also take into account the truly human element, conceptualizing “users” as embodied individuals who have desires and concerns and who function within a social, economic, and political ecology. HCI must also be flexible, given that people’s forms of engagement with technology and the nature of their interactions with it will continually be changing, often becoming more sophisticated, as they grow older. Understanding the new forms of interaction between humans and computers will involve asking questions about the qualitative—process, potential, and change—rather than quantifiable attributes and capabilities alone.
Figures
Figure. The complexity of technologies that HCI now encounters can be attributed to the major transformations that have redefined our relationship with technology. This article explores five such transformations, also reflected in this image. Can you find them?
Figure. The “interface” between humans and computers is harder than ever to define. We can interact with computers just by walking through a public space.
Figure. The growth in hyperconnectivity carries with it both the benefits and the pressures of being connected “anywhere, anytime.”
Figure. The “History Tablecloth,” developed by the Interaction Research Studio (Goldsmith’s College, University of London), is an example of embedding computing in everyday objects. When items are left on the cloth it begins to glow beneath them, creating a slowly expanding halo. When the items are removed, the glow gradually fades.
Figure. The latest billboards (such as those by Quividi) judge the gender and approximate age of people viewing them, with the potential of changing the nature of the advertisements they display. Technologies like these highlight the increasingly hybrid forms that interaction takes, as well as the scope of the “data” used to authenticate such interactions.
Join the Discussion (0)
Become a Member or Sign In to Post a Comment