Autonomous cars promise to give us back the time we spend in traffic, improve the flow of traffic, reduce accidents, deaths, and injuries, and make personal car travel possible for everyone regardless of their abilities or condition. But despite impressive demonstrations and technical advances, many obstacles remain on the road to fully autonomous cars.20 Overcoming the challenges to enabling autonomous cars to safely operate in highly complex driving situations may take some time.
Key Insights
- Driving a car is becoming a task shared between humans and technology, but are humans ready to just push a button and let the computers do the driving?
- Human-computer interaction issues abound when car automation systems attempt to give drivers advice or assume control of the vehicle.
- Even as some drivers are attentive behind the wheel and others lured deeper into distraction, all must be ready to take control when automation encounters corner cases.
Manufacturers already produce partially automated cars, and a spirited competition to deliver the most sophisticated ones is under way. Cars that provide high levels of automation in some circumstances (such as highway driving) have already arrived in the marketplace and promise to be in the hands of a large number of car owners in the next few years.
What does increasing automation require of drivers? The role of the driver in the extreme cases of fully manual or fully autonomous driving is clear. In manual cars, people drive, and in fully autonomous cars they do not drive. But what is the role of a driver in a partially automated car in which some of the driver’s responsibilities are replaced by computers, some of the time? Partial automation makes us part driver and part passenger, having to deal with the familiar problem of working together with computing systems. Even though totally autonomous driving will arrive someday, the transition will be difficult, especially during the period when the automation is both incomplete and imperfect, requiring the human driver to maintain oversight and sometimes intervene and take closer control.28
Here, we review two kinds of emerging car automation systems and discuss the challenges drivers will likely face when expected to work cooperatively with them behind the wheel. These automation systems range from those that offer informational assistance to drivers to those that can assume control of the vehicle for extended stretches of time—or even seize control of the vehicle when the driver wanders into unsafe situations. We draw on the state of the art in driving research, along with decades of previous work that examined the safety effects of automation as it was gradually introduced in the airline cockpit. We discuss a variety of challenges we expect to arise as automation assumes increasing responsibility for driving tasks once performed solely by humans. Some problems seem counterintuitive and some paradoxical, with few of them lending themselves to simple solutions. In the end we invite the reader to consider the evidence we present and decide whether drivers are ready to “go on autopilot” behind the wheel of the next generation of cars.
Provide Advice but Leave the Driver in Charge
The first kind of automobile automation to arrive does not seem quite like automation at all. It does not take over the controls of the car. Rather, it leaves the driver to perform the driving task while it gives advice.
Navigation systems. GPS navigation has already automated the routine but labor-intensive (and often distracting) task of figuring out how to get from point A to point B. Driver reliance on paper maps, mental representations, and out-the-window scanning for navigational clues is being replaced by a combination of moving maps, arrow indicators, verbal instructions, and head-up displays. These devices may seem simple and the advice they provide useful, but a closer examination shows they give rise to numerous problems,7 many involving safety.
Inattention. Navigation systems must be programmed, and these interactions pull drivers’ attention away from the task of driving. Early research prompted the National Highway Traffic Safety Administration (NHTSA) to issue guidelines stating any interaction with in-car information systems should require a maximum of two seconds at a time, and no more than 12 seconds total.40 However, 2013 research by Strayer et al.36 found interacting with voice-controlled navigation systems can be just as distracting as manually operated systems.
Navigation systems can also give rise to a second kind of inattention. When a navigation system performs well over extended periods, drivers may no longer feel they need to pay close attention. Indeed, many psychological studies show people have trouble focusing their attention when there is little or nothing to attend to. In such situations, they tend to reduce their active involvement and simply obey the automation. There is already ample evidence drivers disengage from the navigation task when the automation is programmed to lead the way.26 But how pervasive is this? Casner and Schooler10 found even well-trained airline pilots report engaging in copious amounts of task-unrelated thought, or “mind wandering,” when an advanced navigation system is being used and all is nominally going to plan.
Brittleness. GPS navigation shows us how automation systems can be brittle, solving most problems with ease, until they encounter a difficult, unusual case, and then do not. Consider the case of one driver in England whose navigation system commanded a turn over a cliff when the database on which it relied mistook a footpath for a road.3 Small database errors with large consequences led one early automation human factors expert to coin the phrase “blunders made easy.”39
Trust. We might ask what prompted this driver to follow such an instruction. Drivers can presumably tell the difference between a road and a cliff. It is not, alas, that easy. Studies in aviation have shown automation systems earn our trust following periods of impeccable performance, sometimes to the point we feel that the automation knows best.22 Although it is tempting to explain away accidents as isolated examples of incompetent drivers, our experience in aviation again tells us different. Well-trained pilots are prone to the same sorts of mistakes; for instance, in 1995, the crew of a Boeing 757 flew into a mountain near Buga, Colombia, after following the directions given by their erroneously programmed flight-management system.
Warning systems can lead pilots and drivers alike into trouble when they fail to alert and also when they alert too much.
Quality of feedback. Others have pointed out the amount and quality of feedback provided by systems like GPS units can make all the difference. Some GPS navigation units use their visual displays to show the vehicle is positioned in the center of the road but not where the road leads next or even if it is not a road at all. With limited information about context and surroundings, it is easy for drivers to miss important clues when things go wrong. Some have proposed designing navigation systems to more closely match the way people naturally help each other find their way in a car.17
Skill atrophy. There is good evidence that cognitive skills erode when not practiced regularly. Though we are aware of no long-term studies of navigation-skill atrophy in drivers, Casner et al.9 found significant atrophy in the navigation skills of airline pilots following extended use of a computerized navigation system.
Navigation systems are an excellent example of technology introduced to automate a task for which people already seemed reasonably competent. Yes, drivers got lost before the introduction of navigational systems, but they seldom led to safety-critical incidents. GPS navigation has introduced many human factors complications we did not anticipate.
Driver warning systems. Some kinds of information-automation systems tell us when we are doing (or are about to do) something wrong. Speed-limit alarms can alert us when we inadvertently exceed a pre-set speed limit. Lane-departure warning systems alert us when we drift from our lane or attempt to change lanes when the target lane is occupied by another vehicle. But such advisory systems are not without their limitations.
Complacency. One unintended consequence of alerts and alarm systems is some drivers may substitute the secondary task of listening for alerts and alarms for the primary task of paying attention. Wiener38 termed this effect “primary-secondary task inversion,” pointing out the problem is commonplace among experienced airline pilots. Palmer et al.29 described many cases in which pilots missed an assigned altitude when an altitude alerter (designed to advise pilots of an upcoming altitude) failed to sound. It is easy to imagine drivers allowing themselves to be distracted for prolonged periods and relying on alert systems to call when trouble pops up.
Nuisance alerts. Warning systems can lead pilots and drivers alike into trouble when they fail to alert and also when they alert too much. In aviation, alerts and alarms given in situations pilots do not find alarming cause them to ignore the alerts.5 It is easy to imagine our own reactions to a system that continuously reminds us we are driving five miles per hour over the speed limit. A second problem with alerts is they can be startling. Although human factors engineers have learned to minimize the startle effect of unexpected sound by adjusting loudness, rise time, and other characteristics, the physiological responses to unanticipated signals are difficult to avoid. Lastly, when multiple alerts sound simultaneously, the resulting cacophony can overload and confuse.13 Solutions range from trying to use different modalities for different alerts to trying to prioritize the various alerts, aiming to present only the most significant. An alternative approach would be to present a single holistic display—whether visual, auditory, or haptic or all three—that would present a single cohesive conceptual model of the situation. All these ideas are still at the research stage.
Short timeframes. The automobile is far more dangerous than an airplane in several respects. One is the rapidity with which a response is required. In an airplane flying at cruising altitude of 10 km–12 km, the pilots might have minutes in which to respond. In a car, the available time can sometimes amount to a fraction of a second. Drivers must construct an understanding of the situation, decide how to respond, and do it successfully in short order. Laboratory studies of driver reactions to rear-end-collision alerts show the effectiveness of these alerts falls off quickly when alert times are short.25,44
Summary. Although it sounds simple enough, the idea of drivers “being informed” by automated systems is not straightforward. On the one hand, systems must keep drivers informed of the driving conditions, including the state of the automobile, the road, and other cars. On the other, too much information can lead to distraction and a failure to attend to any of it.
Assume Control of the Vehicle
A second kind of automation can directly control all or part of an automobile. The arrival of such automation represents a steady progression from the totally manual cars we have today to fully automated cars tomorrow. To provide a regulatory framework for the development and deployment of automation that can operate a car’s controls, NHTSA formalized levels to describe the degree of automation. Level 0 is a totally manual car. Level 4 is a fully self-driving car that requires nothing from its occupants, or even that any occupants be present. We describe these levels, along with the human factors complications known to be associated with increasing automation of this type.
Level 0 (the manual car). The Level 0 car is entirely manual. Why discuss cars with no automation? Because the improved stability of modern cars and the smoothness of paved roads already allow drivers to take their eyes off the road and their hands off the steering wheel, giving us a preview of a first problem with vehicle-control automation.
Inattention. Level 0 cars already reduce driving to a remarkably mundane task, sometimes requiring little attention from the driver and luring the driver into distraction.
The original behind-the-wheel diversion was talking with passengers. Some studies conclude in-car conversations can interfere with driving.4,14 Yet other studies demonstrate a “two heads are better than one” effect, increasing the total amount of vigilance in a car when conversation is carefully managed.32,37 These studies reiterate that drivers and passengers have a shared understanding of the driving context and may be able to modulate their talking as the situation demands.18
Entertainment systems are a known distraction. Aside from the driver attention required to tune radio stations and select music, studies demonstrate listening to music takes a toll on driving performance.6
Personal electronics devices (such as mobile phones) provide even more distraction. Why do drivers keep talking, texting, emailing, posting, and even video calling?24 Roy and Liersch31 showed people who engage in such behaviors believe they have superior multitasking skills. Unfortunately, the evidence does not support this view. Multitasking is done by rapidly switching between tasks, not only taking attention from driving but also adding a heavy mental load in reestablishing the context of each task as it is reengaged. Interactions with devices (such as smartphones) can lure people into long excursions away from the driving task. The results can be tragic. In a 2009 smartphone-related fatality, a driver drove 84 miles per hour into the rear of a stopped car, with no brakes applied before impact.41 Revisiting the effect of having passengers in the car, a 2009 study provides evidence passengers can help limit a driver’s use of a personal electronics device behind the wheel.12
Even without the distraction of our technologies, drivers’ minds inevitably wander. He et al.21 and Yanko and Spalek42 showed the prevalence of mind wandering behind the wheel of a conventional car and its effect on driving performance. Knowing where you are going only seems to make the problem worse.43
Level 1 (function-specific automation). NHTSA’s Level 1 refers to cars that use automation to operate a single control. Many modern cars incorporate automated safety systems (such as anti-lock braking, brake assist, electronic stability control, and electronic traction control), but these systems operate only when needed and that operation is largely invisible to the driver. For example, electronic traction control brakes the wheels individually and transfers more torque to wheels with traction; this allows a driver to pull away and accelerate on slippery surfaces. Braking drive wheels individually is not something a driver could do. These automated systems operate in the control loop together with the driver, augmenting the control functions.
A cruise-control system that maintains a target speed is another example of function-specific automation. It is technologically simpler than driving safety systems but from a human factors perspective is much more complex. It requires explicit activation and deactivation by the driver and on the open road frees the driver from having to attend to vehicle speed. Cruise control automates the feedback loop that controls speed. This creates the possibility of the driver mentally drifting out of the feedback loop.35
The tiring task of doing a little less. Dufour15 showed relieving drivers of even one aspect of the driving task results in reports of increased driver drowsiness and reduced vigilance when driving on open stretches of road. But the effects do not stop there. Dufour also showed drivers take more time to respond to sudden events when they use cruise control. The message is clear. If you take drivers out of the role of active control, it is difficult to get them back in when they are needed.
One solution to the problems introduced by today’s cruise-control systems is to add more automation. Aviation human factors expert Earl Wiener termed this the “one more computer” solution. What will be the effect of adding yet another computer to address the problems of driver inattention? Our experience with automation in other domains tells us that rising levels of automation will lead to declining levels of awareness.9,16 Unfortunately, adding computers to the mix is precisely what is being done. This brings us to Level 2 automation.
Level 2 (combined function automation). NHTSA’s Level 2 refers to cars that use automation to control two or more functions of the driving task at once. A key feature of Level 2 automation is it is generally capable of fully controlling the vehicle for limited periods in restricted situations (such as following another car during uneventful freeway cruising or during traffic jams). Most automated systems at Level 2 and above assume control loops that operate without driver involvement. Two examples of Level 2 automation are the highway pilot and the traffic jam pilot systems being marketed today. They combine an adaptive cruise control system capable of adjusting the target cruise speed when a car ahead slows down or speeds up, along with an automatic lane-keeping system that maintains the vehicle within a chosen lane. The initial release of these systems gave drivers hands-free following and lane keeping for up to 10 to 15 seconds, but today’s systems can keep a car driving without attention for tens of minutes.
NHTSA Level 2 assumes the human driver will continue to closely monitor the automation as it follows the car ahead. Manufacturers differ in their requirements for drivers to keep their hands on the steering wheel. Some simply require driver hands to be near the steering wheel in the case a driver takeover is required on short notice.
Inattention (again). Level 2 automation could invite drivers to take their attention away from the driving task for longer stretches of time. If we consider the temptation of handheld devices that are already in use in manual cars today, it is not difficult to imagine where this might lead. As automation becomes more able and reliable, drivers will inevitably do things other than pay attention to driving. They may let their minds wander or even read or take a nap. Distracted drivers today periodically glance up from their handheld devices. Will they continue to glance up with the same frequency as cars provide more sophisticated automation? Driving researchers are studying these situations, and the results are not encouraging.8
More about feedback. The problem of reengaging drivers to assume active control of the vehicle is quite complex with Level 2 automation. The driver must be able to determine, at any moment, what driving functions are being handled by the automation and what functions remain the responsibility of the driver. Eye-tracking studies of airline pilots reveal they persistently misremember the state of the automation, even when they themselves set up the state. They rely on their memory of having pushed a button and habitually ignore system-status displays that tell the real story.34 Incidents in which pilots pressed a button to engage a speed-control function only to later see their speed increase or decrease unexpectedly are commonplace. Pilots sometimes erroneously assume automation functions are available for use when they are not. Automation functions sometimes quietly turn themselves off for no apparent reason. Though flight instructors try to teach pilots to assess the system state by looking at the big picture, problems still abound.
But Level 2 automation can be used in another way—offer assistance during manual driving when the driver wanders into dangerous situations. We have discussed the limitations of warning systems that provide hints to the driver about what to do next. Rather than giving advice, Level 2 automation can be used to simply take control of a vehicle in dire situations. Which is better: give advice or simply take control? Itoh and Inagaki23 compared advice-giving and takeover approaches during inadvertent lane departures and found the takeover approach resulted in greater overall safety. NHTSA27 estimated electronic stability control systems saved 1,144 lives in the U.S. in 2012 alone. These systems monitor the position of the steering wheel and the actual direction of travel of the vehicle. When the system senses a discrepancy between the two—or loss of steering control—it automatically applies differential braking to all four wheels to counter skidding conditions.
Perhaps the most compelling argument in favor of driver takeover systems comes up when we acknowledge more than half of all fatal accidents in 2009 in the U.S. happened in the presence of aggressive or angry driving.1 Imagine the life-saving potential of a system that blocks a driver’s reckless attempt to step on the gas, use the shoulder of the road, or come dangerously close to another vehicle in an attempt to pass it.
Though these examples make a solid case for “automation knows best,” we have also seen many examples in aviation in which pilots fought automation for control of an aircraft. In 1988, during an air show in Habsheim, France, in which an Airbus A320 aircraft was being demonstrated, the automation placed itself in landing configuration when the crew did a flyby of the crowd. Knowing there was no runway there, the flight crew attempted to climb. Automation and flight crew fought for control, and the autoflight system eventually flew the airplane into the trees. In this case, the flight crew knew best but its inputs were overridden by an automated system. Now imagine a case in which the GPS suggested a turn into a road that has just experienced a major catastrophe, perhaps with a large, deep hole in what would ordinarily be a perfectly flat roadway. Now imagine an automated car that forces the driver to follow the instruction. In such cases we want the person or thing that is indeed right to win the argument, but as human and machine are both sometimes fallible, these conflicts are not always easy to resolve.
How do we address the problems associated with Level 2 automation? One solution is to largely eliminate the need for attention and understanding from drivers by adding even more automation, bringing us to Level 3 automation.
Level 3 (limited self-driving automation). NHTSA’s Level 3 refers to cars that use automation to control all aspects of the driving task for extended periods. Level 3 automation does not require the driver’s constant attention, only that the automation provides drivers a comfortable transition time when human intervention is needed. When drivers are needed, the system relies on what is called “conditional driver takeover” in which drivers are summoned and asked to intervene.
Rapid onboarding. One challenge for designers is that people have great difficulty reestablishing the driving context, or as psychologists call it “rapid onboarding.” To make matters worse, automation often fails when it encounters unexpected problems, leaving the driver with only a short time to respond. Driving researchers have begun to show drivers’ onboarding times grow quickly when high levels of automation are combined with complex situations.19 Worse, studies of airline pilots responding to such unexpected events inspire little confidence.11
Manual skill atrophy. Prolonged use of automation leads to deterioration of skills. Airline pilots who use high levels of automation in an airline cockpit continually complain about it. Casner et al.9 found cognitive skills (such as navigating and troubleshooting) were quick to deteriorate in the absence of practice. Fortunately, they also found “hands on” skills are remarkably resistant to forgetting. Although this sounds encouraging, cognitive skills are needed first to determine what manual operations are required.
Tomorrow, we will have accidents that result when drivers are caught even more unaware.
Increasing complexity. Automation systems grow to be quite complex and, as a result, difficult to understand, especially by untrained drivers. Even in aviation, where pilots are well trained, automation systems are complex enough to leave them, not with definitive knowledge about how the systems work, but rather working theories that evolve over time. Pilots and researchers alike talk of “automation surprises” in which the automation does something unexpected, leaving the flight crew having to sort it out.33 The National Transportation Safety Board ruled a contributing factor in the July 6, 2013 Asiana Airlines Flight 214 crash at San Francisco International Airport was a complex user interface to the airplane’s autoflight system that was insufficiently understood and perhaps overly trusted by the flight crew.
The complexity issue is likely to grow. Modern sensor technology makes it possible for vehicles to communicate with each other and negotiate joint maneuvers involving several vehicles (such as multi-vehicle collision avoidance). Drivers will be unable to monitor these communications, in part because they occur frequently, at high speed. Almost anything a driver does in such situations is likely to degrade the automatically computed solution. This is fertile ground for what Perrow30 called “systems-level” or “normal” accidents, where accidents are not caused by the actions of an individual but emerge from the behavior of an entire system.
One of the most daunting challenges will happen when we reach the crossover point where automation systems are not yet robust and reliable enough to operate without humans standing by to take over but yet are too complex for people to comprehend and intervene in a meaningful way.
Automation that operates the controls of a vehicle could magnify the problem of maintaining driver attention, along with the consequences of lapses in driver attention. When drivers are unexpectedly asked to reassume control of the car, they are likely to struggle to get back “in the loop” to assess the situation and be able to respond in time. Some of these struggles arise from having to gather the details of the vehicle’s situation, while others arise from the complexity of the automation itself—when the details of how the automation works might elude the driver’s understanding.
Level 4 (full automation). At Level 4, the car is completely automatic. Once Level 4 has been achieved and fully accepted by the driving public, we expect cars will simply become transportation pods, without any manual controls at all except as a means of instructing the vehicle about the desired destination and giving instructions about the drive itself, much as one instructs a chauffer-driven car today. There will be no need for steering wheel or brake, though there might always be an emergency stop button. Fully automated cars will be just that. There will be no role for drivers, and no need for driving tests, age limits, and concern about sobriety or distraction.
Conclusion
A steady march toward the automated car is clearly under way. The NHTSA levels reflect the belief that automated systems will progressively assume more and more driving tasks until all are done by automation and none are left in the hands of drivers. But due to the many remaining obstacles, and the rate at which cars are replaced on the roadways worldwide, the transition to fully automated driving for a majority of the public will take decades. The safety challenges of partially automated driving will be significant and, at least as of today, underestimated. We thus draw two sets of conclusions, one for drivers, one for car designers.
Drivers. Because car automation systems will gradually increase in capability, drivers will still be required to pay full attention to the driving situation, even if they are not required to actually do anything. They may be required to take control under unannounced and unexpected circumstances, usually with little time to react. Our experience in aviation tells us this transition will not go smoothly for a cadre of cursorily trained drivers in an environment in which milliseconds might mean the difference between life and death. Drivers will expect their cars’ automation systems to function as advertised, and the systems will do so most of the time. And with automation in charge, drivers will learn they can attend more and more to non-driving activities. They will grow to trust the automation to take care of them while they do other things. They will count on automated warnings to alert them when their attention is needed. When the unexpected happens and driver attention is needed with little or no warning, a new kind of accident may emerge, in significant numbers. Today, we have accidents that result when drivers are caught unaware. Tomorrow, we will have accidents that result when drivers are caught even more unaware. We can only echo a plea that is being made to drivers today: Set personal electronic devices aside, resist any temptation to become distracted, and remain focused on the road.
To help maintain driving skill, wakefulness, or attentiveness, car interfaces might periodically ask the driver to assume manual control.
We should also look to the accident record we have today and wonder if, despite such problems, partial automation may not make a corresponding reduction in existing types of accidents. We could see dramatic safety enhancements from automated systems that share the control loop with the driver (such as brake-assist systems and lane-keeping assistance) and especially from systems that take control from the hands of aggressive, distracted, or intoxicated drivers. It is entirely possible that reductions in these categories of accidents could match or even outnumber any increase in accidents caused by other unexpected problems with automation. We expect the most serious problems to arise in systems that take the driver out of the loop, yet these are the very systems drivers want, precisely because they free the driver to do something other than drive.
Car designers. We learned in aviation that interface design indeed has a significant influence on the safety outcomes of automated systems. Drivers will need controls and displays that address the many problems we have outlined here. Driver interfaces will need to simplify and make transparent the process of passing control of the vehicle between driver and automation. The interface must further make clear the process of determining who or what is controlling the car and what the automation is doing and what it plans to do next. A particularly difficult interface challenge presents itself when a driver attempts to engage an automation function that is not ready to be engaged. We have seen too many cases in which experienced well-trained pilots pressed a button and assumed all would go according to plan, only to be surprised later. To help maintain driving skill, wakefulness, or attentiveness, car interfaces might periodically ask the driver to assume manual control.
Given the great time and expense required to design and certify a new airplane, and the often-30-year periods between airline-equipment refreshes, the aviation industry remains limited in its ability to iteratively develop and test new interface concepts. The car industry may have the luxury of being more exploratory in its design efforts and consider many more possible ways of combining human driver and car automation.
Automation in the car is here. In the coming decades, we will all participate in driving research as an enormous uncontrolled experiment takes place on our streets and highways. But with proper care and design, we can help minimize accidents caused by the presence of automation, from too much trust, lack of attention, and atrophied skills. Lisanne Bainbridge of University College London pointed out in her classic 1983 paper “Ironies of Automation,”2 “… the more advanced a control system is, so the more crucial may be the contribution of the human operator.”
Join the Discussion (0)
Become a Member or Sign In to Post a Comment