Brain activity produces electrical signals detectable on the scalp, on the cortical surface, or within the brain. Brain-computer interfaces (BCIs) translate these signals into outputs that allow users to communicate without participation of peripheral nerves and muscles36 (see Figure 1). Because they do not depend on neuromuscular control, BCIs provide options for communication and control for people with devastating neuromuscular disorders (such as amyotrophic lateral sclerosis, or ALS, brainstem stroke, cerebral palsy, and spinal cord injury). The central purpose of BCI research and development is to enable these users to convey their wishes to caregivers, use word-processing programs and other software, and even control a robotic arm or neuroprosthesis. Speculation has suggested that BCIs could be useful even to people with lesser, or no, motor impairment.
It has been known since the pioneering work of Hans Berger more than 80 years ago that the brain’s electrical activity can be recorded noninvasively through electrodes on the surface of the scalp.23 Berger observed that a rhythm of about 10Hz was prominent on the posterior scalp and reactive to light. He called it the alpha rhythm. This and other observations showed the electroencephalogram (EEG) could serve as an index of the gross state of the brain. Despite Berger’s careful work many scientists were initially skeptical, with some suggesting that the EEG might represent some sort of artifact. However, subsequent work demonstrated conclusively that the EEG is indeed produced by brain activity.23
Electrodes on the surface of the scalp are at some distance from brain tissue, separated from it by the coverings of the brain, skull, subcutaneous tissue, and scalp. As a result, the signal is considerably degraded, and only the synchronized activity of large numbers of neural elements can be detected, limiting the resolution with which brain activity can be monitored. Moreover, scalp electrodes pick up activity from sources other than the brain, including environmental noise (such as 50Hz or 60Hz activity from power lines) and biological noise (such as activity from the heart, skeletal muscles, and eyes). Nevertheless, since the time of Berger, many studies have used the EEG to gain insight into brain function, with many of them using averaging to separate EEG from superimposed electrical noise.
EEG research reflects two major paradigms: evoked potentials and oscillatory features. Evoked potentials are transient waveforms, or brief perturbations in the ongoing activity, that are phase-locked to an event (such as a visual stimulus). They are typically analyzed by averaging many similar events in the time-domain. Although oscillatory features in an EEG may occur in response to specific events, they are usually not phase-locked and typically studied through spectral analysis. Historically, most EEG studies have examined phase-locked evoked potentials. Both these major paradigms have been applied in BCIs.36
The term “brain-computer interface” can be traced to Jacques Vidal of the University of California, Los Angeles who devised a BCI system in the 1970s based on visual evoked-potentials.34 His users viewed a diamond-shape red checkerboard illuminated with a xenon flash. By attending to different corners of the flashing checkerboard, they could generate right, up, left, and down commands, enabling them to move through a maze presented on a graphics terminal. An IBM360 mainframe digitized the data, and an XDS Sigma 7 computer controlled the experimental events. Users first provided data to train a stepwise linear discriminant function, then navigated the maze online in real time. Thus, Vidal34 used signal-processing techniques to realize real-time analysis of the EEG with minimal averaging. The waveforms showed by Vidal34 suggested his BCI used EEG activity in the timeframe of the N100-P200 components, with the N and P indicating negative and positive peaks, and the numbers indicating the approximate latency in msec.
Vidal’s achievement was an interesting demonstration of proof of principle. In the early 1970s, it was far from practical, given that it depended on a time-shared system with limited processing capacity. Vidal34 also included in his system online removal of ocular artifacts to prevent them from being used for control. A decade earlier, Edmond Dewan6 of the Air Force Cambridge Research Lab, Bedford MA, instructed users to explicitly use eye movements to modulate their brain waves, showing that subjects could learn to transmit Morse code messages using EEG activity associated with eye movement.
The fact that both Vidal’s and Dewan’s BCIs depended on eye movement made them somewhat less interesting from a scientific or clinical point of view, since they required actual muscle control or eye movement, simply using EEG to reflect the resulting gaze direction.
Varieties of BCI Signals
Farwell and Donchin7 reported the first use of a P300-based spelling device (see Figure 2b) in which a positive potential around 300msec after an event significant to the subject is considered a “cognitive” potential since it is generated in tasks where the subject discriminates among stimuli. Farwell’s and Donchin’s7 users viewed a 6×6 matrix of the letters of the alphabet and several other symbols, focusing attention on the desired selection, as the rows and columns of the matrix were repeatedly flashed to elicit visual evoked potentials. Farwell and Donchin7 found their users were able to spell the word “brain” through the P300 spelling device; in addition, they did an offline comparison of detection algorithms, finding the stepwise linear discriminant analysis was generally best. The fact that the P300 potential reflects attention rather than simply gaze-direction implied this BCI did not depend on muscle, or eye-movement, control, thus representing a significant advance. Several groups have since further developed this BCI method.13
Wolpaw et al.38 reported the first use of sensorimotor rhythms (SMRs) for cursor control (see Figure 2a), or EEG rhythms that change with movement or imagination of movement and are spontaneous in the sense they do not require specific stimuli to occur. People learned to vary their SMRs to move a cursor to hit one of two targets on the top or bottom edge of a video screen. Cursor movement was controlled by SMR amplitude (measured by spectral analysis). A distinctive feature of this task is that it required users to rapidly switch between two states to select a particular target. The rapid bidirectional nature of the Wolpaw et al.38 paradigm made it distinct from prior studies that produced long-term unidirectional changes in brain rhythms; for example, users were required to maintain an increase in the size of an EEG rhythm for minutes at a time. In a series of subsequent studies, this group showed that the signals controlling the cursor were actual EEG activity and that covert muscle activity did not mediate this EEG control.18,31
These initial SMR results were subsequently replicated by others21,24 and extended to multidimensional control.37 These P300 and SMR BCI studies together showed that noninvasive EEG recording of brain signals can serve as the basis for communication-and-control devices.
A number of laboratories have explored the possibility of developing BCIs using single-neuron activity detected by microelectrodes implanted in the cortex12,30 (see Figure 2c). Much of the related research has been done in non-human primates, though trials have also been done with humans.12 Other studies have shown that recordings of electrocorticographic (ECoG) activity from the surface of the brain can also provide signals for a BCI15; to date they indicate that invasive recording methods can also serve as the basis for BCIs. Meanwhile, important issues concerning their suitability for long-term human use have yet to be resolved.
Earlier studies demonstrating operant conditioning of single units in the motor cortex of primates,9 hippocampal theta rhythm of dogs,2 and sensorimotor rhythm in humans29 showed brain activity could be trained with operant techniques. However, these studies were not demonstrations of BCI systems for communication and control since they required subjects to increase brain activity for periods of many minutes, showing that brain activity could be tonically altered in a single direction through training. However, communication-and-control devices require that users be able to select from at least two distinct alternatives; that is, there must be at least one bit of information per selection. Effective communication-and-control devices require users to rapidly switch between multiple alternatives.
In addition to electrophysiological measures, researchers have also demonstrated the feasibility of magnetoencephalography (MEG),20 functional magnetic resonance imaging (fMRI),28 and near-infrared systems (fNIR).4 Current technology for recording MEG and fMRI is both expensive and bulky, making it unlikely for practical applications in the near term; fNIR is potentially cheaper and more compact. However, both fMRI and fNIR are based on changes in cerebral blood flow, an inherently slow response.11 Electrophysiological features represent the most practical signals for BCI applications today.
System Design
Communication-and-control applications are interactive processes requiring users observe the results of their effort to maintain good performance and correct mistakes. For this reason, BCIs must run in real time and provide real-time feedback to users. While many early BCI studies satisfied this requirement,24,38 later studies were often based on offline analyses of prerecorded data1; for example, the Lotte et al.16 review of studies evaluating BCI signal-classification algorithms found most used offline analyses. Indeed, the current popularity of BCI research is probably due in part to the ease offline analyses are performed on publicly available data sets. While such offline studies may help guide actual online BCI investigations, there is no guarantee that offline results will generalize to online performance. Users’ brain signals are often affected by BCI outputs that are in turn determined by the algorithm the BCI is using. It is thus not possible to predict results precisely from offline analyses that cannot account for these effects.
Blankertz et al.3 identified several trends in the results of a BCI data-classification competition. Most winning entries used linear classifiers, the most popular being Fisher’s discriminant and linear support vector machines (SVMs). The winning entries for data sets with multichannel oscillatory features often used common spatial patterns. In their review of the literature on BCI classifiers, Lotte et al.16 concluded that SVMs are particularly efficient, attributing the efficiency to their regularization property and immunity to the curse of dimensionality. They also concluded that combinations of classifiers seem efficient, noting a lack of comparison of classifiers within the same study using otherwise identical parameters.
Muller and Blankertz21 advocated a machine-learning approach to BCIs in which a statistical analysis of a calibration measurement is used to train the system. The goal is to develop a “zero-training” method providing effective performance from the first session, contrasting it with one based on training users to control specific features of brain signals.38 A system that can be used without extensive training is appealing since it requires less initial effort on the part of both the BCI user and the system operator. Operation of such a system is based on the as-yet uncertain premise that users can repeatedly and reliably maintain the specified correlations between brain signals and intent. Figure 3 outlines three different conceptualizations of where adaptation might take place to establish and maintain good BCI performance: In the first, the BCI adapts to the user; in the second, the user adapts to the BCI; and, in the third, user and system adapt to each other.
A number of BCI systems are designed to detect user performance of specific cognitive tasks. Curran et al.3 suggested that cognitive tasks (such as navigation and auditory imagery) might be more useful in driving a BCI than motor imagery. However, sensorimotor rhythm-based BCIs may provide several advantages over systems that depend on complex cognitive operations; for example, the structures involved in auditory imagery are also likely to be driven by auditory sensory input. Wolpaw and McFarland37 reported that with extended practice users report motor imagery is no longer necessary to operate a sensorimotor rhythm-based BCI. As is typical of many simple motor tasks, performance becomes automatized through extended practice. Automatized performance may be less likely to interfere with mental operations users might wish to engage in concurrent with their BCI use; for example, composing a manuscript is much easier if the writer does not need to think extensively about each individual keystroke.
As noted, EEG recording may be contaminated by non-brain activity (such as line noise and muscle activity); see Fatourechi et al.8 for a review. Activity recorded from the scalp represents the superposition of many signals, some originating in the brain, some elsewhere. These signals include potentials generated by retinal dipoles, or eye movement and blinks, and facial muscles. It is noteworthy that mental effort is often associated with changes in eye-blink rate and muscle activity.35 BCI users might generate these artifacts without being aware of what they are doing simply by making facial expressions associated with effort.
Facial muscles can generate signals with energy in the spectral bands used as features in an SMR-based BCI18 Muscle activity can also modulate SMR activity; for example, users can move their right hands in order to desynchronize the mu rhythm over the left hemisphere. This sort of mediation of the EEG through peripheral muscle movements was a concern in the early days of BCI development. As noted earlier, Dewan6 trained users to send Morse code messages using occipital alpha rhythms modulated by voluntary movements of eye muscles. For this reason, Vaughan et al.33 recorded EMG from 10 distal limb muscles, while BCI users used central mu or beta rhythms to move a cursor to targets on a video screen. EMG activity was very low in these well-trained users. Most important, the correlations between target position and EEG activity could not be accounted for through EMG activity. Similar studies have been done with BCI users moving a cursor in two dimensions,37 showing that SMR modulation does not require actual movements or muscle activity.
Applications
Several recent BCI spelling systems are based on different EEG signals, including the mu rhythm22,26 and the P300.31 The Mu rhythm systems made use of machine-learning paradigms that minimized training, with users of both mu-based systems reportedly averaging between 2.37 characters/minute22 and 2.853.38 characters/minute.26 The P300 system averaged 3.66 selections/minute.31 Townsend et al.24 noted the reported rate depends on how the figure is computed, but study authors do not always provide details. Omitting time between trials increased Townsend et al.31 results from 3.66 to 5.92 characters/second. In any case, these systems perform within a similar general range. At current BCI character rates, only users with limited options could benefi.
BCI systems have also been developed for control applications; for example, several groups have shown that human subjects can use their EEG activity to drive a simulated wheel-chair.10,14 Bell et al.1 showed the P300 could be used to select among complex commands to a partially autonomous humanoid robot; for a review of the use of BCI for robotic and prosthetic devices see McFarland and Wolpaw.19
Several commercial concerns recently produced inexpensive devices purported to measure EEG. Both Emotiv and Neurosky developed products with a limited number of electrodes that do not use conventional gel-based recording technology27 and are intended to provide input for video games. Not clear is the extent to which they use actual EEG activity, as opposed to scalp-muscle activity or other non-brain signals. Given the well-established prominence of EMG activity in activity recorded from the head, it seems likely that such signals account for much of the control these devices provide.27
Conclusion
In a review of the use of BCI technology for robotic and prosthetic devices, McFarland and Wolpaw19 concluded that the major problem facing BCI applications is how to provide fast, accurate, reliable control signals, as well as other uses of BCIs. Current BCI systems that operate using actual brain activity can provide communication-and-control options of practical value mainly for people severely limited in their motor skills and thus have few other options. Widespread use of BCI technology by individuals with little or no disability is unlikely in the short-term and would require much greater speed and accuracy than has so far been demonstrated in the scientific literature.
Sensorimotor rhythm-based BCIs may provide several advantages over systems that depend on complex cognitive operations.
Noninvasive and invasive methods would both benefit from improved recording methods. Current invasive methods do not deal adequately with the need for long-term performance stability. The brain’s complex reaction to an implant is still imperfectly understood and might impair long-term performance. Noninvasive EEG electrodes require some level of skill in the person placing them, as well as in periodic maintenance to ensure sufficiently good contact with the skin; more convenient and stable electrodes are under development. Improved methods for extracting key EEG features and translating them into device control, as well as user training, would also help improve BCI performance.
Recent developments in computer hardware provide compact portable systems that are extremely powerful. Use of digital electronics has also led to improved size and performance of EEG amplifiers. Thus it is no longer necessary to use a large time-shared mainframe, as it was with Vidal34; standard laptops easily handle the vast majority of real-time BCI protocols. Signal-processing and machine-learning algorithms have also been improved. Coupled with discovery of new EEG features for BCI use and development of new paradigms for user training, such improvements are gradually increasing the speed and reliability of BCI communication and control, developments facilitated by the BCI2000 software platform.25
BCI2000 is a general-purpose research-and-development system incorporating any brain signal, signal-processing method, output device, and operating protocol. BCI2000 consists of a general standard for creating interchangeable modules designed according to object-oriented principles (see Figure 4), including a source module for signal acquisition, signal-processing module, and user-application module. Configuration and coordination of these modules is accomplished through a fourth operator module; several source modules, signal-processing modules, and user applications have been created for the BCI2000 standard (see http://www.bci2000.org/BCI2000/Home.html).
The Wadsworth Center recently began developing a system for home use by individuals with severe motor impairments.32 Its basic hardware (see Figure 5) consists of a laptop computer with 16-channel EEG acquisition, a second screen placed in front of the user, and an electrode cap; software is provided by the BCI2000 general-purpose system.25 The initial users had late-stage ALS, with little or no voluntary movement, and found conventional assistive communication devices inadequate for their needs. The P300-based matrix speller is used for these applications due to its relatively high throughput for spelling and simplicity of use. A 49-year-old scientist with ALS has used this BCI system on a daily basis for approximately three years, finding it superior to his eye-gaze system (a letter-selection device based on eye-gaze direction) and using it from four to six hours per day for email and other communication purposes.32
How far BCI technology will go and how useful it will be depend on future research developments. However, it is apparent that BCIs can serve the basic communication needs of people whose severe motor disabilities prevent them from using conventional augmentive communications devices, all of which require muscle control.
Acknowledgments
This work was supported in part by grants from the National Institutes of Health HD30146 (NCMRR, NICHD) and EB00856 (NIBIB & NINDS) and the James S. McDonnell Foundation. We thank Chad Boulay and Peter Brunner for their comments on the manuscript.
Figures
Figure 1. Basic design and operation of a BCI system.
Figure 2. Current human BCI systems.
Figure 3. Three approaches to BCI design.
Figure 4. BCI2000 design consists of four modules: operator, source, signal processing, and application.
Figure 5. Hardware in the Wadsworth Center’s home BCI system, including 16-channel electrode cap for signal recording, solid-state amplifier, laptop, and additional monitor as user display.
Figure. BCIs are a direct communication pathway between the brain and external devices. EEG measurements at the Danish Master’s program in Medicine & Technology;
Join the Discussion (0)
Become a Member or Sign In to Post a Comment