Brain-computer interfacing using modulations of alpha activity induced by covert shifts of attention
© Treder et al; licensee BioMed Central Ltd. 2011
Received: 25 January 2011
Accepted: 5 May 2011
Published: 5 May 2011
Visual brain-computer interfaces (BCIs) often yield high performance only when targets are fixated with the eyes. Furthermore, many paradigms use intense visual stimulation, which can be irritating especially in long BCI sessions. However, BCIs can more directly directly tap the neural processes underlying visual attention. Covert shifts of visual attention induce changes in oscillatory alpha activity in posterior cortex, even in the absence of visual stimulation. The aim was to investigate whether different pairs of directions of attention shifts can be reliably differentiated based on the electroencephalogram. To this end, healthy participants (N = 8) had to strictly fixate a central dot and covertly shift visual attention to one out of six cued directions.
Covert attention shifts induced a prolonged alpha synchronization over posterior electrode sites (PO and O electrodes). Spectral changes had specific topographies so that different pairs of directions could be differentiated. There was substantial variation across participants with respect to the direction pairs that could be reliably classified. Mean accuracy for the best-classifiable pair amounted to 74.6%. Furthermore, an alpha power index obtained during a relaxation measurement showed to be predictive of peak BCI performance (r = .66).
Results confirm posterior alpha power modulations as a viable input modality for gaze-independent EEG-based BCIs. The pair of directions yielding optimal performance varies across participants. Consequently, participants with low control for standard directions such as left-right might resort to other pairs of directions including top and bottom. Additionally, a simple alpha index was shown to predict prospective BCI performance.
A brain-computer interface (BCI) serves to decode user intention from brain signals, enabling a direct communication between brain and computer. Since the main target group of BCIs is patients with motor impairments, it is vital that the control of a BCI does not involve motor activity. However, this is not always the case. For instance, for the widely used Matrix speller (a.k.a. P300-speller), evidence accumulates that BCI control is efficient only when the target symbol is fixated with the eyes [1–3]. Different routes have been taken to circumvent the problem of gaze dependence. For instance, one may fall back on other sensory modalities such as spatial auditory [4, 5] and tactile feedback . Alternatively, one may rely on other paradigms such as motor imagery [7, 8]. However, motor imagery paradigms face the problem that a subset of participants does not obtain significant BCI control, a problem that is only partially solved [9–11]. Also in the visual domain, there have been promising approaches to gaze-independent BCIs. For instance, recently, three visual gaze-independent spellers have been introduced . In contrast to the Matrix speller, the selection process was broken down into two successive steps, and for the best speller, mean symbol selection accuracy amounted to about 97%. Liu et al.  combined a similar visual design with a visual search task and reported a peak performance of 96.3%. In another study, rapid serial visual presentation of symbols was used, with a mean symbol selection accuracy of up to 90% for selecting one symbol out of thirty .
Note, however, that these paradigms rely on visual stimulation. In particular, they exploit the fact that the event-related potential (ERP) associated with a visual stimulus can be modulated by attention. In the present study, we take a more fundamental approach. It has been shown that covert spatial attention shifts are accompanied by power changes in the alpha band (8-12 Hz) of the electroencephalogram (EEG) at posterior electrode sites . Therefore, rather than measuring the effects of attention on the neural response to visual stimulation, we directly tap the neural process underlying covert attention shifts. This approach has several advantages over conventional paradigms based on ERPs. First, continuous visual stimulation, which can be tedious and irritating especially in long BCI sessions, is superfluous. Second, for some application domains such as spatial navigation, it seems more intuitive to shift attention to the desired location rather than to perform a task such as counting the occurrences of a flashing target. Third, a BCI based on changes in oscillatory alpha activity potentially allows for asynchronous control. That is, the user initiates a covert attention shift whenever he or she wants to issue a command, whereas in an ERP paradigm, the user has to adhere to the pace and timing of the visual stimulation sequence.
Results and discussion
Overall response accuracy was 86.62% ± 8.46% SEM. The accuracies in the valid and invalid condition were compared using a paired-samples t-test and found to be not significantly different (p = .199). In contrast, the geometric means of the reaction times were significantly smaller in the valid condition than in the invalid one (t = 4.49, p < .01), indicating that the participants attended correctly the cued positions (valid: 719 ms ± 51 ms SEM; invalid: 881 ms ± 76 ms SEM).
We repeated the analysis on the subset of trials wherein the target latency was 2000 ms, since only this subset was used for neurophysiological analysis and classification (see next paragraph). For this subset, overall response accuracy amounted to 87.2% ± 8.6%. The accuracies in the valid and invalid condition were not significantly different (p = .233). The geometric means of the reaction times were significantly smaller in the valid condition than in the invalid one (t = 3.92, p < .01; valid: 742 ms ± 55 ms; invalid: 896 ms ± 84 ms).
For each participant, and for each of the fifteen possible pairs of directions, we performed binary classification using logistic regression and computed classification accuracy under a ten-fold cross-validation scheme [26, 27]. In order to reduce sensitivity to overfitting, an L2 regularizer was added to the classifier's objective function . This regularizer is controlled by a regularization parameter that effectively shrinks the estimated regression coefficients towards zero. In order to determine the optimal regularization parameter a grid search was performed and the smallest parameter value was chosen that gave highest accuracy as computed with five-fold cross-validation using just the training data of the outer ten-fold cross-validation. Subsequently, the classifier was retrained using all training data in order to test the classifier on the test data. Significance levels were calculated by comparing classification outcomes with an assignment of all outcomes to the majority class using a McNemar test . For comparative purposes, classification was repeated using L1 regularization, but it was found to yield lower classification accuracy than L2 regularization.
Since alpha power peaks over occipital electrodes sites, the subset of electrodes comprising PO3,4,7-10, and Oz,1,2, was selected as input to the classifier. We focused only on alpha synchronization, because the preceding alpha desynchronization did not show distinctive patterns for the different directions. For each electrode, a single spectral feature was extracted by estimating bandpower in the alpha range (8-12 Hz) for the 500-2000 ms interval using the Welch method. In other words, the interval was split into 8 segments with 50% overlap between segments. Each segment was windowed using a Hamming window. Spectral power was estimated in each segment and then averaged across segments. During cross validation, for each participant, data was normalized to have zero mean and a standard deviation of one in the training set of the outer fold.
Left hemisphere versus right hemisphere contribution
As a consequence, one would expect an asymmetric impact of electrode position on BCI performance, with left hemisphere electrodes contributing more to classification success than right hemisphere electrodes. As Figure 5b suggests, this is indeed the case. For most participants, classification on left hemisphere electrodes yields better scores than classification on right hemisphere electrodes. Nevertheless, taking into account both hemispheres usually improves performance, suggesting that right hemisphere electrodes add independent information. To compare these three conditions quantitatively, we performed a 1-way analysis of variance (ANOVA) on the peak performances in the three conditions. We found a significant effect of the electrode subset (left, right, or both hemispheres) on BCI performance (F = 6.11, p < .01).
Tukey-Kramer post-hoc tests revealed that classification using both hemispheres gives better accuracy than classification using left hemisphere only. The other contrasts were not significant.
Alpha rhythm based predictor of BCI performance
In light of the availability of numerous BCI systems and the fact some users do not obtain significant BCI control, prediction of BCI performance using simple neurophysiological indices is a topic that is gaining increasing attention . Our aim was to use posterior alpha power from the resting EEG as a predictor of BCI performance. To this end, we investigated the relaxation data recorded prior to each experiment. We considered the epochs wherein participants relaxed with eyes closed. After current source density filtering , the spectral peak in the 8-12 Hz alpha range was extracted for each electrode.
Shifts of covert visual attention induce changes in alpha power over posterior electrode sites. Initial analyses revealed that an early desynchronization was of little discriminative value regarding the direction of attention shifts. We believe that this early desynchronization may be related to the preparation of covert attention shifts. A subsequent synchronization, however, yielded distinctive topographic patterns for the different directions and served as a basis for classification.
Using regularized logistic regression, significant binary classification performance was obtained for each participant, with a mean accuracy of 73.65% for the best pair of directions. A classification accuracy of 70% was proposed as performance threshold above which BCI performance can be considered as robust [32, 33]. In the present study, six participants had a peak performance above 70%, and two participants had a performance that was slightly lower (66% and 69%). Interestingly, this figure is close to the accuracy obtained in earlier MEG studies, in spite of the significantly higher spatial resolution of MEG as compared to EEG [18, 21]. This suggests that changes in alpha power following covert attention shifts are rather broadly distributed in visual cortex and, hence, can be mapped with sufficient precision using EEG.
Mean classification accuracy obtained in the present study is similar to the accuracy obtained by Kelly et al. . However, there are significant methodological differences. First, Kelly et al. used visual stimulation in form of two flickering stimuli. It is unclear how the flickering affects the ease of deploying attention to the visual periphery. Second, just as we did, Kelly et al. used cross-validation to estimate classification performance. However, epochs were partly overlapping. In other words, the training set (used to train the classifier) partly contained information about the test set (used to verify the classifier), which might have led to an overestimation of classification accuracy.
The pair of directions yielding the highest classification performance varies considerably across participants (see grey double arrows in Figure 3). For all but one participant, locations at opposite sides of the fixation point yield optimal performance. Furthermore, for seven participants, highest performance is achieved with a combination of left and right directions. Mostly, this combination also has a vertical offset (i.e., top-left combined with bottom-right, or bottom-left with top-right). For the other participant (iah), peak performance is achieved when attention is shifted in the vertical direction. This indicates that left versus right is not necessarily the optimal pair of directions. Therefore, participants with low control for these directions may resort to other pairs of directions including top and bottom.
Furthermore, we found an asymmetry regarding the contribution of electrode sites to classification success. In particular, left hemisphere electrodes contributed more to classification success than right hemisphere electrodes. This is in line with evidence that the left hemisphere supports mainly attention shifts to the right hemifield, while the right hemisphere is involved in attention shifts to both the right and the left hemifields .
Prediction of BCI performance
Due to the proliferation of BCI research in the last decade, there exists now a wide palette of BCI systems. However, there is no a priori criterion for assigning a particular BCI system or a particular input modality (such as event-related potentials or sensorimotor rhythm) to a new BCI user, despite the fact that there is high variability across users regarding the efficiency of particular BCI paradigms. As a result, BCI users might use a system that does not yield optimal performance. This problem is aggravated by the fact that a non-negligible proportion of participants fails to exhibit significant BCI control. For paradigms based on the modulation of the sensorimotor rhythm (SMR), this proportion amounts to 15-30% of the participant population .
Consequently, there is growing need for efficient screening procedures that allow for the estimation of prospective BCI performance. To be useful, screenings should be obtained within few minutes using a simple paradigm, in order to prevent a tedious and, upon failure, frustrating calibration procedure. For instance, Blankertz et al. showed that the mu rhythm generated in motor cortex is predictive of BCI performance in a motor imagery paradigm . The predictor was obtained from a 2 minutes measurement during which participants were instructed to relax with eyes open. It showed a correlation of r = .53 with BCI performance.
In a similar fashion, we developed a predictor of BCI performance based on a 3 minutes relaxation measurement with eyes closed. For each participant, the invidiual alpha peak was extracted and power was combined for electrodes PO3 and PO4. A correlation of r = .66 was found between the alpha index and peak BCI performance, suggesting that BCI performance can be predicted from a simple resting EEG measurement.
The present study suggests that modulations of alpha power associated with covert attention shifts form a viable input modality for EEG-based BCIs. Furthermore, an alpha index obtained during a short relaxation measurement can predict prospective BCI performance. Analogous to the motor imagery paradigm, where different types of imagery (e.g., movement of left hand, right hand, and foot) are tested preliminary and the best pair is chosen, eligible participants might then be screened for different directions of covert attention shifts. In order to maximize performance, the BCI would be tuned to the pair of directions that provides the best classification accuracy.
Eight healthy volunteers (seven male, one female), aged 18-27 years, participated in this study. One of the participants was a co-author (NS), all others were naïve with respect to BCIs. All had normal or corrected-to-normal vision. All participants gave written consent and the study was performed in accordance with the Declaration of Helsinki.
Task and Stimuli
The main experiment was preceded by a six minutes relaxation measurement. It comprised two alternating phases, namely an eyes closed phase, wherein participants simply relaxed and closed their eyes, and an eyes open phase, wherein they observed a small polygon on the computer screen changing shape and color. The duration of each phase was 15 s with 2 s breaks in between, and the total measurement lasted for about 6 minutes.
In the main experiment, participants performed a cued visual attention task. The course of a trial is depicted in Figure 1. First, a white central fixation dot surrounded by six white target discs was presented. The discs had a size of 3.27° of visual angle and they were presented at an eccentricity of 9° from the fixation dot. A cue appearing for 200 ms in the center of the screen indicated the target location. Participants had to shift attention to the cued disc while strictly fixating the central dot. Instead of arrows, we used an omnidirectional cue to reduce the danger of evoking event-related potentials specific to the direction of the cue. The cue was a hexagon with each of the six faces pointing to one of the target discs. Three of the faces were grey and the other three were colored blue, red, and green, respectively. One of these colors was used as target indicator, that is, the participant had to covertly direct and maintain attention to the disc to which this color was pointing. The use of one of the three colors as target color was counterbalanced across participants. After a variable duration (500-2000 ms) the target appeared for 200 ms in the disc as either a '+' or a '×'. Participants indicated which symbol they had perceived by pressing with their thumb on one of two buttons lying in the palm of the right and left hands. Two different targets had been chosen to reduce readiness potentials for pressing a button, as suggested by . After 200 ms, a star-shaped masker (' *') was presented at the target location for 200 ms in order to prevent an afterimage of the target and thereby increase task difficulty.
Each participant completed 600 trials in six blocks of 100 trials with two-minute breaks between blocks. Cues were valid in 80% of the trials. In the other 20% of the cases, the target appeared at a different random location. The target symbol was randomly chosen, with equal chances for '+' and '×'. Target latency (i.e., the time between cue onset and target onset) was 2000 ms in 50% of the trials. To ensure that the participants shift their attention immediately after the appearance of the cue, 30% of the trials featured a short target latency of 500 ms. In the remaining trials, the target latency was randomized between 500 ms and 2000 ms in order to ensure that attention is sustained continuously until target appearance.
EEG was recorded from a Brain Products (Munich, Germany) 64 channel actiCAP, digitized at a sample rate of 1000 Hz, with impedances kept below 20 kΩ. We used electrodes Fp2, AF3,4, Fz, F1-10, FCz, FC1-6, T7,8, Cz, C1-6, TP7,8, CPz, CP1-6, Pz, P1-10, POz, PO3,4,7-10, Oz,1,2 and Iz,1,2, placed according to the international 10-10 system and referenced against a nose reference. Additionally, an EOG electrode labelled EOGvu was placed below the right eye. Vertical and horizontal bipolar EOG channels were created by referencing Fp2 against EOGvu, and F10 against F9, respectively. Stimuli were presented on a 24" TFT screen with a refresh rate of 60 Hz and a resolution of 1920 × 1200 px2. The experiment was implemented in Python using the open-source BCI framework Pyff  with Pygame http://pygame.org. Data analysis and classification were performed with MATLAB (The MathWorks, Natick, MA, USA) using custom functions and the Fieldtrip toolbox for EEG/MEG-analysis (Donders Institute for Brain, Cognition and Behaviour, Radboud University Nijmegen, the Netherlands. See http://www.ru.nl/neuroimaging/fieldtrip).
- Treder MS, Blankertz B: (C)overt attention and visual speller design in an ERP-based brain-computer interface. Behav Brain Funct 2010, 6: 28. 10.1186/1744-9081-6-28PubMed CentralView ArticlePubMedGoogle Scholar
- Brunner P, Joshi S, Briskin S, Wolpaw JR, Bischof H, Schalk G: Does the "P300" Speller Depend on Eye Gaze? J Neural Eng 2010, 7: 056013. 10.1088/1741-2560/7/5/056013PubMed CentralView ArticlePubMedGoogle Scholar
- Bianchi L, Sami S, Hillebrand A, Fawcett IP, Quitadamo LR, Seri S: Which physiological components are more suitable for visual ERP based brain-computer interface? A preliminary MEG/EEG study. Brain Topogr 2010, 23: 180-185. 10.1007/s10548-010-0143-0View ArticlePubMedGoogle Scholar
- Schreuder M, Blankertz B, Tangermann M: A New Auditory Multi-class Brain-Computer Interface Paradigm: Spatial Hearing as an Informative Cue. PLoS ONE 2010,5(4):e9813. 10.1371/journal.pone.0009813PubMed CentralView ArticlePubMedGoogle Scholar
- Höhne J, Schreuder M, Blankertz B, Tangermann M: Two-dimensional auditory P300 Speller with predictive text system. In Conf Proc IEEE Eng Med Biol Soc 2010, 1: 4185-4188.Google Scholar
- Brouwer AM, van Erp JBF: A tactile P300 brain-computer interface. Front Neuroscience 2010,4(19):036003.Google Scholar
- Blankertz B, Dornhege G, Krauledat M, Müller KR, Curio G: The non-invasive Berlin Brain-Computer Interface: Fast Acquisition of Effective Performance in Untrained Subjects. Neuroimage 2007,37(2):539-550. 10.1016/j.neuroimage.2007.01.051View ArticlePubMedGoogle Scholar
- Guger C, Ramoser H, Pfurtscheller G: Real-time EEG analysis with subject-specific spatial patterns for a Brain Computer Interface (BCI). IEEE Trans Neural Syst Rehabil Eng 2000,8(4):447-456.View ArticleGoogle Scholar
- Blankertz B, Sannelli C, Halder S, Hammer EM, Kübler A, Müller KR, Curio G, Dickhaus T: Neurophysiological Predictor of SMR-Based BCI Performance. Neuroimage 2010,51(4):1303-1309. 10.1016/j.neuroimage.2010.03.022View ArticlePubMedGoogle Scholar
- Vidaurre C, Sannelli C, Müller KR, Blankertz B: Machine-Learning Based Co-adaptive Calibration. Neural Comput 2011,23(3):791-816. 10.1162/NECO_a_00089View ArticlePubMedGoogle Scholar
- Vidaurre C, Blankertz B: Towards a Cure for BCI Illiteracy. Brain Topogr 2010, 23: 194-198. 10.1007/s10548-009-0121-6PubMed CentralView ArticlePubMedGoogle Scholar
- Treder MS, Schmidt NM, Blankertz B: Towards gaze-independent visual brain-computer interfaces. Front Comput Neurosci 2010. [Conference Abstract: Bernstein Conference on Computational Neuroscience 2010]Google Scholar
- Liu T, Goldberg L, Gao S, Hong B: An online brain-computer interface using non- flashing visual evoked potentials. J Neural Eng 2010,7(3):036003. 10.1088/1741-2560/7/3/036003View ArticlePubMedGoogle Scholar
- Acqualagna L, Treder MS, Schreuder M, Blankertz B: A novel brain-computer interface based on the rapid serial visual presentation paradigm. Conf Proc IEEE Eng Med Biol Soc 2010, 1: 2686-2689.Google Scholar
- Sauseng P, Klimesch W, Stadler W, Schabus M, Doppelmayr M, Hanslmayr S, Gruber WR, Birbaumer N: A shift of visual spatial attention is selectively associated with human EEG alpha activity. Eur J Neurosci 2005, 22: 2917-2926. 10.1111/j.1460-9568.2005.04482.xView ArticlePubMedGoogle Scholar
- Kelly SP, Lalor EC, Reilly RB, Foxe JJ: Independent Brain Computer Interface Control using Visual Spatial Attention-Dependent Modulations of Parieto-occipital Alpha. Proceedings of the 2nd International IEEE EMBS Conference on Neural Engineering, Arlington, Virginia 2005.Google Scholar
- Rihs T, Michel C, Thut G: Mechanisms of selective inhibition in visual spatial attention are indexed by alpha-band EEG synchronization. Eur J Neurosci 2007,25(2):603-10. 10.1111/j.1460-9568.2007.05278.xView ArticlePubMedGoogle Scholar
- van Gerven M, Jensen O: Attention modulations of posterior alpha as a control signal for two-dimensional brain-computer interfaces. J Neurosci Methods 2009, 179: 78-84. 10.1016/j.jneumeth.2009.01.016View ArticlePubMedGoogle Scholar
- van Gerven M, Bahramisharif A, Heskes T, Jensen O: Selecting features for BCI control based on a covert spatial attention paradigm. Neural Netw 2009, 22: 1271-1277. 10.1016/j.neunet.2009.06.004View ArticlePubMedGoogle Scholar
- Bahramisharif A, Heskes T, Jensen O, van Gerven MA: Lateralized responses during covert attention are modulated by target eccentricit. Neurosci Lett 2011, 491: 35-39. 10.1016/j.neulet.2011.01.003View ArticlePubMedGoogle Scholar
- Bahramisharif A, van Gerven M, Heskes T, Jensen O: Covert attention allows for continuous control of brain-computer interfaces. Eur J Neurosci 2010, 31: 1501-1508. 10.1111/j.1460-9568.2010.07174.xView ArticlePubMedGoogle Scholar
- Schmidt NM, Blankertz B, Treder MS: Alpha-modulation induced by covert attention shifts as a new input modality for EEG-based BCIs. Proceedings of the 2010 IEEE Conference on Systems, Man and Cybernetics (SMC2010) 2010, 481-487.View ArticleGoogle Scholar
- Kayser J: Current source density (CSD) interpolation using spherical splines - CSD Toolbox (Version 1.0). 2009. [New York State Psychiatric Institute: Division of Cognitive Neuroscience.]Google Scholar
- Tallon-Baudry C, Bertrand O, Delpuech C, Permier J: Oscillatory gamma-band (30-70 Hz) activity induced by a visual search task in humans. J Neurosci 1997, 17: 722-734.PubMedGoogle Scholar
- Worden MS, Foxe JJ, Wang N, Simpson GV: Anticipatory biasing of visuospatial attention indexed by retinotopically specific alpha-band electroencephalography increases over occipital cortex. J Neurosci 2000, 20: RC63.PubMedGoogle Scholar
- Tibshirani R: Regression shrinkage and selection via the lasso. J Royal Statist Soc B 1996, 58: 267-288.Google Scholar
- Kohavi R: A Study of Cross-Validation and Bootstrap for Accuracy Estimation and Model Selection. In IJCAI'95 Proceedings of the 14th international joint conference on Artificial intelligence. Volume 14. Montréal, Canada: Morgan Kaufmann; 1995:1137-1143.Google Scholar
- Hoerl A, Kennard R: Ridge regression. In In Encyclopedia of Statistical Sciences. Volume 8. New York: Wiley; 1988:129-136.Google Scholar
- Salzberg SL: On comparing classifiers. Pitfalls to avoid and a recommended approach. Data Mining and Knowledge Discovery 1997,1(3):317-328. 10.1023/A:1009752403260View ArticleGoogle Scholar
- Mesulam MM: Spatial attention and neglect: parietal, frontal and cingulate contributions to the mental representation and attentional targeting of salient extrapersonal events. Phil Trans Roy Soc London B 1999, 354: 1325-1346. 10.1098/rstb.1999.0482PubMed CentralView ArticlePubMedGoogle Scholar
- Blankertz B, Lemm S, Treder MS, Haufe S, Müller KR: Single-trial analysis and classification of ERP components - a tutorial. Neuroimage 2011,56(2):814-825.View ArticlePubMedGoogle Scholar
- Kübler A, Neumann N, Wilhelm B, Hinterberger T, Birbaumer N: Predictability of Brain-Computer Communication. Int J Psychophysiol 2004,18(2-3):121-129. 10.1027/0269-8803.18.23.121View ArticleGoogle Scholar
- Kübler A, Mushawar VK, Hochberg LR, Donoghue JP: BCI meeting 2005--Workshop on clinical issues and applications. IEEE Trans Neural Syst Rehabil Eng 2006,14(2):131-134. 10.1109/TNSRE.2006.875585View ArticlePubMedGoogle Scholar
- Venthur B, Scholler S, Williamson J, Dähne S, Treder MS, Kramarek MT, Müller KR, Blankertz B: Pyff - A Pythonic Framework for Feedback Applications and Stimulus Presentation in Neuroscience. Front Neuroscience 2010, 4: 179.View ArticleGoogle Scholar
This article is published under license to BioMed Central Ltd. This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.