Speaking and cognitive distractions during EEG-based brain control of a virtual neuroprosthesis-arm
© Foldes and Taylor; licensee BioMed Central Ltd. 2013
Received: 26 April 2013
Accepted: 17 December 2013
Published: 21 December 2013
Brain-computer interface (BCI) systems have been developed to provide paralyzed individuals the ability to command the movements of an assistive device using only their brain activity. BCI systems are typically tested in a controlled laboratory environment were the user is focused solely on the brain-control task. However, for practical use in everyday life people must be able to use their brain-controlled device while mentally engaged with the cognitive responsibilities of daily activities and while compensating for any inherent dynamics of the device itself. BCIs that use electroencephalography (EEG) for movement control are often assumed to require significant mental effort, thus preventing users from thinking about anything else while using their BCI. This study tested the impact of cognitive load as well as speaking on the ability to use an EEG-based BCI.
Six participants controlled the two-dimensional (2D) movements of a simulated neuroprosthesis-arm under three different levels of cognitive distraction. The two higher cognitive load conditions also required simultaneously speaking during BCI use. On average, movement performance declined during higher levels of cognitive distraction, but only by a limited amount. Movement completion time increased by 7.2%, the percentage of targets successfully acquired declined by 11%, and path efficiency declined by 8.6%. Only the decline in percentage of targets acquired and path efficiency were statistically significant (p < 0.05).
People who have relatively good movement control of an EEG-based BCI may be able to speak and perform other cognitively engaging activities with only a minor drop in BCI-control performance.
Users of BCI-controlled devices, such as an upper-limb neuroprosthesis [1–4], must be able to use their device while talking and performing other cognitive tasks. Talking could potentially degrade EEG-controlled BCIs due to power spectral changes associated with verbal and cognitive engagement and the large electrical signals from muscles under the scalp. However, EEG-based BCI systems are usually evaluated with the subjects sitting quietly and focusing exclusively on the BCI task. This study tested the effects of speaking and cognitive load on the ability to command an upper-limb neuroprosthesis using EEG. All study activities were approved by the Institutional Review Board of the Louis Stokes Cleveland Veteran’s Affairs Medical Center.
Participants performed a ‘center-out-and-hold’ task where they had 12 seconds to move a cursor located at the fingertip of the virtual arm from the center of the workspace to one of four radial targets. Participants had to keep the fingertip/cursor touching the target for one second for the trial to count as a success.
To assess the effects of performing cognitive tasks and speaking on movement control, participants were tested under three levels of cognitive load (CL), two of which involved speaking. Random letters were played out loud every two seconds as participants performed the center-out-and-hold task. Participants were instructed to either: 1) only focus on controlling the virtual neuroprosthesis arm (‘No CL’), 2) repeat each letter immediately after hearing it (‘Moderate CL’), or 3) remember and repeat the previous letter immediately after hearing the current letter in a ‘one-back’ fashion (‘High CL’). The second and third conditions both required speech encoding and production but engaged working memory to different degrees . Subjects five and six were asked to speak each cued letter five times instead of once to generate more speech-related activity while maintaining the same cognitive load. Subjects five and six also only used 16 instead of 32 electrodes distributed over the same area due to connector issues.
Letters were played during the ‘No CL’ condition to ensure the auditory environment was the same across all tests. No speaking or cognitive tasks were performed during the initial open-loop data collection that was used to build the decoder. After one block of practice, each participant generated at least 40 center-out-and-hold movements under each of the three conditions. The order of test conditions was varied across subjects and each subject repeated their sequence of conditions twice to minimize learning and fatigue confounds.
Performance was assessed with three metrics: a) the percentage of trials that were successful, b) the average time required to successfully reach a target, and c) the average path efficiency (i.e. shortest path to the target divided by the length of the actual path taken).
This study sought to quantify the impact of additional cognitive load on an EEG-BCI where participants used different combinations of hand and foot movements to command the 2D velocity of the fingertip of a virtual neuroprosthesis. Our results demonstrated that the added cognitive load imposed by the letter repetition tasks produced only a moderate drop in control performance on average. However, given the added challenges of using different types of BCIs, it is important to further characterize how cognitive tasks and speech production impact BCI performance.
We have shown previously that strong jaw muscle activity associated with teeth clenching can be detected on EEG electrodes across much of the scalp . Therefore we anticipated that jaw muscle activity associated with talking might broadly increase the power in the recorded signals during the moderate-CL and high-CL tasks. Such an increase in power would have resulted in a command bias toward the left-distal direction because that is the direction associated with increased mu and beta power due to resting the hands and feet. However, the trajectories in Figure 2 did not show any systematic skewing toward the left-distal part of the workspace. Therefore, these results suggest the muscle activity required for simple speech production was not enough to cause a significant bias in directional control. However, additional studies are needed to determine if more animated speech, facial expressions, and additional cognitive burdens (e.g. mental calculations, emotions) may still disrupt BCI use.
The ability to use an EEG-BCI can vary widely between individuals . This study only included participants that naturally had movement-related modulation because it is likely that relatively good modulation will be needed for an individual to adopt a BCI into their daily life. People with more difficulty using a sensorimotor-rhythm-based BCI may have more problems performing cognitive tasks during BCI use. It is expected that these users could improve their control with additional training . However, further studies are needed that include individuals with paralysis and people who have improved their EEG modulation with training to fully assess the practicality of BCI technology for a broad range of users.
The project described was supported by Dept. of Veterans Affairs Merit Review #B4195R and Pre-Doctoral Rehabilitation Engineering Fellowship, National Institutes of Health NINDS F31-NS065710, NINDS R01-NS058871, NIBIB T32-EB004314, NINDS N01-NS-5-2365, and NICHD N01-HD-5-3403. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health. We are very grateful to Holle Carey and Melissa Schmitt for their assistance in collecting this data.
- Peckham PH, Knutson JS: Functional electrical stimulation for neuromuscular applications. Ann Rev Biomed Eng. 2005, 7: 327-360. 10.1146/annurev.bioeng.6.040803.140103.View ArticleGoogle Scholar
- Chae J, Kilgore K, Triolo R, Creasey G: Functional neuromuscular stimulation in spinal cord injury. Phys Med Rehabil Cli. 2000, 11: 209-226. xGoogle Scholar
- Wolpaw JR, Birbaumer N, McFarland DJ, Pfurtscheller G, Vaughan TM: Brain-computer interfaces for communication and control. Clin Neurophysiol. 2002, 113: 767-791. 10.1016/S1388-2457(02)00057-3.View ArticlePubMedGoogle Scholar
- Collinger JL, Foldes ST, Bruns TM, Wodlinger B, Gaunt R, Weber DJ: Neuroprosthetic Technology for Individuals with Spinal Cord Injury. J Spinal Cord Med. 2013, 36 (4): 258-272. 10.1179/2045772313Y.0000000128.PubMed CentralView ArticlePubMedGoogle Scholar
- Foldes ST, Taylor DM: Offline comparison of spatial filters for two-dimensional movement control with noninvasive field potentials. J Neural Eng. 2011, 8: 046022-10.1088/1741-2560/8/4/046022.View ArticlePubMedGoogle Scholar
- Chadwick EK, Blana D, Van den Bogert AJT, Kirsch RF: A real-time, 3-D musculoskeletal model for dynamic simulation of arm movements. Ieee T Bio-Med Eng. 2009, 56: 941-948.View ArticleGoogle Scholar
- Chadwick EK, Blana D, Simeral JD, Lambrecht J, Kim SP, Cornwell AS, Taylor DM, Hochberg LR, Donoghue JP, Kirsch RF: Continuous neuronal ensemble control of simulated arm reaching by a human with tetraplegia. J Neural Eng. 2011, 8: 034003-10.1088/1741-2560/8/3/034003.PubMed CentralView ArticlePubMedGoogle Scholar
- Frankemolle AMM, Wu J, Noecker AM, Voelcker-Rehage C, Ho JC, Vitek JL, McIntyre CC, Alberts JL: Reversing cognitive-motor impairments in Parkinson’s disease patients using a computational modelling approach to deep brain stimulation programming. Brain. 2010, 133 (Pt 3): 746-761.PubMed CentralView ArticlePubMedGoogle Scholar
- Foldes ST, Taylor DM: Discreet discrete commands for assistive and neuroprosthetic devices. Ieee T Neur Sys Reh. 2010, 18: 236-244.View ArticleGoogle Scholar
- Blankertz B, Sannelli C, Halder S, Hammer EM, Kübler A, Müller K-R, Curio G, Dickhaus T: Neurophysiological predictor of SMR-based BCI performance. Neuroimage. 2010, 51: 1303-1309. 10.1016/j.neuroimage.2010.03.022.View ArticlePubMedGoogle Scholar
This article is published under license to BioMed Central Ltd. This is an open access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.