Gaming control using a wearable and wireless EEG-based brain-computer interface device with novel dry foam-based sensors
© Liao et al; licensee BioMed Central Ltd. 2012
Received: 16 June 2011
Accepted: 28 January 2012
Published: 28 January 2012
A brain-computer interface (BCI) is a communication system that can help users interact with the outside environment by translating brain signals into machine commands. The use of electroencephalographic (EEG) signals has become the most common approach for a BCI because of their usability and strong reliability. Many EEG-based BCI devices have been developed with traditional wet- or micro-electro-mechanical-system (MEMS)-type EEG sensors. However, those traditional sensors have uncomfortable disadvantage and require conductive gel and skin preparation on the part of the user. Therefore, acquiring the EEG signals in a comfortable and convenient manner is an important factor that should be incorporated into a novel BCI device. In the present study, a wearable, wireless and portable EEG-based BCI device with dry foam-based EEG sensors was developed and was demonstrated using a gaming control application. The dry EEG sensors operated without conductive gel; however, they were able to provide good conductivity and were able to acquire EEG signals effectively by adapting to irregular skin surfaces and by maintaining proper skin-sensor impedance on the forehead site. We have also demonstrated a real-time cognitive stage detection application of gaming control using the proposed portable device. The results of the present study indicate that using this portable EEG-based BCI device to conveniently and effectively control the outside world provides an approach for researching rehabilitation engineering.
KeywordsElectroencephalography (EEG) Brain-computer interface Dry EEG sensor Cognitive applications
The monitoring of brain activity is widely used for investigative neuroscience and rehabilitation engineering . The brain-computer interface (BCI) technique has become a major tool that provides a direct communication pathway between the brain and the external world by translating signals from brain activities into machine codes or commands [2–5]. The acquisition of brain activities by BCIs can be divided into two different categories : invasive BCIs [6, 7] and noninvasive BCIs [8, 9]. An invasive BCI is implanted directly into the grey matter of the brain to obtain the highest quality of brain activity signals or to send external signals into the brain . However, invasive BCIs depend on surgical techniques and are potentially risky because of the interaction between the device and brain tissues when used in the long term. Therefore, noninvasive BCIs have become another major BCI research direction. These noninvasive devices are worn on the outside of the head and are removable. Recently, electroencephalogram (EEG)-based BCIs have been shown to provide a feasible and noninvasive method to communicate between the human brain and external devices [10, 11]. The use of EEG signals has become the most common approach for BCIs because of their usability and strong reliability [12, 13]. In recent years, the advanced designs of the sensors and system techniques have made it possible to integrate the sensors into portable acquisition devices to measure a wide variety of physiological signals. The development of EEG-based BCIs and their corresponding applications have also been reported [14–16]. A BCI system that is based on steady-state visual-evoked potentials (SSVEPs) has been commonly used for controlling functional neuroprostheses [8, 17–19]. Gollee et al. used a BCI system that was based on SSVEPs combined with a functional electrical stimulation (FES) system to allow the user to control stimulation settings and parameters . In addition, a P300-based BCI has also been developed for disabled users [10, 20–22]. The current applications of P300-based BCI systems range from controlling a virtual hand  to neuroprostheses [21, 23]. EEG-based BCIs provide a reliable, fast, and efficient solution for the communication between humans and computers. However, most of the above-mentioned BCIs focus on feasible applications by using general systems or sensors. Measuring EEG signals with a portable BCI device in a comfortable manner during daily life is still an important issue that requires further study .
The most frequently used wet- or micro-electro-mechanical-system (MEMS)-type EEG sensors for EEG-based BCI devices have some limitations , such as skin abrasion and the required use of conductive gel; moreover, they are time-consuming, uncomfortable, and often painful for users [25–27]. These sensors are also inappropriate for long-term EEG measurements because the EEG signal quality may degrade over an extended period of time because of skin regeneration and/or the drying of the conductive gel [24, 28]. In addition, most of the non-gel-based dry electrodes were made using the MEMS technique [25, 26, 29, 30]. However, the dry MEMS electrode technique relies on invasive penetration into the skin to acquire the EEG signals [25, 26]. In addition to the drawback of skin penetration, MEMS electrodes are also more costly to manufacture than gel-based or other types of dry electrodes. Our recent study utilized dry foam-based electrodes to acquire forehead EEG signals without any skin preparation or gels . However, the size of most of the EEG-based BCI devices is too large for them to be considered a portable device , which is inconvenient for users. Therefore, developing a portable EEG-based BCI device of a smaller size (smaller than 5 × 5 × 5 cm3) with zero-preparation, dry EEG sensors is an important goal.
In this study, we developed a wearable, EEG-based BCI device with a novel dry foam-based sensor and demonstrated a cognitive application of gaming control. This device consisted of a wireless EEG acquisition device and a computer. The wireless EEG acquisition device included dry foam sensors and a wireless EEG acquisition module. The proposed dry foam sensors worked without the application of conductive gel; however, they were able to provide good conductivity to effectively acquire an EEG signal. Moreover, this sensor can be properly integrated into the wireless EEG acquisition device. In contrast to other portable BCI devices using the wet sensors, which require a skin preparation process [32, 33], users using the proposed device can monitor their EEG states more quickly, comfortably and effectively during daily life and can transmit EEG signals to a personal computer to process signals directly. In addition, a real-time focusing detection algorithm  was implemented in our device as an EEG-based gaming interface to detect the real-time cognitive state of the user in a comfortable manner. The use of this device complements other existing BCI approaches for investigating the cognitive states of neuronal activation and behavioral responses in daily life.
Materials and methods
A. Design of the dry EEG sensors
B. Wireless EEG acquisition module
C. The mechanism of the wearable EEG-based BCI device
The quick-placement mechanism for the proposed EEG-based BCI device was designed to let the dry EEG sensors attach to the user's forehead (F10) easily and quickly, as shown in Figure 1A. This device consists of three dry foam sensors and a wireless EEG acquisition module that contains a battery. An elastic band was adjustable to fit the users' head sizes, as indicated in Figure 1A. This mechanism was also used to maximize the skin-sensor contact area to maintain low impedance while probing the EEG signals using the dry EEG sensors . This mechanism did not lead to any permanent or detrimental effects to the forehead skin. Noted that all of the channels of the porposed device are both used the dry foam-based electrodes. The application of the wearable EEG acquisition device allowed users to monitor their EEG signals more conveniently and comfortably.
D. Gaming control via users' focus levels measured by EEG signals with the proposed device
X indicates the recorded samples in 2-s, where X n is the nth sample. Y is the power spectrum of X, which is calculated by the FFT; Y n indicates the power in the nth rhythm. The average power within the alpha band P α is obtained by averaging the value of Y in the range from 8 to 12 Hz. PR α is the inverse of this average power in the alpha rhythm. The FF value is assumed to be equal to PR α . The power of the alpha rhythm has a negative relationship with the value of the FF. If the user is not focused, the power of the alpha rhythm will increase, and the value of the FF will decrease.
Lastly, a comparison of the user's current FF value with that at baseline was used to confirm whether or not the user was in a focused state and then to determine the FL based on the user's focused state. We assumed based on user feedback that the user was in a focused state in the beginning (baseline) and defined the user's FF at baseline as the baseline FF (BFF), which is the average of the FFs within the initial ten seconds. After we determined the BFF, the FF values were calculated every 2 s and were compared to the BFF. If the current FF value was higher than the BFF value, the user was considered to be in the focused state. If the current FF value was lower than the BFF value, the user was considered to be in the unfocused state. Finally, the values of the FL variation were determined according to the user's mental focus state. If the user was focused, the FL increased and vice-versa.
To apply this algorithm in our game, the gaming process consisted of ten trials, and each trial persisted for ten seconds, during which a shot was executed. The BFF was calculated during the initial ten seconds, and then the game began. For every shot, the FL was initialized to zero and increased or decreased according to the FF value. The FF values were calculated every 2 s and were then compared to the BFF. If the FF value was higher than the BFF during that 2 s, the FL increased by one level. If not, the FL decreased by one level. When the user pushed the mouse button, a circle on the target indicated the focus zone based on the user's FL level. This circle indicated the possible deviation of the shot from the center of the target and was scaled relative to the FL. If the FL was high, the circle became small, indicating that the possible deviation of the shot would be small and that the arrow would be close to the center of the target, and vice versa. Users attempted to focus during the game to make the FL as high as possible and to get a high score. After each shot, the score was calculated as the deviation of the shot from the center of the target and was summed to the user's total score, which was shown on the screen (Figure 3A). After ten trials, the total score was the sum of the ten scores from the ten shots. Noted that the users in all of the experiments performed the task without any pre-training or practice
E. Verification of the FL algorithm with the proposed EEG device and dry sensors: comparison of the users' focused mental state with the FL algorithm
To confirm that the FL algorithm represented the user's level of focus, we compared the FL algorithm to a general measurement method for the focused mental state. According to the previous studies on mental focus, the most commonly used method for measuring the state of mental focus is called the "short-term memory test" [39–41]. In the beginning of this test, the user watches a rapid series of pictures over a few seconds. Next, a picture is shown to the user and the user must indicate whether or not this picture had been shown before. Previous authors indicated that the accuracy of this test is high when the user is in the focused state and low when the user is in the unfocused state . Belojevic et al. confirmed that the accuracy of this test was high when users take the test in silence, indicating that the users were more focused, while the accuracy of the test was low when conducted under noisy conditions, indicating that the users were in an unfocused state .
Ten users participated in this short-term memory experiment, and all of them were right-handed and aged 24-27. All experiments took place during the afternoon with a computer and earphones, and users were asked to sit comfortably in front of the computer without crossing their legs [40, 41]. After all of the short-term memory trials, then we also asked users to perform measurement experiment of the FF values and play the archery game with the proposed BCI device under quiet and noisy conditions. Finally, a t-test was performed on the FF values, the user focus levels (results from the short-term memory experiment) and the scores from the archery game to establish the relationship between these variables [40, 41].
Results and Discussion
A. Characteristics of the proposed wearable EEG-based BCI device
B. Results of the relationships between short-term memory testing, FF values and BCI gaming scores under quiet and noisy conditions
Results of the short-term memory experiment under quiet and noisy conditions.
* Paired t-test.
Results of the FF values and gaming scores under quiet and noisy conditions.
In addition, to ensure the relationship between mental focus and the measured FF values, a Pearson product-moment correlation was performed to determine whether the results of the short-term memory experiment were truly related to the measured FF values [40, 46]. According to the Pearson correlation results, indicate that the results of the short-term memory experiment were highly positively related to the measured FF values under both quiet (r = 0.918) and noisy (r = 0.658) conditions [40, 46]. According to the results of the Pearson correlation, the measured FF values were significantly positively correlated to the results of the short-term memory experiment. Thus, the measured FF values truly represented the users' level of mental focus. The measured FF values were also used to quantify the gaming scores (FL values).
The total gaming scores under quiet and noisy conditions are shown in Table 2. The average score was 9.01 under quiet conditions and 7.39 under noisy conditions. The game scores showed a significant difference between noisy and quiet conditions (p < 0.05). Note that the game scores are positively correlated to the measured FF values. Thus, it is significant that the game scores are lower if the user performs the test under noisy conditions or in the unfocused state. These experimental results show that the FF values are an indicator of the focused state and that the FL algorithm is a reliable method for measuring the users' level of focus via EEG signals. Our results indicate that using the FL algorithm to measure the focus level of the user is useful not only in the context of short-term memory, but also in the measurement of daily life activities. Traditionally, users only perform the short-term memory experiment during testing; these methods are not used to test the users' focused state in other cognitive experiments . Applications of the short-term memory experiment have not been explored in combination with other cognitive testing procedures . However, according to our results, using the novel EEG-based BCI device with the FL algorithm, users can undertake the cognitive experiment without any inconvenience while simultaneously undergoing measurement of the focused state. Moreover, with the proposed device, it is possible to display real-time feedback to remind the users' focus state, which traditional short-term memory tests cannot do. Accordingly, this is a significant advantage of the FL algorithm with our portable EEG-based BCI device, which does not require any skin preparation, over traditional approaches.
In the present study, we proposed a wearable EEG-based BCI device with dry EEG sensors for cognitive state monitoring. In addition, we demonstrated its use during EEG-based gaming control. The use of dry EEG sensors provides several advantages: 1) in contrast to conventional EEG sensors, the dry foam-based sensors can be used without conductive gel; 2) the elasticity of the substrate of the dry EEG sensors allows them to adapt to irregular skin surfaces to maintain low sensor-skin impedance; and (3) the fabrication process is inexpensive, comparing with other types of dry sensors. Our experimental results demonstrated successful, stable EEG measurements using the dry foam-based EEG sensors through the corresponding wireless EEG acquisition device; these results were almost identical to those seen with conventional EEG sensors in which conductive gel are used. Therefore, in contrast to the conventional EEG-based BCI devices using the wet sensors, our device with dry foam-based EEG sensors have the potential for allowing routine and repetitive measurements. Moreover, a portable, wireless and low-power-consumption EEG acquisition module was successfully used for long-term EEG monitoring. The dry EEG sensors and the wireless EEG acquisition module were embedded into a wearable EEG acquisition device. Using our wearable EEG-based BCI device without conductive gel will allow users to monitor their EEG states more comfortably during daily life.
A cognitive application of EEG-based gaming control was also demonstrated in this study using this portable device. A personal computer was used as the platform to run a real-time focused feature detection algorithm and an EEG monitoring program, which were used to monitor the user's cognitive state. Our data indicate that this wearable EEG-based BCI device and the corresponding algorithm can be reliably used to control outside-world applications for general users or researchers. This device complements other existing BCI approaches for investigating the human cognitive states of neuronal activation and behavioral responses in daily life.
This work was supported in part by the UST-UCSD International Center of Excellence in Advanced Bioengineering, sponsored by the Taiwan National Science Council I-RiCE Program under Grant Number NSC-100-2911-I-009-101. This work was also supported in part by the Aiming for the Top University Plan of National Chiao Tung University, the Ministry of Education, Taiwan, under Grant Number 100W9633, in part by the National Science Council, Taiwan, under Contract NSC-100-2321-B-009-003 and in part by the VGHUST Joint Research Program, Tsou's Foundation, Taiwan, under Contract VGHUST101-G5-2-1.
- Thakor NV: Biopotentials and electro-physiology measurement. In The measurement, Instrumentation, and Sensors Handbook. CRC Press; 1999:74–1.
- Grimaldi G, Manto M: Neurological Tremor: Sensors, Signal Processing and Emerging Applications. Sensors 2010, 10:1399–1422.PubMedView Article
- Mason SG, Birch GE: A general framework for brain computer interface design. IEEE Transactions on Neural Systems and Rehabilitation Engineering 2003, 11:70–85.PubMedView Article
- Rao R, Scherer R: Brain-Computer Interfacing [In the Spotlight]. Signal Processing Magazine, IEEE 2010, 27:152–150.View Article
- Thakor N: In the Spotlight: Neuroengineering. IEEE Reviews in Biomedical Engineering 2009, 2:18–20.PubMedView Article
- Song YK, Borton DA, Park S, Patterson WR, Bull CW, Laiwalla F, Mislow J, Simeral JD, Donoghue JP, Nurmikko AV: Active microelectronic neurosensor arrays for implantable brain communication interfaces. Neural Systems and Rehabilitation Engineering, IEEE Transactions on 2009, 17:339–345.View Article
- Chen Y-Y, Lai H-Y, Lin S-H, Cho C-W, Chao W-H, Liao C-H, Tsang S, Chen Y-F, Lin S-Y: Design and fabrication of a polyimide-based microelectrode array: Application in neural recording and repeatable electrolytic lesion in rat brain. Journal of Neuroscience Methods 2009, 182:6–16.PubMedView Article
- Xiaorong G, Dingfeng X, Ming C, Shangkai G: A BCI-based environmental controller for the motion-disabled. IEEE Transactions on Neural Systems and Rehabilitation Engineering 2003, 11:137–140.View Article
- Royer AS, Doud AJ, Rose ML, He B: EEG control of a virtual helicopter in 3-Dimensional space using intelligent control strategies. IEEE Transactions on Neural Systems and Rehabilitation Engineering 2010, 18:581–589.PubMedView Article
- Chen W-d, Zhang J-h, Zhang J-c, Li Y, Qi Y, Su Y, Wu B, Zhang S-m, Dai J-h, Zheng X-x, Xu D-r: A P300 based online brain-computer interface system for virtual hand control. Journal of Zhejiang University - Science C 2010, 11:587–597.View Article
- Muller-Putz GR, Scherer R, Neuper C, Pfurtscheller G: Steady-state somatosensory evoked potentials: suitable brain signals for brain-computer interfaces? IEEE Transactions on Neural Systems and Rehabilitation Engineering 2006, 14:30–37.PubMedView Article
- Mason S, Jackson M, Birch G: A general framework for characterizing studies of brain interface technology. Annals of Biomedical Engineering 2005, 33:1653–1670.PubMedView Article
- Mason S, Bashashati A, Fatourechi M, Navarro K, Birch G: A Comprehensive Survey of Brain Interface Technology Designs. Annals of Biomedical Engineering 2007, 35:137–169–169.PubMedView Article
- Rebsamen B, Guan C, Zhang H, Wang C, Teo C, Ang MH Jr, Burdet E: A Brain Controlled Wheelchair to Navigate in Familiar Environments. IEEE Transactions on Neural Systems and Rehabilitation Engineering 2010, 1–1.
- Casson A, Yates D, Smith S, Duncan J, Rodriguez-Villegas E: Wearable Electroencephalography. IEEE Engineering in Medicine and Biology Magazine 2010, 29:44–56.PubMedView Article
- Derong L, Zhongyu P, Lloyd SR: A neuralnetwork method for detection of obstructive sleep apnea and narcolepsy based on pupil size and EEG. IEEE Transactions on Neural Networks 2008, 19:308–318.View Article
- Gollee H, Volosyak I, McLachlan AJ, Hunt KJ, ser A: An SSVEP-based brain-computer interface for the control of functional electrical stimulation. IEEE Transactions on Biomedical Engineering 2010, 57:1847–1855.PubMedView Article
- Ortner R, Allison BZ, Korisek G, Gaggl H, Pfurtscheller G: An SSVEP BCI to control a hand orthosis for persons with tetraplegia. IEEE Transactions on Neural Systems and Rehabilitation Engineering 2010, 19:1–5.PubMedView Article
- Nielsen KD, Cabrera AF, Omar Feix do N: EEG based BCI-towards a better control. Brain-computer interface research at aalborg university. IEEE Transactions on Neural Systems and Rehabilitation Engineering 2006, 14:202–204.PubMedView Article
- Haihong Z, Cuntai G, Chuanchu W: Asynchronous P300-Based Brain--Computer Interfaces: A Computational Approach With Statistical Models. IEEE Transactions on Biomedical Engineering 2008, 55:1754–1763.View Article
- Hoffmann U, Vesin J-M, Ebrahimi T, Diserens K: An efficient P300-based brain-computer interface for disabled subjects. Journal of Neuroscience Methods 2008, 167:115–125.PubMedView Article
- Pires G, Nunes U: A Brain Computer Interface methodology based on a visual P300 paradigm. Intelligent Robots and Systems, 2009. IROS 2009. IEEE/RSJ International Conference on 2009, 4193–4198.View Article
- Mugler EM, Ruf CA, Halder S, Bensch M, Kubler A: Design and implementation of a P300-based brain-computer interface for controlling an internet browser. IEEE Transactions on Neural Systems and Rehabilitation Engineering 2010, 18:599–609.PubMedView Article
- Chi YM, Jung TP, Cauwenberghs G: Dry-contact and noncontact biopotential electrodes: methodological review. IEEE Reviews in Biomedical Engineering 2010, 3:106–119.PubMed
- Griss P, Tolvanen-Laakso HK, Meriläinen P, Stemme G: Characterization of micromachined spiked biopotentials electrodes. IEEE Transactions on Biomedical Engineering 2002, 49:597–604.PubMedView Article
- Griss P, Enoksson P, Tolvanen-Laakso HK, Meriläinen P, Ollmar S, Stemme G: Micromachined electrodes for biopotential measurement. IEEE Journal of Microelectromechanical Systems 2001, 10:10–15.View Article
- Kübler A: Brain painting: evaluation of BCI use in the field-problems and prospects. presented at the 2nd tools for brain-computer interaction workshop, Rome 2010.
- Lin C-T, Lin F-C, Chen S-A, Lu S-W, Chen T-C, Ko L-W: EEG-based brain-computer interface for smart living environmental auto-adjustment. Journal of Medical and Biological Engineering 2010, 30:237–245.View Article
- Ruffini G, Dunne S, Farrés E, Marco-Pallarés J, Ray C, Mendoza E, Silva R, Grau C: A dry electrophysiology electrode using CNT arrays. Sensors and Actuators A: Physical 2006, 132:34–41.View Article
- Ko L-W, Chiou J-C, Lin C-T, Hong C-T, Jung T-P, Liang S-F, Jeng J-L: Using novel MEMS EEG sensors in detecting drowsiness application. In Proceeding of IEEE Biomedical Circuits and Systems Conference (BioCAS 2006). London, UK; 2006:33–36.
- Lin CT, Liao LD, Liu YH, Wang IJ, Lin BS, Chang JY: Novel Dry Polymer Foam Electrodes for Long-Term EEG Measurement. IEEE Transactions on Biomedical Engineering 2011, 58:1200–1207.PubMedView Article
- Kim Y, Baek H, Kim J, Lee H, Choi J, Park K: Helmet-based physiological signal monitoring system. European Journal of Applied Physiology 2009, 105:365–372.PubMedView Article
- Farshchi S, Nuyujukian PH, Pesterev A, Mody I, Judy JW: A tinyOS-enabled MICA2-based wireless neural interface. IEEE Transactions on Biomedical Engineering 2006, 53:1416–1424.PubMedView Article
- Lin Chin-Teng, Pal NR, Chien-Yao C, Tzyy-Ping J, Li-Wei K, Sheng-Fu L: An EEG-based subject- and session-independent drowsiness detection. presented at the IEEE International Joint Conference on Neural Networks 2008.
- Lin CT, Liao LD, Liu YH, Wang IJ, Lin BS, Chang JY: Novel dry polymer foam electrodes for long-term EEG measurement. IEEE Transactions on Biomedical Engineering 2010.
- Lin C-T, Ko L-W, Chiou J-C, Duann J-R, Huang R-S, Chiu T-W, Liang S-F, Jung T-P: Noninvasive neural prostheses using mobile and wireless EEG. Proceedings of the IEEE 2008, 96:1167–1183.View Article
- Ferree ThomasC, Phan Luu, Russell GeraldS, Tucker DM: Scalp electrode impedance, infection risk, and EEG data quality. Clinical Neurophysiology 2001, 112:536–544.PubMedView Article
- Lin CT, Chung IF, Ko LW, Chen YC, Liang SF, Duann JR: EEG-based assessment of driver cognitive responses in a dynamic virtual-reality driving environment. IEEE Transactions on Biomedical Engineering 2007, 54:1349–1352.PubMedView Article
- Goldstein EB: Cognitive psychology. London: Thomson Leaning; 2005.
- Belojevic G, Slepcevic V, Jakovljevic B: Mental performance in noise: The role of introversion. Journal of Environmental Psychology 2001, 21:209–213.View Article
- Gathercole SE: Cognitive approaches to the development of short-term memory. Trends in cognitive sciences 1999, 3:410–419.PubMedView Article
- Dobbs S, Furnham A, McClelland A: The effect of background music and noise on the cognitive test performance of introverts and extraverts. Applied Cognitive Psychology 2010.
- Smith D, Baranski J, Thompson M, Abel S: The effects of background noise on cognitive performance during a 70 hour simulation of conditions aboard the International Space Station. Noise and Health 2003, 6:3–16.PubMed
- Berti S, Schroger E: A comparison of auditory and visual distraction effects: behavioral and event-related indices. Cognitive Brain Research 2001, 10:265–273.PubMedView Article
- Grimnes S: Impedance measurement of individual skin surface electrodes. Medical and Biological Engineering and Computing 1983, 21:750–755.PubMedView Article
- Coolican H: Research methods and statistics in psychology. 5th Revised edition. Hodder Education; 2009.
This is an Open Access article distributed under the terms of the Creative Commons Attribution License (http://creativecommons.org/licenses/by/2.0), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.