An EEG-based brain-computer interface for gait training

Transcription

1 An EEG-based brain-computer interface for gait training Dong Liu 1,2, Weihai Chen 1, Kyuhwa Lee 2, Zhongcai Pei 1,andJosédelR. Millán 2 1. School of Automation Science and Electrical Engineering, Beihang University (BUAA), , Beijing, China beihangld@buaa.edu.cn 2. Defitech Chair in Brain-Machine Interface (CNBI), École Polytechnique Fédérale de Lausanne (EPFL), Campus Biotech H4, 1202, Geneva, Switzerland jose.millan@epfl.ch Abstract: This work presents an electroencephalography (EEG)-based Brain-computer Interface (BCI) that decodes cerebral activities to control a lower-limb gait training exoskeleton. Motor imagery (MI) of flexion and extension of both legs was distinguished from the EEG correlates. We executed experiments with 5 able-bodied individuals under a realistic rehabilitation scenario. The Power Spectral Density (PSD) of the signals was extracted with sliding windows to train a linear discriminate analysis (LDA) classifier. An average classification accuracy of 0.67±0.07 and AUC of 0.77±0.06 were obtained in online recordings, which confirmed the feasibility of decoding these signals to control the gait trainer. In addition, discriminative feature analysis was conducted to show the modulations during the mental tasks. This study can be further implemented with different feedback modalities to enhance the user performance. Key Words: Brain-computer Interface (BCI), electroencephalography (EEG), motor imagery (MI), gait training 1 INTRODUCTION Brain-computer Interfaces (BCIs) are direct communication and control systems between the brain and an external device, bypassing the physiological output pathways of peripheral nerves and muscles [1]. For severely disabled individuals suffering from illnesses such as amyotrophic lateral sclerosis (ALS), spinal cord injury (SCI) or strokes, BCI provides a promising approach for them to interact with the surroundings. In the past decades, non-invasive EEG-based BCI has been developed in both experimental contexts and clinical trials, e.g., brain-actuated intelligent wheelchair [2], powered lower-limb exoskeleton [3] and robotic arm [4]. EEG-based BCI can be categorized into spontaneous and evoked paradigms, according to the types of control signals. Evoked BCI exploits brain patterns in response to visual, auditory or tactile stimulus. Typical examples are steady state visually evoked potential (SSVEP) and P300. SSVEP is brain activity modulated in the visual cortex evoked by repetitive visual stimuli which has distinctive frequencies. It has been used to control a mobile robot [5]. Furthermore, P300, elicited about 300 ms after the desired target stimulus presented within a random sequence of nontarget stimuli, can be implemented for the user to navigate unknown and evolving scenarios [6]. The advantages of these systems include high information transfer rate (ITR), minimum training sessions and high accuracy. However, the subjects are easy to be fatigue with the consecutive stimulus and they have to keep following the cues all the time in order to transfer the information, which limits the applica- This work was supported by the National Natural Science Foundation of China under Grant No , No , and the China Scholarship Council under Contract bility of evoked BCIs. Spontaneous BCI, on the other hand, is fully operated by the user without any external stimulus. The subject can send their commands voluntarily to interact with the devices, e.g., neuroprosthesis and wheelchair. One of the most commonly-used neurophysiological signatures in such BCI systems is the modulation of sensorimotor rhythms (SMRs) during motor imagery (MI). SMRs can display event-related desynchronisation (ERD) and eventrelated synchronization (ERS) [7], which reflects the power changes in certain frequency bands, e.g., mu (7 13 Hz) and beta bands (15 30 Hz). Imagination of kinaesthetic movement of left hand, right hand and/or both feet can produce distinctive lateralized patterns on the primary sensory and motor cortex. Therefore, MI of left and right hand has been widely exploited as EEG correlates in BCI frameworks for motor training. Much attention has been focused on the development of brain-controlled robots for assistive strategy and rehabilitation tools using upper-limb MI [8]. More recently, interest has been devoted to studies about the gait rehabilitation. Brain-actuated robotic devices have been shown as an alternative to conventional physical therapy. In these paradigms, either healthy subjects or patients were required to imagine movement of the arms to modulate EEG activity to generate motor commands to control the lower-limb movement. For instance, MI of hand movement was used to control the Lokomat (Hocoma AG, Volketswil, Switzerland), a commercially available robotic walker [9]. Another work by Lee et al used MI of both hands to build a cascaded ERD classifier, to control the Rex (Rex Bionics LTD, Auckland, New Zealand), a hand-free, self-supporting robotic mobility device [10]. In contrast, directly performing lower-limb MI as mental

2 Feature engineering Signal acquisition Classification/regression Action generation Subject Feedback interface External device Figure 1: Architecture of a brain-controlled system. In the current study, the external device is a lower-limb gait training exoskeleton. (a) Figure 2: Visual cues presented to the subject during the experiment: (a) Baseline period and (b) MI period. (b) tasks would be an intuitive yet challenging method for gait rehabilitation oriented BCI. Cortical homunculus, a physical representation of functional region within the brain, has shown a relatively small brain region mapping to leg movements [11]. The cortical areas of legs, feet and toes are located closely at primary motor cortex from the frontal lobe. Besides, Brodmann cortical areas show that left and right legs are controlled by overlapping regions from the primary somatosensory cortex. As a result, the implementation of discriminative features from MI of left and right legs are difficult to be applied in non-invasive BCI. However, a recent study has shown that the direction of imagined visual motion can be decoded using functional magnetic resonance imaging (fmri) [12]. Similar patterns were performed in human gait therapy, e.g., repetitive and cyclic knee and ankle movement. Other lower-limb BCIs include sitting and standing classification from pre-movement s- tates [13] and ankle dorsiflexion MI detection from readiness potential, a type of slow cortical potentials (SCPs) [14]. The main goal of this work is to build an EEG-based BCI with lower-limb MI as control signals. In contrast to hand MI, we used MI of directional movement of both legs to differentiate flexion and extension. There are three characteristics of the proposed framework. First, directly usingmioflegflexion and extension is natural and intuitive, which is in consistent with the physical gait training procedures. Second, this paradigm is a spontaneous BCI, without time-locked to any external stimulus. The subject can initialize the task in a self-paced manner and get involved for a long period with full manipulation. Third, visual feedback was provided to form a closed loop. The subjects can modulate their mental strategies accordingly in order to enhance the performance. Furthermore, we performed feature analysis to show the discriminate power in the classification of leg flexion and extension. Another classification was conducted between mental task and rest period (baseline) in order to show the brain pattern modulations. In this work, we will first describe our BCI framework as a pattern recognition system. Then we will continue by materials and methods used for classification of leg flexion and extension, as well as feature analysis. Results will be shown followed by the discussion and conclusion. 2 MATERIALS AND METHODS 2.1 A brain-controlled system A BCI is an artificial intelligence system which collects and analyzes biological neural signals. To date, specific patterns in brain signals can be identified with a pattern recognition architecture, composed of five consecutive stages: signal acquisition, feature engineering, classification/regression, action generation and feedback interface, as shown in Figure 1. Signal acquisition captures the brain signal using neuroimaging techniques, e.g., EEG in the current work. Other processes, e.g., notch filtering, band pass filtering and artifact rejection, might also be integrated in this module. Feature engineering include feature extraction and feature selection. Typical features include signal envelope or amplitude in the time domain, power or phase lag in the frequency domain, and Mahalanobis distance. Feature selection is performed to overcome the curse of dimensionality, as well as to find the subject-specific patterns corresponding to the mental tasks. Classification/regression are the main part as a decoder, which are typical supervised machine learning techniques for discrete and continuous output towards various applications. Actions are generated based on the decision made from the decoder, and then transferred to devices such as wheelchair, speller and even drone. A certain set of feedback, e.g., visual, tactile and proprioceptive, is provided to inspect the real-time performance of the user. In this work, the five modules were merged to translate the brain signals to control the robot, and visual feedback was presented for online testing. 2.2 Experimental protocol Five subjects (three females, mean age 24.68±1.04) participated in the experiments. They were all naive BCI users with normal or corrected-to-normal vision. None of them reported any known neurological or psychiatric disease. The experimental protocol was approved by the local research ethical committee and all participants gave their informed consent. Each subject did two sessions of recordings: offline for training and online for testing. Each session consists of five successive runs with a rest period in between. A customized gait training exoskeleton called the legopress (by LSRO, EPFL, Lausanne, Switzerland) was implemented in

3 Figure 3: BioSemi-32 layout, and other configurations can be found at Figure 4: Customized GUI to show the EEG signals in real time. this study. This robot is a simple and cost-effective device for gait retraining. More importantly, it can simulate the real movement of leg extension and flexion as a natural approach for inducing sensory feedback. In each run, trials of extension and flexion were randomized and balanced and a total of 60 trials were performed (around 10 min). This yielded 300 trials for one session and each recording was around 1.5 h including experiment preparation and robot familiarization. During the experiment, the subject was seated in the lego- Press with a monitor in front of the robot. The visual cues were presented with a customized Python script ( and synchronized with the lower-level routines. As shown in Figure 2, visual bars pointing upwards indicated leg extension MI and downwards for leg flexion MI. The preparation period in each trial was 2 s followed by MI period lasting 4 s for training sessions. In testing sessions, the MI period depended on the user performance and the time out was 5 s. The interval between these two sessions for each subject was around 2 weeks in order to simulate the realistic rehabilitation scenario. 2.3 Signal acquisition and preprocessing EEG signals were recorded from 32 channels according to the international 10/20 system using a Biosemi Active T- wo system. The sampling frequency was set to 2048 Hz. Ground was replaced by the Driven Right Leg (DRL) passive electrode and all signals were referenced to the Common Mode Sense (CMS) active electrode placed 1 cm to the left of POz. The layout of biosemi32 cap can be seen in Figure 3. Before recording, the signals were inspected with a GUI (eegviewer), as shown in Figure 4. Between runs, the waveforms were also checked by the experimenter to ensure the quality of signal. For signal preprocessing, peripheral channels were first removed, since they were prone to body movement and physiological electrical artifact, e.g., electromyogram (EMG) and electrooculogram (EOG). We kept 17 channels: F3, FC1, FC5, C3, CP1, CP5, P3, Pz, P4, CP6, CP2, C4, FC6, Figure 5: Sample-based and trial-based classification error from offline and online sessions, respectively. FC2, F4, Fz, and Cz to reduce the signal contamination. In order to increase the Signal-to-Noise Ratio (SNR), a s- mall Laplacian spatial filter was used by removing the local activity of neighboring electrodes as formula (1). N k e i (t) =e i (t) 1/N k e j (t) (1) j=1 where e i (t) was the value of ith channel and N k was the number of considered nearest neighbor channels. The DC component of the spatially filtered signal was further removed as follows, e i (t) =e i (t) e i (t T,t) (2) where e i (t T,t) was the mean value from time t T to t in the ith channel, and T is the length of time window. 2.4 Feature selection As there were power changes in the EEG signals during MI tasks, we extracted the features in the frequency domain. The Power Spectral Density (PSD) was calculated over 1 s windows (T =1)overlapped for 0.75 s using Hamming windows of 500 ms for all the channels, with 62.5 ms increments. The PSD was estimated in the range of 4 to 48

4 Figure 6: Classification result of one subject across the validation sets for session 1. Hz with a resolution of 1 Hz. Therefore, the number of features was 765, given the remaining EEG channels (17) and the frequency components (45). We did a log transformation on the features in order to fit the normality assumption for the following analysis. After feature extraction, we performed feature selection using Canonical Variate Analysis (CVA), also known as multivariate discriminant analysis [15]. CVA was used to extract Canonical Discriminant Spatial Patterns (CDSPs) on the original feature space, e.g., powers after log transformation. Given the two mental tasks, i.e., leg flexion and extension, CVA maximized the separability between the t- wo classes. We kept 10 features for further classification. 2.5 Classification We used linear discriminant analysis (LDA) to classify the two mental tasks. LDA is a simple and well-known classification approach based on normal distribution assumption and Bayes rule. To calculate the sample-based accuracy, we relied on a 5-fold cross validation. It is worth noting that the feature selection aforementioned was only performed on training data. Furthermore, chronological order of the data was maintained in order to yield a better,and less optimistic, estimation of accuracy. In addition to sample-based classification, trial-based performance, also known as BCI command accuracy, was e- valuated based on the posterior probabilities from the sliding windows. An evidence accumulation shown in formula (3) was implemented as an exponential smoothing filter, in order to eliminate effects from the outliers. p t = α p t 1 +(1 α) p t (3) where p t was the integrated probability at time t and α was the smoothing factor. As α defined a trade-off between the command speed and confidence, we empirically set it to 0.95 in the current study. When the integrated probability reached the user-specific threshold, an action was generated and the command would send to the robot to execute either flexion or extension movement accordingly. Finally, we reported the results in terms of both accuracy, and Figure 7: Classification result of one subject across the validation sets for session 2. Table 1: Average AUC for each subjects in the two sessions subject ID s1 s2 s3 s4 s5 session session sensitivity-specificity in the receiver operating characteristics (ROC) space. 2.6 Feature analysis A post-processing of feature analysis was conducted to estimate the discriminant power (DP) for the two mental tasks. Given the number of features as c and number of classes as k, the feature matrix T was defined as: T = k T i (4) i=1 The normalized eigenvalues were defined as λ u γ u = k 1 u=1 λ (5) u where λ u was the eigenvalues of the CDSP matrix. DP of single feature can be computed as formula (6). N k 1 i=1 u=1 DP e = 100 γi ut i 2 eu N c k 1 (6) i=1 e=1 u=1 γi u ti 2 eu where N was the number of runs, and t i eu was the element from feature matrix with e =1,..., c. The calculation of DP took the stability of features among data chunks into consideration, since it penalized the features not consistent during runs. We reported the DP with both topographic and heat maps between MI of leg extension and flexion. Furthermore, DP between focus and rest was shown to estimate the brain patterns during the mental tasks. 3 RESULTS 3.1 classification Figure 5 depicts the classification error from both sessions. As expected, the BCI command accuracy was significantly better than sample-based performance (two-sample

5 Figure 8: Heat maps to shown the DP of a typical subject. Peripheral channels were removed for the analysis. Figure 9: Topographic maps to shown the DP of a typical subject. The DP of peripheral channels was set to 0. t-test, p < 0.01 and p = 0.017, respectively). No significant difference was found (two-sample t-test, p > 0.05) between these two sessions. The average BCI command accuracy of offline and online sessions were 0.63±0.05 and 0.67±0.07, respectively. Furthermore, chance level was calculated based on binomial distribution [16], which yielded an accuracy of The performance of all subjects was significantly better than the chance level, with an accuracy of 0.76 in the best case. Results of binaryclassification with ROC curves are shown in Figure 6 and 7. Each curve represents the performance of one subject averaged across 5-fold cross validation. The chance level was calculated by shuffling the labels for the training data and performing 1000 times 5-fold cross validation, as shown in dotted red lines. The mean Area Under the Curve (AUC) values across the cross validation are illustrated in Table 1. The mean AUC of the two sessions were 0.73±0.02 and 0.77±0.06, respectively. 3.2 Feature analysis Based on CVA, DP for the classification was estimated for each subject. Discriminative features of a typical subject are shown in both heat maps and topoplots, as displayed in Figure 8 and 9, respectively. The DP is distributed at primary motor cortex, mainly in the mu band. C3 and C4 are the discriminant channels for the classification. Features, i.e., channel and frequency component pairs, selected by CVA from offline and online sessions are shown in Table 2 and Table 3, respectively. Although the DP is highly subjectspecific, mu and beta bands located at frontal and central brain areas are the most frequently selected features. Table 2: Features selected by CVA for all subjects in the offline session. s1 s2 s3 s4 s5 C3/11 Hz Cz/27 Hz C3/30 Hz C3/11 Hz FC6/42 Hz C3/12 Hz FC5/42 Hz C3/29 Hz C4/10 Hz FC6/41 Hz C4/11 Hz FC1/41 Hz F4/30 Hz C4/23 Hz Fz/21 Hz C4/12 Hz FC5/41 Hz FC6/18 Hz Fz/11 Hz F3/19 Hz C3/10 Hz CP6/41 Hz F4/39 Hz C3/10 Hz F3/35 Hz C3/23 Hz CP6/19 Hz F4/31 Hz Fz/23 Hz CP6/07 Hz CP2/12 Hz CP6/24 Hz F3/30 Hz Fz/22 Hz FC1/19 Hz C3/13 Hz CP6/40 Hz C3/31 Hz CP1/11 Hz P4/19 Hz C4/10 Hz FC5/39 Hz CP5/39 Hz Fz/12 Hz C3/11 Hz CP1/11 Hz FC5/40 Hz CP1/30 Hz CP1/12 Hz FC1/34 Hz Table 3: Features selected by CVA for all subjects in the online session. s1 s2 s3 s4 s5 CP6/29 Hz Cz/23 Hz FC6/28 Hz P4/07 Hz F3/29 Hz FC5/30 Hz Cz/20 Hz FC6/29 Hz C4/10 Hz FC5/30 Hz CP6/30 Hz Cz/21 Hz FC6/30 Hz FC5/15 Hz F3/30 Hz FC5/28 Hz Fz/04 Hz FC6/27 Hz CP5/10 Hz F3/26 Hz FC5/29 Hz Cz/24 Hz FC6/26 Hz C4/11 Hz F3/22 Hz CP6/28 Hz Cz/22 Hz FC6/25 Hz CP6/07 Hz F3/25 Hz FC5/27 Hz F3/17 Hz CP6/30 Hz CP6/10 Hz FC6/28 Hz FC5/26 Hz Cz/25 Hz FC6/24 Hz C3/07 Hz FC6/27 Hz FC6/29 Hz Fz/05 Hz CP6/28 Hz Fz7/07 Hz FC5/22 Hz FC5/24 Hz CP5/14 Hz CP6/29 Hz P4/22 Hz F3/21 Hz Furthermore, the DP of focus vs. rest was estimated, to compare with the leg extension vs. flexion, as shown in Figure 10. The channels selected in the leg extension vs. flexion are FC5 and CP6, located in Brodmann Area 40 and 44, with the function of motor planning and somatosensory integration. On the other hand, the features of MI vs. rest are located in the the primary motor cortex, especially at Cz, which is in consistent with previous works of lowerlimb MI [7]. 4 Discussion Brain-actuated robotics usually exploit spontaneous EEG correlates, e.g., MI, as control signals. To design a BCI for gait rehabilitation, directly performing lower-limb MI, e.g., leg extension and flexion is a natural and intuitive strategy. In this study, we built classifiers and executed experiments in a closed loop to tackle this issue. The goal is to evaluate whether we can find significant difference between MI of leg extension and flexion, as well as discriminative features employed in the mental tasks. Although the DP is highly subject-specific and session-specific, we showed promising results with both sample-based and BCI command accuracy. The results reveal that it is possible to distinguish MI of leg extension and flexion, based on the proposed pattern recognition framework. In the current work, CVA was used for feature selection and LDA classifier was built to differentiate the mental tasks. We assume that the log transformed PSD follows a Gaussian distribution and the two classes are linearly separable. However, the brain signal might be highly nonlinearity and more sophisticated machine learning techniques, e.g., dimensionality reduction and classification, could be intro-

6 (a) Figure 10: Topographic maps to compare the DP between leg extension vs. flexion (a) and focus vs. rest (b). (b) duced to enhance the capability of the model. Besides, heathy subjects participated in the experiments as a proof of concept. Our ultimate users would be motor disabled subjects, e.g., paraplegic patients, to help them with the proposed brain-controlled gait trainer for therapy. The main limitation of a MI-based BCI is the long period required for training. In this study, each subject was recorded with two sessions with visual feedback provided during online testings. The users learned fast on the modulation of their brain patterns for the mental tasks, although none of them had any experience with MI. Further improvement might be achieved with more experimental sessions. Still, a large number of subjects, including end-users, should be incorporated in further work to reach a stronger conclusion [17]. In addition to visual feedback, multiple modalities of feedback, e.g., proprioceptive or functional electrical stimulation (FES) would be introduced and compared in further work. ACKNOWLEDGEMENT We would like to thank Dr. Mohamed Bouri from LSRO, EPFL, Switzerland, for modification on the gait training exoskeleton. We also appreciate all subjects who participated in the recordings. REFERENCES [1] J. R. Wolpaw, N. Birbaumer, D. J. McFarland, G. P- furtscheller, and T. M. Vaughan, Brain-computer interfaces for communication and control, Clinical neurophysiology 113, no. 6: , [2] J. d. R. Millán, F. Galán, D. Vanhooydonck, E. Lew, J. Philips, and M. Nuttin, Asynchronous non-invasive brain-actuated control of an intelligent wheelchair, Proc. 29th Annual Intl. Conf, [3] K. Lee, D. Liu, L. Perroud, R. Chavarriaga, and J. D. R. Millán, Endogenous control of powered lower-limb exoskeleton, Wearable Robotics: Challenges and Trends, pp , [4] J. Meng, S. Zhang, A. Bekyo, J. Olsoe, B. Baxter, and B. He, Noninvasive Electroencephalogram Based Control of a Robotic Arm for Reach and Grasp Tasks, Scientific Reports 6: 38565, [5] X. Gao, D. Xu, M. Cheng, and S. Gao, A BCI-based environmental controller for the motion disabled, IEEE Transactions on Neural Systems and Rehabilitation Engineering, 11: , [6] I. Iturrate, J. M. Antelis, A. Kubler, and J. Minguez, A noninvasive brain-actuated wheelchair based on a P300 neurophysiological protocol and automated navigation, IEEE Transactions on Robotics 25, no. 3: , [7] G. Pfurtscheller, F. L. Da Silva, Event-related EEG/MEG synchronization and desynchronization: basic principles, Clinical Neurophysiology, 110: , [8] G. Dornhege, J. d. R. Millán, T. Hinterberger, D. McFarland, and K. Muller, Toward brain-computer interfacing, MIT press, [9] A. R. Donati, S. Shokur, E. Morya, D. S. Campos, F. L. Brasil, Long-term training with a brain-machine interfacebased gait protocol induces partial neurological recovery in paraplegic patients, Scientific Reports 6, [10] K. Lee, D. Liu, L. Perroud, R. Chavarriaga, and J. D. R. Millán, A brain-controlled exoskeleton with cascaded event-related desynchronization classifiers, Robotics and Autonomous Systems, [11] W. Penfield and E. Boldrey, Somatic motor and sensory representation in the cerebral cortex of man as studied by electrical stimulation, Brain: A journal of neurology, [12] T. C. Emmerling, J. Zimmermann, B. Sorger, M. A. Frost, and R. Goebel, Decoding the direction of imagined visual motion using 7 T ultra-high field fmri, Neuroimage, 125, 61-73, [13] T. C. Bulea, S. Prasad, A. Kilicarslan, and J. L. Contrerasvidal, Sitting and standing intention can be decoded from s- calp EEG recorded prior to movement execution, Frontiers in Neuroscience, 8(6), [14] I. K. Niazi, N. Jiang, O. Tiberghien, J. F. Nielsen, K. Dremstrup, and D. Farina, Detection of movement intention from single-trial movement-related cortical potentials, Journal of neural engineering, 8(6): , [15] F. Galán, P. W. Ferrez, F. Oliva, J. Guardia, and J. d. R. Millán, Feature extraction for multi-class BCI using canonical variates analysis, IEEE International Symposiumon on Intelligent Signal Processing, pp. 1-6, [16] M. Gernot, R. Scherer, C. Brunner, R. Leeb, and G. P- furtscheller, Better than random: A closer look on BCI results, International Journal of Bioelectromagnetism 10, pp , [17] R. Leeb, S. Perdikis, L. Tonin, A. Biasiucci, M. Tavella, M. Creatura, and J. d. R. Millán, Transferring brainccomputer interfaces beyond the laboratory: successful application control for motor-disabled users, Artificial Intelligence in Medicine 59: , 2013.