EEG Features in Mental Tasks Recognition and Neurofeedback

Transcription

1 EEG Features in Mental Tasks Recognition and Neurofeedback Ph.D. Candidate: Wang Qiang Supervisor: Asst. Prof. Olga Sourina Co-Supervisor: Assoc. Prof. Vladimir V. Kulish Division of Information Engineering School of Electrical and Electronic Engineering Nanyang Technological University Institute for Media Innovation Nanyang Technological University 1

2 Outline Background & Motivation & Objective Proposed Algorithms Conclusion & Future Works Demos & Publication List 2

3 EEG: EEG provides wonderful tools for brain state monitoring. High temporal resolution. Tremendous algorithms are available for time series. Successful medical applications. Neurofeedback: Motivation Neurofeedback systems provide visual/audio feedback according to EEG signal. It is useful for brain training. Use neurofeedback to enhance the work performance. Use neurofeedback to treat ADHD patients. 3

4 This project is inter-disciplinary: biosignals medical application serious game pattern recognition cognitive informatics psychology Research Objectives: Research Objective Design an experiment protocol for mental tasks recognition. Study nonlinear model and propose effective EEG features for mental tasks recognition. Propose faster, more accurate algorithms with less EEG channels for mental tasks recognition. Propose neurofeedback strategies. Design and implement 2D and 3D neurofeedback games. Develop a protocol to use neurofeedback game for psychological disorder treatment and optimum concentration level searching. Use proposed concentration level recognition techniques to provide a feedback loop in e-learning system. 4

5 Outline Background & Motivation & Objective Proposed Algorithms Conclusion & Future Works Demos & Publication List 5

6 Related Works Relative power training in EEG based neurofeedback. Theta/Beta training 1. Increase theta band power. Decrease beta band power. Active alpha training 2. Increase alpha band power. Decrease EMG power. 1. T. M. Sokhadze, et al., "EEG biofeedback as a treatment for substance use disorders: Review, rating of efficacy, and recommendations for further research," Applied Psychophysiology Biofeedback, vol. 33, pp. 1-28, S. Hanslmayr, et al., "Increasing individual upper alpha power by neurofeedback improves cognitive performance in human subjects," Applied Psychophysiology Biofeedback, vol. 30, pp. 1-10,

7 EEG Database for Mental Tasks A well-known EEG database for mental tasks classification recorded by Zak Keirn 1 is available. Seven subjects participated the experiment for two session. In each session, subjects performed 5 different mental tasks for 5 trials. Relax Counting Letter composition Multiplication Rotation 1. Z. Keirn, Alternative modes of communication between man and machine, Master s thesis, Electrical Engineering Department, Purdue University,USA,

8 Related Works N. Liang et. al. 1 processed the mental tasks EEG database in Autoregressive features were used. Different classifiers were compared, multi-class SVM classifier can achieve the best accuracy. With multi-class SVM classifier, 52.07% accuracy were reported for multi-class classification. 1. N. Liang, P. Saratchandran, G. Huang, and N. Sundararajan, Classification of mental tasks from eeg signals using extreme learning machine, International Journal of Neural Systems, vol. 16, no. 1, pp ,

9 EEG Signal Processing EEG data were processed according to the following procedure. EEG Signal Segmentation Ocular Artifact Removal Feature Extraction Feature Selection Classification 9

10 EEG Signal Segmentation EEG signals were divided into segments with 512 samples (overlapping with 480 samples). Segment 1 Segment samples

11 Ocular Artifact Removal Ocular artifacts were detected by applying with a fixed-weight leakage normalized stochastic least mean fourth algorithm 1 on EOG channel. Segments contains OAs were discarded. 1. P.Celka, B.Boashash, and P.Colditz, Preprocessing and time-frequency analysis of new born eeg seizures, IEEE Engineering in Medicine and Biology Magazine,vol.20, no.5, pp.30 39,

12 Feature Extraction Six group of features were extracted from each clean segment. Feature Type No. of feature Time cost (ms) Relative Power (PSD) 5 1 Autoregressive (AR) coefficient 6 70 Higher Order Crossing (HOC) Generalized Higuchi Fractal Dimension Spectrum (GHFDS) Entropy Statistical

13 Feature Extraction Generalized fractal dimension spectrum. 13

14 Feature Selection To speed up multi-class svm evaluation, we applied feature selection method before classification. Following features selection schemes were considerate and compared. Random Forests (RF) scheme could achieve the best performance. 14

15 Classification Multi-class SVMs were used as classifier. RBF kernel was applied and C-gamma parameters were selected with grid search procedure. 15

16 Mental tasks classification results when different features were used. Classification Result Statistical features could achieve better accuracy than AR features which were used in N. Zhang s research 1. In their paper, the accuracy is 52.07%. Combine all features could enhance the performance. 1. N. Liang, P. Saratchandran, G. Huang, and N. Sundararajan, Classification of mental tasks from eeg signals using extreme learning machine, International Journal of Neural Systems, vol. 16, no. 1, pp ,

17 Classification Result Benefits of feature selection. 17

18 Classification Result Benefits of feature selection. 18

19 Experiment Setup: Arithmetic Task Experiment EEG recording device 14-channels, Sampling frequency: 128 Hz, A/D resolution: 16-bit. PC for processing data CPU: Intel Core 2 Quad Q9400 (2.66 Hz * 4), RAM: DDR GB, EEG processing software EEG recording : Emotiv Testbench, EEG processing: Numpy. Subjects 10 subjects. 19

20 Arithmetic Task Experiment Data Acquisition Protocol: Session 1 Relaxation Session (Relax, no task to fulfill) Session 2 Arithmetic Session (Working on 3-digit arithmetic problems) 20

21 Classification Result Comparison between different type of EEG features. 21

22 Classification Result Comparison between different type of EEG features. 22

23 EEG channel rank. Classification Result 23

24 EEG channel rank. Classification Result 24

25 Outline Background & Motivation & Objective Proposed Algorithms Conclusion & Future Works Demos & Publication List 25

26 Conclusion A well-known EEG database for mental tasks recognition was also used. Arithmetic task experiment was also designed and carried out to collect the labeled EEG data. Proposed and implemented Fractal Dimension Model Study. Generalized Higuchi Fractal Dimension Spectrum. Proposed and implemented Mental tasks recognition algorithms. Statistical features could achieve the best accuracy (55.23% ). Combine all features could enhance the accuracy (59.82%). With random forests feature selection method, the no. of features used in classification can be reduced to 77 and the classification can be maintained (60.41%). (F8, F3, AF3, O2) channels are important for arithmetic task classification. Proposed and implemented neurofeedback games based on novel EEG features. 26

27 Future Works Parallelize the feature extraction step with MapReduce Model. Develop real-time mental tasks recognition application based on Hadoop framework. Design neurofeedback novel algorithm and compare the working performance enhancement with alpha train neurofeedback. 27

28 Outline Background & Motivation & Objective Proposed Algorithms Conclusion & Future Works Demos & Publication List 28

29 Blooby Demo Real-time EEG monitoring tool. Demonstrate EEG Properties on 3D models. Support real-time mode and playback mode. Support interactive operation. 3 type of indicators. 29

30 Neurofeedback Demos Neurofeedback games. 30

31 Book Section: O. Sourina, Q. Wang, Y. Liu, M. K. Nguyen, EEG-enabled Human-Computer Interaction and Applications, in Towards Practical Brain-Computer Interfaces, B. Allison, etc., Springer, in press, 2011 Journal Papers: Publication List Sourina, O., Wang, Q., Liu, Y.,, Nguyen, M. K., Fractal-based Brain State Recognition from EEG in Human Computer Interaction, Communications in Computer and Information Science, In Press Wang, Q., Sourina, O., Nguyen, M. K., Fractal dimension based neurofeedback in serious games, Visual Computer, Vol.27, No. 4, pp Sourina, O., Wang, Q., Nguyen, M. K., EEG-based "Serious" games and monitoring tools for pain management, Studies in Health Technology and Informatics, Vol.163, pp Sourina, O., Liu, Y., Wang, Q., Nguyen, M. K., EEG-based personalized digital experience, Lecture Notes in Computer Science, Vol.6766, pp Conference Papers: Wang, Q., Sourina, O., Nguyen, M. K., EEG-based "Serious" Games Design for Medical Applications, Proc Int. Conf. on Cyberworlds, 2010, pp Sourina, O., Wang, Q., Liu, Y.,, Nguyen, M. K., A real-time fractal-based brain state recognition from EEG and its applicationse, Proc Biosignals, 2011, pp

32 Q & A 32

33 Feature Extraction Generalized fractal dimension spectrum. 33

34 Statistical features 1. Feature Extraction 1. R. Picard, E. Vyzas, and J. Healey, Toward machine emotional intelligence: Analysis of affective physiological state, IEEE Transactionson Pattern Analysis and Machine Intelligence, vol. 23, no. 10, pp ,

35 Relative Power features 1. Feature Extraction 1. S. Sanei and J. A. Chambers, EEG Signal Processing. San Francisco: WILEY,

36 Autoregressive coefficients. Feature Extraction AR(6) model is used to model EEG segments. 1. N. Liang, P. Saratchandran, G. Huang, and N. Sundararajan, Classification of mental tasks from eeg signals using extreme learning machine, International Journal of Neural Systems, vol. 16, no. 1, pp ,

37 Higher order crossing. Feature Extraction Difference operator is defined as: The q order difference operator is defined as: The crossing number is summarize as follow: 1. S. He and B. Kedem, Higher order crossings spectral analysis of an almost periodic random sequence in noise, IEEE Transactionson Information Theory, vol. 35, no. 2, pp ,

38 Feature Extraction Entropy. Entropy could be used as another important quantification feature in nonlinear dynamical analysis of time series which is related to the rate of information production. We calculated three types of entropy which could be applied to short and noisy time series: approximate entropy 1 sample entropy 1 SVD entropy 2 1. J. Richman and J. Moorman, Physiological time-series analysis using approximate and sample entropy, American Journal of Physiology Heart and Circulatory Physiology, vol.278, no.647-6, pp.h2039 H2049, S. Faul, G. Boylan, S. Connolly, W. Marnane, and G. Lightbody, Chaos theory analysis of the new born eeg-is it worth the wait?, pp ,

39 Feature Selection Random forests. Random Forests (RF) method proposed by Breiman 1 was used as the supervised feature selection scheme. This method could deal with the situation when there are many more features than observations. This method also reduces the risk of overfitting L. Breiman, Random forests, Machine Learning, vol. 45, no. 1, pp. 5 32, S. Diaz-Uriarte and R. A. deandres, Gene selection and classification of microarray data using random forest, BMC Bioinformatics, vol. 7, no. 3,

40 Feature Selection Other features selection schemes. LASSO 1 Stability selection 2 F-score 3 All these scheme is implemented by scikit-learning python library R. Tibshirani, Regression shrinkage and selection via the lasso: A retrospective, Journal of the Royal Statistical Society. SeriesB:Statistical Methodology, vol. 73, no.3, pp , N. Meinshausen and P. Buhlmann, Stability selection, Journal of the Royal Statistical Society. SeriesB:Statistical Methodology, vol.72, no.4, pp , Y. Chen and C.Lin, Combining svms with various feature selection strategies, Studies in Fuzziness and Soft Computing, vol. 207, pp , F. Pedregosa, G. Varoquaux, A. Gramfort, V.Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J.Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, Scikit-learn: Machine Learning in Python, Journal of Machine Learning Research, vol. 12, pp ,