Emotion Classification along Valence Axis Using Averaged ERP Signals

Transcription

1 Emotion Classification along Valence Axis Using Averaged ERP Signals [1] Mandeep Singh, [2] Mooninder Singh, [3] Ankita Sandel [1, 2, 3]Department of Electrical & Instrumentation Engineering, Thapar University, Patiala, INDIA Abstract Emotions are combination of physiological expression, biological reactions and mental states. Emotion is considered to be the important feature of humans. By emotions human can communicate, express his view, and share his feelings. Most of the people feel surprise, fear, happiness, sadness, disgust and anger many times, these are called basic emotions.eeg is the best way to analyze the emotions. This paper describes the extraction of averaged event related potential features from the processed EEG signals. The average ERP features selected for emotion classification are P100, N100, P200, N200, P300 and N300 and their latencies by using the images provided by International Affective Picture System for emotion elicitation. The features are collected from 3 subjects for five electrodes namely Cz, fp1, fp2, p3 and p4. The attributes thus obtained are used as attributes for Support Vector Machine to classify emotion into two classes namely Low Valence High Arousal (LVHA) and High Valence High Arousal (HVHA). An accuracy of 100% has been achieved for Cz electrode and combination of p3-p4 and 91.67% accuracy has been achieved on combination of fp1-fp2 using 5 th order SVM. Using averaged ERP, the accuracy of emotion classification along valence axis improves and order of SVM decrease as compared to unaveraged ERP. 1. Introduction Emotion is defined A natural inborn state of mind deriving from one circumstance, mood or relationship with others. Emotions can be expressed either verbally through emotion vocabulary, or by expressing non-verbal ques such as intonation of voice, facial expressions and gestures [1]. Emotions are the best understood as the subjective representation of feelings. Emotion is a mental and physiological state associated with a wide variety of feelings, behavior and thoughts. It is a subjective experience depending on individual to individual basis and their past experience [2]. Emotions are judgments about the extent the current situation meet our goal. Happiness is the evaluation that your goals are being satisfied, as when winning the lottery solves your financial problems and being asked out holds the promise of satisfying your romantic needs. Similarly, sadness is the evaluation that your goals are not being satisfied, and anger is the judgment aimed at whatever is blocking the accomplishment of your goals. According to second theory, emotions are perceptions of changes in our body such as hate rate, breathing rate, and perspiration and hormone levels [3]. Number of Approaches have been tested for quantifying and analyzing human emotions, facial images [4], speech signals [5] and auto numerous nervous signals [6]. EEG is measured as a dominant tool in neuroscience research and clinical diagnosis because it permits to record from surface of the scalp, electrical signals arising from the brain [7] [8]. Fig.1 Basic Emotions [9]

2 Frantzidis.C et al. (2010) discussed the classification of neurophysiological data obtained through EEG on 56(28 males and 28 females) volunteers by using pictures selected from the International Affective Picture System [10], into four emotional states. The classification of the emotional states high valence high arousal HVHA, Low valence high arousal LVHA, high valence low arousal HVLA, low valence low arousal LVLA were achieved by using the Mahalanobis distance (MD) based classifier and support vector machines (SVMs). EEG was recorded with Ag/AgCl electrodes placed at 19 positions on the scalp as per the International System. The sampling rate was chosen as 500 Hz. Classification was done by Mahalanobis distance (MD) and Support Vector Machine (SVM). The overall accuracy achieved for both the classes were 79.5% for MD and 81.3% for SVM [11]. Mandeep Singh et al. (2013) took the enterface data provided by Savran et al. (2006) [12] [13] [14] [15] for 3 participants and performed classification along valence axis using naive bayes classifier and extracted the ERP feature. The overall accuracy achieved for ERP feature was 56% and by changing the feature to STFT, accuracy obtained was 51%, similarly with the PSD feature, accuracy was 56% and after combining all the three features accuracy obtained was 64% which was comparatively high among all [2], and by using Artificial Neural Network, the overall accuracy achieved for 3 participants by using Artificial Neural Network was 76.59% [16]. Frantzidis.C et al. (2008) used physiological signals from skin conductance and central nervous system and used Neural Network classifier for classification along arousal axis. The classification results obtained for emotion state are 80% for joy, 100% for fear, 80% for happiness and 70% for melancholy [17]. Hosseini, Seyyed Abed et al. (2010) described the emotion classification into two states. The features were extracted using Higher Order Spectra methodology, Using Genetic Algorithm for feature selection. SVM as a classifier gave average accuracy of 82% for the two emotional stress states [18]. 2. Data Collection EEG data is acquired from three subjects for the classification of emotions. All the subjects are male, right handed and physically fit and has no health issues. BIOPAC system is used as hardware for capturing brain signals for emotion classification. The system comprised of EEG amplifiers, EEG gel used in electrode, earlobes attached to the subject. EEG cap has 20 electrodes that could be used in bipolar or unipolar and placed on the cap using international system. EEG gel is used in the electrodes so that contact could be made between the electrodes and the scalp. The EEG gel is filled into the electrodes with the help of a syringe. The level of the impedance is maintained below 10 KOhm. Only data of five electrodes are taken. Five electrodes chosen are Cz, p3, p4, f3 and f4.cz is the central electrode, p3 and p4 are the parietal electrodes and fp1 and fp2 are the frontal electrodes. Stimulus to the subjects for evoking emotions is provided by using images from the IAPS system provided by NIHM Centre of University of Florida [10]. The images are of different types such as snake, nature, child, mutilation, grave etc. The subject is allowed to wear EEG cap over the head. EEG gel is filled on those electrodes whose data is to be captured. The subject is asked to sit properly while capturing data. The subject is not allowed to move, yawn, to blink his eyes, or to make any other movements. For evoking an emotion along the valence axis, an image corresponding to low valence class is shown to the subject for a period of 1second followed by a white cross with a black background for a period of 1.5 seconds to normalize the subject. The next image belonging to another class namely high valence followed the cross image for a period of 1 second. A total of 80 images are shown in this manner to each subject and his EEG is acquired for recognition of emotion into two classes. The software used to capture EEG signals is the Acknowledge 4.2 software [19]. Duration of the stimulus which is shown to the subject is less than 5 minutes. The images are of low valence low arousal, low valence high arousal [20] [21]. 3. Hardware There can be many ways to capture EEG signals from a human being. One method being the fnirs (functional Near Infrared Spectroscopy) sensor which is used to capture frontal brain activity and to capture rest of the brain activity EEG sensor is used and rest of the sensors for capturing body processes. Bio semi Active 2 acquisition system was used by Savran et al. (2006) [12]. The hardware used in this acquisition is BIOPAC MP150 system. The MP System is a computer-based data acquisition system that is used to perform many of the same functions such as chart recorder, other data viewing device. The MP data acquisition unit (MP150 or MP100) is the heart of the MP System. The MP unit takes incoming signals and converts them into digital signals that can be processed with the computer. Data collection generally involves taking incoming signals (usually analog) and sending them to the computer, where they are displayed on the screen and stored in the computer s memory. These signals can then be stored for future examination, much as a word processor stores a document or a statistics saves a data file. Graphical and numerical representations of the data can also be produced for use with other programs [22].

3 4. Signal Conditioning After the acquisition of EEG signals, signal conditioning is very important part which should be performed in order to get better results. The EEG signals may contain some interference such as noise and artifacts and this interference should be removed to get better results. Acqknowledge software is very helpful while applying filter operations [19]. Filtered operations present in acqknowledge software are digital IIR filters, adaptive filters, comb band stop filter The filters used to remove noise and artifacts are low pass filter with bandwidth of 40 Hz, second filter used is high pass filter of 0.5Hz and last is the comb band stop filter with frequency of 50Hz. Fig.2 shows the filtered EEG signals. Fig.2 Filtered EEG Signals 5. Feature Extraction Acqknowledge software which is used to capture EEG signals is also used to extract features form it. ERP feature is being extracted from the signals. Values of P100, N100, P200, N200, P300, N300 and their latencies are being extracted.p100 is the positive ongoing peak with in a time limit of ms also called P1 and similarly N100 or N1 is the negative ongoing peak within time limit of ms. P200 is a second positive peak observed about 200ms, varying from 180 to 220ms. N200 in particular is a negative-going wave that also peaks from ms P300 is a positive peak observed at 300 ms, varying between 280 and 320ms.The features extracted for P100 and N100 is between the time limit of ms and for P200-N200 the time limit is ms and similarly for P300- N300 value the time limit range is ms. 6. Averaging ERP Total of 80 EEG signals are taken after evoking emotions on valence axis by showing the relevant images. Of these 80 signals, 40 signals are acquired by showing low valence emotion images and 40 signals are acquired using high valence emotion images. Each of 40 signals are converted into single signal and then taking average of these signals. After that extraction of ERP feature is done. All these features are extracted using Acqknowledge software. The reason of taking average is to remove random signals in time domain. 7. Acqknowledge Software Data is acquired by using EEG Cap which has 20 electrodes used in bipolar embedded in it. The placement of electrodes on EEG Cap is done through system of placement of electrodes. The electrodes are attached to BIOPAC MP150 amplifiers. The ear lobe or the mastoid is used as a reference electrode. The EEG signals coming from the electrodes are acquired by the Acqknowledge 4.2 software which is provided by BIOPAC MP Classification A classifier is a system which divides the data into different classes and gives relationship between the extracted feature and emotion related to that part of EEG data captured as EEG signal does not depict any emotion. To know which emotions are present in the signal, classification of the signal is done. Classification is done with the help of Support Vector Machine in two classes mainly low valence and high valence. Support Vector Machine is used as a

4 classifier.svm is responsible for classification between two classes that is between low valence and high valence. The extracted features that are event related potential whose values such as P100, N100, P200, N200, P300 and N300 with their corresponding latencies are fed to the classifier for classification. 9. Results The average ERP is taken of 5 electrodes in combination. Five electrodes are Cz, P3, P4, Fp1 and Fp2 and out of which Cz is the electrode which is being analyzed and p3-p4 and fp1-fp2 are taken in combination and being analyzed depending upon the values of the ERP feature. Average ERP is being analyzed on the basis of polynomial order.lower order polynomial is taken. The results are shown from Fig.1 to Fig.6 Fig.1 Accuracy of 3 electrodes for 3 participants for order 1 Fig.1 shows that while taking ERP and their corresponding latencies as attributes to SVM, an accuracy of 75% is obtained on Cz electrode, 70.83% on fp1-fp2 electrodes and 62.5% accuracy is achieved on p3-p4 electrodes for subject 1. For subject 2, 58.33% accuracy is achieved on Cz electrode, 50% on fp1-fp2 electrodes and 58.33% is obtained on p3-p4 electrodes and similarly, 58.33% accuracy is achieved on Cz electrode, 66.67% is achieved on fp1-fp2 electrode and 50% on p3-p4. Fig.2 Accuracy of 3 electrodes for 3 participants for order 2

5 Similarly the charts shown in figures 2, 3, 4, 5 and 6 shows the accuracies obtained for the three subjects on ERP features and their corresponding latencies when different polynomial orders are taken for emotion classification. Fig.3 Accuracy of 3 electrodes for 3 participants for order 3 Fig.3 shows that while taking ERP and their corresponding latencies as attributes to SVM, an accuracy of 91.67% is obtained on Cz electrode, 70.83% on fp1-fp2 electrodes and 66.67% accuracy is achieved on p3-p4 electrodes for subject 1. For subject 2, 66.67% accuracy is achieved on Cz electrode, 62.5% on fp1-fp2 electrodes and 50% is obtained on p3-p4 electrodes and similarly, 91.67% accuracy is achieved on Cz electrode, 66.67% is achieved on fp1-fp2 electrode and 87.5% on p3-p4. Fig.4 Accuracy of 3 electrodes for 3 participants for order 4 On classification with a polynomial order of 4, lower accuracies are obtained on fp1-fp2 electrodes of subject 2 and higher accuracy related to fp1-fp2 electrodes are 79.17% for subject 1 and 3.

6 Fig.5 Accuracy of 3 electrodes for 3 participants for order 5 Good results are obtained on almost all the electrodes except p3-p4 for subject 3.The accuracy as high as 100% is obtained on Cz electrode. Fig. 6 Accuracy of 3 electrodes for 3 participants for order 6 At this polynomial order, the accuracy remained high on all five electrodes for three subjects under observation. To perform the subject wise analysis for classification of emotions, the figures 7, 8 and 9 are shown herein for different polynomial orders.

7 Fig. 7 Accuracy of subject 1 for 3 electrodes Fig.7 shows that best results are obtained on Cz electrode at polynomial order 5 and 6 for subject 1. Fig.8 Accuracy of subject 2 for 3 electrodes Lower accuracies are obtained for subject 2 on polynomial order 5.Cz electrode again tops the chart by displaying higher accuracies at almost both the polynomial order that are 5 and 6. Fig.9 Accuracy of subject 3 for 3 electrodes

8 Fig.9 also asserts that when using polynomial order 5 and 6, better results are obtained for subject 3 with p3-p4 electrode toppings the table by displaying higher accuracies 8. Conclusion In this work data is acquired for emotion classification along valence axis that is positive valence and negative valence. The electrodes are chosen Cz, fp1, fp2 p3 and p4. Data is acquired at the sampling rate of 500Hz. The acquired data for 40 similar EEG signals evoked by showing high and low valence images is averaged to remove random signals in time domain. The data collected is used for extracting ERP features that are P100, N100, P200, N200, P300 and N300 with their corresponding latencies and after that features are classified using SVM. The overall accuracy obtained on Cz is 100% for subject 1 and subject 3, 91.67% accuracy is obtained on fp1-fp2 for subject 1 and 100% accuracy is achieved on p3-p4 electrode for subject 3. It has been concluded that accuracy is proportional to polynomial order. Lower order gives lower accuracy and after increasing the order, accuracy automatically increases. Averaging of the signals helps in achieving better accuracy with much lesser order as compared to that done without averaging of the signals. The highest accuracy for all the five electrodes in combination is shown in Fig.10. Fig.10 Accuracy of all 3 electrodes for 3 Subjects Reference [1] Mandeep Singh, Mooninder Singh, Surabhi Gangwar, Emotion Recognition Using Electroencephalography (EEG): A Review, vol. 7, no.1, pp. 1-5, Dec [2] Mandeep Singh, Mooninder Singh, Nikhil Singhal, Emotion Recognition along Valence Axis Using Naïve bayes Classifier, vol. 7, no.7, pp.51-55, Dec [3]What are emotions, available at [4] B. Lee, J. Chun, and P. Park, Classification of facial expression using svm for emotion care service system, in Software Engineering, Artificial Intelligence, Networking, and Parallel/Distributed Computing, SNPD 08. Ninth ACIS International Conference, pp. 8 12, Aug 2008 [5] S. Wang, X. Ling, F. Zhang, and J. Tong, Speech emotion recognition based on principal component analysis and back propagation neural network, in Measuring Technology and Mechatronics Automation (ICMTMA), International Conference on, vol. 3, pp , March 2010 [6] K. Kim, S. Bang, and S. Kim, Emotion recognition system using short-term monitoring of physiological signals, Medical and Biological Engineering and Computing, vol. 42, pp , [7] Robert Horlings, Emotion recognition using brain activity, Man-Machine Interaction Group TU Delft 27 March [8] Emotions, available at [9] Mandeep Singh, Sunpreet Kaur, Epilepsy Detection using EEG: An Overview, International Journal of Information Technology & Knowledge Management, vol. 6, no1, pp.3-5, 2012.

9 [10] M. M, Bradley & Lang, P. J., The International Affective Picture System (IAPS) in the Study of Emotion and Attention, Handbook of Emotion Elicitation and Assessment, Oxford University Press, [11] C. A. Frantzidis, C. Bratsas, C. L Papadelis, E. Konstantinidis, C. Pappas, & Bamidis, P.D Toward emotion aware computing: An integrated approach using multichannel neurophysiological recordings and affective visual stimuli. IEEE Transactions on Information Technology In Biomedicine vol.14 no.3, pp , 2010 [12] A. Savran, K. Ciftci, G. Chanel, J.C. Mota, L.H. Viet, B. Sankur, L. Akarun, A. Caplier, and M. Rombaut Emotion detection in the loop from brain signals and facial images.iee Computer society vol.2 issue1 pp ,Doi [13] Mandeep Singh, Mooninder Singh and Surbhi Gangwar, Feature Extraction from EEG for Emotion Classification, International Journal of Information Technology & Knowledge Management, vol.7, no.1, Dec [14] Mandeep Singh, Mooninder Singh and Nikhil Singhal, Emotion Quantification Along Valence Axis Using EEG Signals, International Journal of Information Technology & Knowledge Management, vol.7, no.1, Dec [15] Mandeep Singh, Mooninder Singh and Surabhi Gangwar, Arousal Detection Using EEG Signals, International Journal of Information Technology & Knowledge Management, vol.7, no.1 pp.11-15, Dec [16] Mandeep Singh, Mooninder Singh and Nikhil Singhal, ANN Based Emotion Recognition Along Valence Axis Using EEG, International Journal of Information Technology & Knowledge Management, vol.7 no.1, Dec [17] C.A. Frantzidis, C.D Lithari, A.B Vivas, C.L. Papadelis, C. Pappas, & Bamidis Towards emotion aware computing: a study of arousal modulation with multichannel event-related potentials, delta oscillatory activity and skin conductivity responses In Proceedings of BIBE,.pp.1-6, [18] S.A. Hosseini, & M. Khalilzadeh, A Higher order spectra analysis of EEG signals in emotional stress states. IEEE computer society, Second international conference on information technology and computer science.doi ( [19] Acqknowledge 4 software guide, available at [20] Mandeep Singh, Mooninder Singh, Ankita Sandel, Data Acquisition Technique for EEG based Emotion Classification, International Journal of Information Technology & Knowledge Management, vol. 7, 2014 [21] Mandeep Singh, Mooninder Singh, Ankita Sandel, Emotion Classification along Valence Axis using ERP Signals, International Journal of Information Technology & Knowledge Management, vol.7, 2014 [22] MP System Hardware Guide, available at