EEG Comparison Between Normal and Developmental Disorder in Perception and Imitation of Facial Expressions with the NeuCube

Transcription

1 EEG Comparson Between Normal and Developmental Dsorder n Percepton and Imtaton of Facal Expressons wth the NeuCube Yuma Omor 1(B), Hdeak Kawano 1, Aknor Seo 1, Zohreh Gholam Doborjeh 2, Nkola Kasabov 2, and Maryam Gholam Doborjeh 2 1 Kyushu Insttute of Techonology, Ktakyushu , Japan omor.yuma780@mal.kyutech.jp, kawano@ecs.kyutech.ac.jp 2 Knowledge Engneerng and Dscovery Research Insttute, Auckland Unversty of Technology, Auckland 1142, New Zealand zohreh.gholamdoborjeh@aut.ac.nz Abstract. Ths paper s a feasblty study of usng the NeuCube spkng neural network (SNN) archtecture for modelng EEG bran data related to percevng versus mmckng facal expressons. We collected EEG patterns durng percepton and mtaton of facal expressons for each emoton. Comparng the collected data n percevng and mmckng facal expressons, EEG patterns were very smlar. Ths fact suggests that t seems that there are mrror neurons on facal expresson n the human bran. Recently, some studes have been reported that the mrror neuron system does not work well n the case of subjects wth bran dsorders. In ths study, we calculated dfferences between EEG patterns when we perceved facal expressons and mmckng facal expressons for healthy people and developmental dsorders. Keywords: EEG data SNN Mrror neuron system Developmental dsorders 1 Introducton Facal expresson s a fundamental tool n human communcaton. Understandng the facal expresson effects on a thrd person s of a crucal mportance to develop a comprehensve communcaton. Neuropsychologcal studes reported that communcatons through facal expressons are hghly related to the Mrror Neuron System (MNS). MNS prncple has been ntroduced n 1990s by Rzzolatt when he dscovered smlar areas of the bran became actvated when a monkey performed an acton and when a monkey observed the same acton performed by another [1]. The MNS n human were also confrmed by an experment usng functonal magnetc resonance magng (fmri) data [2]. Dfferent facal expressons of emoton have dfferent effects on the human bran actvty. c Sprnger Internatonal Publshng AG 2017 D. Lu et al. (Eds.): ICONIP 2017, Part IV, LNCS 10637, pp ,

2 EEG Comparson Between Normal and Developmental Dsorder 597 The bran processes of percevng an emotonal facal expresson and mmckng expresson of the same emoton are spato-temporal processes. The analyss of collectng Spato-Temporal Bran Data (STBD) related to these processes could reveal personal characterstcs or abnormaltes that would lead to a better understandng of the bran processes related to the MNS. Ths can be acheved only f the models created from the STBD can capture both spato and temporal components from ths data. Despte of the rch lterature on the problem, such models stll do not exst. Recently, a bran-nspred Spkng Neural Network (SNN) archtecture, called NeuCube [4 6], has been proposed to capture both the tme and the space characterstcs of STBD, such as EEG, fmri, DTI, etc. In contrast to tradtonal statstcal analyss methods that deal wth statc vector-based data, the Neu- Cube has been successfully shown to be a rch platform for STBD mappng, learnng, classfcaton and vsualzaton [7 9]. In ths paper, we examned dfferences n bran actvty patterns between healthy people and developmental dsabltes by calculatng the dfference EEG data of facal expresson task (both percevng and mmckng) n two knds of emotonal faces (anger, happness). The models allow for a detal understandng on the problem. 2 The NeuCube Spkng Neural Network Archtecture The NeuCube archtecture [4] conssts of: an nput encodng module; a 3D recurrent SNN reservor/cube (SNNc); an evolvng SNN classfer. The encodng module converts contnuous data streams nto dscrete spke trans. As one mplementaton, a Threshold Based Representaton (TBR) algorthm s used for encodng. The NeuCube s traned n two learnng stages. The frst stage s unsupervsed learnng based on spke-tmng-dependent synaptc plastcty (STDP) learnng [10] n the SNNc. The STDP learnng s appled to adjust the connecton weghts n the SNNc accordng to the spatotemporal relatons between nput data varables. The second stage s a supervsed learnng that ams at learnng the class nformaton assocated wth each tranng sample. The dynamc evolvng SNNs (desnns) [11] s employed as an output classfer. In ths study, the NeuCube s used for modelng and learnng of the case study EEG data correspondng to dfferent facal expressons. 3 The Case Study STBD: EEG Data Evoked by Facal Expresson The subjects were 11 Japanese adult males, 10 healthy person and 1 development dsabled n the case study of the facal expresson task. As facal stmul, JACFEE collecton [12] was used, consstng of 56 color photographs of 56 dfferent ndvduals. Each ndvdual llustrates one of the two dfferent emotons,.e. anger, happness. The collecton s equally dvded nto male and female populatons (28 males, 28 females).

3 598 Y. Omor et al. Durng the experments, subjects were wearng an EEG headset (Emotve EPOC+) whch conssts of 14 electrodes wth the samplng rate of 128 Hz and the bandwdth s between 0.2 and 45 Hz. The EEG data was recorded whle the subjects were performng two dfferent facal expresson tasks. Durng the frst presentaton, subjects were nstructed to perceve dfferent facal expresson mages shown on a screen, and n the second presentaton they were asked to mmc the facal expresson mages. We used fve patterns of facal expresson mages per emoton n these experments. Each facal expresson mage was exposed for 5 s followed by randomly 5 to 10 s nter stmulus nterval (ISI) as shown n Fg. 1. Fg. 1. The facal expresson-related task: the order of emoton expressons s alternaton of anger and happness. Each subject watched 10 mages durng an experment. 4 Analyss of the Spatotemporal Connectvty n a Traned SNNc of a NeuCube Model A 3D bran-lke SNNc s created to map the Talarach bran template of 1471 spkng neurons [13, 14]. The spato-temporal data of EEG channels were encoded nto spke trans and entered to the SNNc va 14 nput neurons whch spatal locatons n the SNNc correspond to the system locaton of the same channels on the scalp. The SNNc s ntalzed wth the use of the small world connectvty [4]. We nput EEG data obtaned from fve patterns of facal expressons mages nto one SNNc, and created a model for each subject. Table 1 shows that Neu- Cube parameter values used n the smulatons. Table 1. NeuCube parameter values used n the smulatons. Parameter Value TBR 0.5 Small world connectvty dstance 2.5 STDP rate 0.01 Tranng teraton 1 Tranng tme length 0.2

4 EEG Comparson Between Normal and Developmental Dsorder 599 Durng the unsupervsed STDP learnng, the SNNc connectvty evolves wth respect to the spke transmsson between neurons. Stronger neuronal connecton between two neurons means stronger nformaton (spkes) exchanged between them. Table 2 shows the numercal dfferences of EEG data between mtaton and percepton n 2 facal expressons (ANGRY and HAPPY) from each subject. The defntons of the L1-dfference D L1 and the L2-dfference D L2 are shown n Eqs. (1) and (2). D L1 = D L2 = N N w percevng (w percevng w mmckng, (1) w mmckng ) 2, (2) where w represents weght parameter between neurons n SNNc. Table 2. The dfference between facal expressons (percevng and mmckng) of 2 knds of emoton (Angry and Happy). Subject D L1 D L2 ANGRY HAPPY ANGRY HAPPY A (Developmental dsorder) B(healthy) C(healthy) D(healthy) E(healthy) F(healthy) G(healthy) H(healthy) I(healthy) J(healthy) K(healthy) AVG of healthy STDDV of healthy As shown n Table 2, the dfference n the developmental dsorder s hgher than the one n the healthy subjects. Especally, L1-dfference n ANGRY and L2-dfference n HAPPY show a sgnfcant dfference between a developmental dsorder and healthy subjects. Indeed the number of samples n the experment s qute small, but we beleve that ths fact mplcates a possblty to use the dfference between weght connectons learnt by the NeuCube as an ndex to evaluate a knd of socal ablty.

5 600 Y. Omor et al. 5 Concluson In ths paper, we used the NeuCube archtecture of SNN [4] for mappng and learnng of EEG data recorded from subjects when they were performng a facal expresson-related task. From Table 2, t was found that the person wth developmental dsablty has a larger dfference between EEG data of percepton and mtaton than healthy people. Ths fndng can prove the prncple of the mrror neurons n the human bran. Ths s only the frst study n ths respect. Further studes wll requre more subject data to be collected for a more models developed before the proposed method s used for cogntve studes and medcal practce. References 1. Gallese, V., Fadga, L., Fogass, L., Rzzolatt, G.: Acton recognton n thepremotor cortex. Bran 119, (1996) 2. Lacobon, M., Woods, R.P., Brass, M., Bekkerng, H., Mazzotta, J.C., Rzzolatt, G.: Cortcal mechansms of human mtaton. Scence 186, (1999) 3. Bnkofsk, F., Buccno, G., Setz, R.J., Rzzolatt, G., Freund, H.-J.: Afrontoparetal crcut for object manpulaton n man: evdence from an fmristudy. Eur. J. Neurosc. 11, (1999) 4. Kasabov, N.: NeuCube: a spkng neural network archtecture for mappng, learnng and understandng of spato-temporal bran data. Neural Netw. 52, (2014) 5. Tu, E., Kasabov, N., Yang, J.: Mappng temporal varables nto the NeuCube for mproved pattern recognton, predctve modellng and understandng of stream data. In: IEEE Transactons on Neural Networks and Learnng Systems, pp IEEE Press, New York (2016) 6. Kasabov, N., Scott, E., Tu, E., Marks, S., Sengupta, N., Capecc, E.: Evolvngspato- temporal data machnes based on the NeuCube neuromorphc framework: desgn methodology and selected applcatons. Neural Netw. 78, 1 14 (2016) 7. Doborjeh, M.G., Capecc, E., Kasabov, N.: Classfcaton and segmentaton of fmri spato-temporal bran data wth a neucube evolvng spkng neural network model. In: IIEEE Internatonal Symposum on Crcuts and Systems, pp IEEE Press, Melbourne (2014) 8. Doberjeh, M.G., Wang, G., Kasabov, N., Kydd, R., Russell, B.R.: A Neucube- Spkng neural network model for the study of dynamc bran actvtes durng a GO/NO GO task: a case study on usng EEG data of healthy vs addcton vs treated subjects. IEEE Trans. Bomed. Eng. 63, (2016) 9. Doborjeh, M.G., Kasabov, N.: Dynamc 3D clusterng of spato-temporal bran data n the NeuCube spkng neural network archtecture on a case study of fmri data. In: Ark, S., Huang, T., La, W.K., Lu, Q. (eds.) ICONIP LNCS, vol. 9492, pp Sprnger, Cham (2015). do: / Song, S., Mller, K.D., Abbott, L.F.: Compettve Hebban learnng throughspketmng-dependent synaptc plastcty. Nat. Neurosc. 3, (2000) 11. Kasabov, N., Dhoble, K., Nuntald, N., Indver, G.: Dynamc evolvng spkng neural networks for on-lne spato-and spectro-temporal pattern recognton. Neural Netw. 41, (2013)

6 EEG Comparson Between Normal and Developmental Dsorder Matsumoto, D., Ekman, P.: Japanese and Caucasan facal expressons of emoton (IACFEE) [Sldes]. Intercultural and Emoton Research Laboratory, Department of Psychology, San Francsco State Unversty, San Francsco (1988) 13. Talarach, J., Tournoux, P.: Co-planar Stereotaxc Atlas of the Human Bran: 3- Dmensonal Proportonal System: An Approach to Cerebral Imagng. Theme Medcal Publshers, New York (1988) 14. Koessler, L., Mallard, L., Benhadd, A., Vgnal, J.P., Felblnger, J., Vespgnan, H., Braun, M.: Automated cortcal projecton of EEG sensors: anatomcal correlaton va the nternatonal system. Neuromage 46, (2009) 15. Alfano, K.M., Cmno, C.R.: Alteraton of expected hemspherc asymmetres: valence and arousal effects n neuropsychologcal models of emoton. Bran Cogn. 66, (2008) 16. Kawano, H., Seo, A., Doborjeh, Z.G., Kasabov, N., Doborjeh, M.G.: Analyss of smlarty and dfferences n bran actvtes between percepton and producton of facal expressons usng EEG data and the NeuCube spkng neural network archtecture.in:hrose,a.,ozawa,s.,doya,k.,ikeda,k.,lee,m.,lu,d.(eds.) ICONIP LNCS, vol. 9950, pp Sprnger, Cham (2016). do: /