The impact of numeration on visual attention during a psychophysical task; An ERP study

Transcription

1 The impact of numeration on visual attention during a psychophysical task; An ERP study Armita Faghani Jadidi, Raheleh Davoodi, Mohammad Hassan Moradi Department of Biomedical Engineering Amirkabir University of Tech Tehran, Iran Armita.faghani@gmail.com rahelehdavoodi@gmail.com mhmoradi@aut.ac.ir Abstract- Attention is amongst the high level cognitive task which is associated with complex process in the brain. But in almost all researches, this process is associated with a secondary task, (like numeration). Some believe that the link between these two cognitive activities (i.e. attention and numeration) in human brain is very close and intricate. However, the goal of this research is to evaluate the side effects of counting on visual attention in order to clarify the definition of Attention through brain signals. In previous related researches, only the qualitative impacts of counting have been dealt with and endeavor in this field is a new effort. We used a novel psychophysics task to explore the impact of this extra task on visual top down attention. EEG was recorded during the task from 48 subjects in occipital, Parietal and frontal lobes. Target-locked ERPs for attention with and without numerating were constructed. Time features corresponding to P300 component were extracted for all eight channels separately. Common feature selection method and classifiers were employed to separate two classes (attention with numeration and pure attention). The results indicate that our task was capable of separation and some of the predefined ERP, time features are meaningful while attention is with numeration. As a result, we have introduced ERP features which belong to this separation and also determined the most relevant brain areas. To our knowledge, this is the first time that this quantitative separation is performed. Keywords- Visual attention; secondary task; numeration; P300; ERP; psychophysics I. INTRODUCTION Attention is one of the most interesting and challenging cognitive aspects of brain from the viewpoints of the nature and ALSO processing and analysis. On the other hand, due to its outstanding impact on reduction or increase of intensity of other activities in the brain, it has found a specific importance among the researchers. In the literature, attention a visual selective cognitive process, on one subject or feature of an environment, while ignoring the other features and also removing the impact of some of the (deviating) factors, in order to have an effective dealing with important factors. Numeration is also a mental task which is performed during the attention process in most Ali Yoonessi School of Advanced Medical Technologies Tehran University of Medical Science Tehran, Iran yoonessi@gmail.com experiments in order to evaluate the correctness of performance but it is necessary to consider the effects of this secondary task on attention, i.e. one should remember that true attention is the one without any other mental activities. So, it is essential to distinguish between these two phenomena. Some believe that the link between these two cognitive activities is very close and as a result in their researches, they have used attention along with counting in the definition and numeration is used for attention in the experiment [1]. Some others argue about the qualitative impact of counting on attention and claim that attention and focus accelerate during numeration [2]. Though the impact of different factors such as fatigue or anger on visual attention has been studied in some of the experiments, but no specific research has been done on the analysis of brain signals ( EEG or ERP) to distinguish between these two cognitive tasks (numeration versus attention) [3], [4], [5]. P300 is the most common component in the Event- related Potentials (ERP) and in almost all ERP researches, the use of P300, is very noticeable among other existing ERP components (for example P100 and N200). This component is related to a positive peak for about 300 milliseconds after stimulus onset in vocal stimulations. But in other stimulations like visual tasks, the P300 time window might increases even up to 1000 milliseconds. It is worth mentioning, in most cases, P300 has the greatest amplitude in parietal area and least amplitude in frontal area. Totally, P300 studies reveal that the amplitude of P300 wave has an inverse relation with the possibility of target stimulation and direct relation with rate of concept of this stimulation [6]. To do so, in this paper, we used a novel task to investigate the side effects of numeration on visual attention via electrical brain signal (EEG). Psychophysics task was designed and target and non-target stimuli were displayed to participants. They should attend to target images while ignore non-target ones. The attention process was performed in two ways. At the first time, subjects should attend to target images without counting while at second, they count the number of targets in /14/$ IEEE 338

2 addition to attention and at the end of the sequences they expressed the number. EEG signal was recorded during the task and ERP epochs were extracted for target and non-target stimuli separately. In the processing stage, time features corresponding to P300 component were extracted and selected using different feature selection methods and classification was performed for separation of the classes. II. MATERIALS AND METHODS A. Subjects Forty eight healthy volunteers between 19 and 26 years of age were paid for their participation. All had normal or corrected-to normal visual acuity and provided informed consent. Three participants were excluded from analyses due to an insufficient number of trials after EEG artifact rejections. Finally, 45 subjects (23 males and 22 females) with a mean age of 21, 68 ±2, 39 years, remained. Five of them are left-handed and the other are right-handed, with normal or corrected vision and no history neurological abnormalities. B. Stimuli and procedure The stimuli were presented on a CRT monitor with a gray background. We used a set of 100 images, consisting of three categories including 80 pictures of scenes, 10 human face images and 10 fruit images. All images were changed to the same size and dimension. The image brightness was become equal in all images by averaging over all them. All pictures were changed to black and white in order to reduce the diversion factors of attention. Experiment consisted of four blocks. One hundred images with an interval of 1 second were shown to the subjects in two consecutive blocks. In first stage, subjects were asked to only pay attention to the target images and simultaneously ignore non targets. Target images were mixture of fruit and human face pictures and non-targets were pictures of scenes. In the second stage, subjects were asked to do a secondary task along the attention, i.e. count the number of target images. Two subsequent classes are specified in Table I. TABLE I. Visual attention TWO SUBSEQUENT CLASSES OF ATTENTION Mental task attention to target images attention to target images + Counting the number of targets Target stimuli 10 fruit images + 10 face images 10 fruit images + 10 face images This experiment was a part of a complex task and more details of it could be found in [7]. 100 s Fig 1. Example of the stimulus sequence [7] C. Recording and analysis EEG signal was recorded using 8 active electrodes attached to Electro-Cap electrode system (g.tec) with international 10/20 system sites O1/O2, PO8/PO7, P6/P5, F3/F4. Left earlobe was used as reference and Cz placement as ground. An electrode placed over the right eye was used for monitoring vertical eye movements and blinks and two electrodes on the right side of the right eye and on the left side of the left eye were used for monitoring horizontal eye movements. EEG was filtered using a band pass of Hz, with the sampling rate of 256 Hz. A 50Hz notch filter was used. The impedance of the electrodes was kept below5 kω [7]. D. Preprocessing The task of preprocessing is to prepare the signals for subsequent processing. The required parts of pre-processing are frequency and amplitude filtering, baseline and artifact removal. Frequency filtering was done as follows: 0.5 to 60 Hz band pass filter and 50 Hz notch filter to remove the noise of city power were performed by MATLAB codes of g.tec device during the recording. EEG was segmented into 100 epochs each one is about 1200-milliseconds in each trail. Then, baseline of each epoch and also environmental artifacts such as blinking and other noise signals were removed from the epochs. Baseline was corrected to the activity in the 200 to 0ms preceding the stimulus onset. Trials with artifacts (>60 μv) in any of the electrodes were rejected off-line. EEG for three subjects were eliminated due to noise and other technical problems. In all trials, epochs were extracted for each stimulus separately from 200 milliseconds before the stimulus onset to 1 second after the stimulus onset [7]. Target-locked ERPs for both conditions were extracted. Recorded epochs during the attention to target stimuli were set as "attention states" and epochs during the attention to target stimuli and numeration were set as "attention with numeration states". The preprocessing step is shown in detail in fig 2. 1 s Target Non-target 339

3 EEG Recording 50 Hz Notch Filter [0.5-60Hz] Band pass Filter TABLE II. EXTRACTED TIME FEATURE FROM P300 TIME WINDOW [9] Artifact Removing Baseline Removing Epoch Extraction Feature Maximum amplitude Formula S = Max S(t) Maximum amplitude latency t = {t S(t) = S } ERP Extraction Fig 2. Preprocessing procedure and ERP extraction E. Processing After ERP extraction, the signals were divided into two classes (i.e. first class including only attention to target images, the second class including counting the target pictures in addition to attention) for 45 subjects in 8 channels. The P300 component with the latency window of 300 to 1000 milliseconds was specified. Ten time features for 8channels were extracted from the P300 time window according to [9]. Details of the time features can be observed in table II. The total number of features is equal to 80 (i.e. 10 time features x 8 channels). Due to the large number of features, feature selection is inevitable. T test criteria is a usual method for feature selection. Classification was done by three common classifiers: LDA 1, KNN 2 and SVM 3, and LOO 4 cross validation was employed. Details on the processing methods (i.e. t test criteria, classifiers and cross validation methods) are in [9]. Features were sorted according to t test criteria and first ten optimum features were selected among all extracted time features. The selected features were classified by all three classifiers. In the next step, we have investigate different brain channels in order to determine the most relevant brain areas. So, time features for each channel were selected and classified separately. S = Min S(t) latency t = {t S(t) = S } Absolute total area AS = ΣS(t) Total absolute area SA = Σ S(t) Latency /amplitude ratio Peak-to-peak Peak-to-peak time window Peak-to-peak slope ALAR = t S pp = S S tpp = t t Spp = pp tpp III. RESULTS Classification accuracy results for different classifiers are demonstrated in table III. As it is seen, best results were obtained by SVM classifier with 77.7%. TABLE III. Classifier RESULTS FOR DIFFERENT CLASSIFIERS SVM 77.7% LDA 68.6% KNN 55.5% Accuracy Five top result that selected by feature selection were shown in Table IV. refer to right frontal area is best feature which is bolded. 1 Linear discriminant analysis 2 K-Nearest Neighbors 3 Support Vector Machine 4 Leave One Out TABLEIV. channel F4 F3 O2 PO8 TOP RESULT OF FEATURE SELECTION(T-TEST) feature and peak-to-peak slope Maximum amplitude 340

4 As shown in Table V, the best accuracy refers to occipital and right frontal areas, which are bolded in the table and are related to P300 component features. TABLE V. CLASSIFICATION RESULTS BASED ON CHANNELS Channels Accuracy percentage O1 73.3% O2 73.3% PO8 68.8% PO7 66.6% P6 58.3% P5 57.7% F3 55.5% F4 77.7% At the final step of processing, the most relevant channels with best classification accuracies are considered and best features for each channel introduced in table VI. Fig 4. Two classes grand average ERP waveforms in channel O1, red wave belong to pure attention, green wave belong to attention with numeration. TABLE VI. TOP FEATURES BASED ON ACCURACY Channel Feature F4 O1 O2 Maximum amplitude peak-to-peak slope Two-class grand average ERP waveforms in two top channels were shown in fig 3 and fig 4.Amplitude difference in P300 window is clearly seen. IV. DISCUSSION In current study visual attention and a secondary cognitive task (i.e. numeration) were considered. Here we have used a novel psychophysical task which manipulates these two independently. Top-down attention was manipulated by considering two types of target stimuli (i.e. human face and fruit) and so the effects of attending to special type of targets were eliminated. As it was shown in the results, we could separate an extra mental task (numeration) from attention by means of nearly simple pattern recognition techniques. All features and methods were common in this context and as a result, we could easily express that these two tasks are really different and the effects of numeration on neural correlates of visual attention is undeniable. Here, we have examined time features corresponding to P300 time window due to previous researches and investigation on other ERP time windows are still open. Considering the results, in frontal areas, the best feature was related to the minimum of amplitude. As it was said about the previous researches, if subjects were asked to do a specific mental task along the attention, P300 component with the least amplitude is seen in frontal area. In this project, the specific assignment is numeration of the target images. Fig 3. Two classes grand average ERP waveforms in channel F4, red wave belong to pure attention, green wave belong to attention with numeration. The best result in different brain channels was in the area of frontal. Other top features which were extracted from the time window related to P300 component, has been occipital and parietal area. On the other side, it has been seen that the right part of brain (considering the noticeable percentage of authenticity of channel separation) has shown a good detachment power between these two cognitive activities. 341

5 ACKNOWLEDGMENT The Authors thank all participants for their contribution in this study. REFERENCES [1] P. Cavanagh, SH. He, chapter 3- attention mechanisms for counting in stabilized and in dynamic displays, [2] J. Wilder, E. Kowler, B. Schnitzer, T. Gersch and B. Dosher, attention during active visual tasks: counting, pointing or simply looking, Vision research, vol 49, pp , [3] M. Boksem, T. Meijman, M. Lorist, Effect of mental fatigue on attention: An ERP study, Cognitive Brain Research, , 2005 [4] L. Faber mail, N. Maurits, M. Lorist, Mental Fatigue Affects Visual Selective Attention [5] D. Cohen, C. Eckhard, K. Schagat, Attention allocation and habituation to anger-related stimuli during a visual task, Aggressive Behavior, vol 24, pp , [6] J. Polich, Neuropsychology of P300, In S.J. Luck & E.S. Kappenman, Handbook of event-related potential components, Oxford University Press, in press, [7] R. Davoodi, M. H. Moradi, and A. Yoonessi, "Neural correlates of attention differ from consciousness during a novel psychophysical task," in Biomedical Engineering (ICBME), th Iranian Conference of, 2012, pp [8] I. Kalatzis, N. Piliouras, E. Ventouras, C. C. Papageorgiou, A. D. Rabavilas, and D. Cavouras, "Design and implementation of an SVMbased computer classification system for discriminating depressive patients from healthy controls using the P600 component of ERP signals," Computer Methods and Programs in Biomedicine, vol. 75, pp , [9] Bishop, Christopher M. Pattern recognition and machine learning. Vol. 1. New York: springer,