A Study on Assisting System for Inputting Character using NIRS by the Stimulus Presentation Method of Bit Form

Transcription

1 pp A Study on Assisting System for Inputting Character using NIRS by the Stimulus Presentation Method of Bit Form Tetsuya Shimokawa Non-member (Tokyo University of Science, simokawa@ms.kuki.tus.ac.jp) Keywords: Near Infrared Spectroscopy (NIRS), Brain-Computer Interface (BCI), prefrontal cortex, mental arithmetic, single-trial classification Recent developments in non-invasive neuroimaging technologies have allowed for research on a brain-computer interface (BCI). A BCI is a system that operates a machine, for example a PC, without body movement, using only brain activity. Thus, a BCI is expected to help people with disabilities, assisting them in communicating or making decisions. In this study, we focused on assisting with communication, and in particular, spelling. Therefore, our purpose in conducting this study was to develop a BCI system for assisting with spelling. To develop our system, we used near-infrared spectroscopy (NIRS) to measure brain activity in the prefrontal cortex region. Mental arithmetic was the method of stimulating the activity of the brain. NIRS is an instrument that measures the change in hemoglobin concentration in the blood by the amount of near-ir light change by focusing on the hemoglobin absorption of near-ir light. NIRS is increasingly accepted as a measurement instrument for BCI because its spatial resolutions are higher than those of EEG, it contains less noises than EEG, and it is much smaller in size compared to fmri instrumentation. Restraint with NIRS is minimal compared to fmri. Therefore, NIRS is more suitable for everyday use. Furthermore, brain activity measures in the prefrontal region are suitable input for a system that assists in communication, because the prefrontal region is known to be important for recognition, information selection, and decision-making. Fig. 1 is an outline of images of our BCI system. ( 1 ) The system shows the interface screen on a display. The participant sees information on this display and makes a decision according to the task set. ( 2 ) As the participant makes decisions, the system uses NIRS to measure changes in the participant s brain activity in the prefrontal region. ( 3 ) The measured data are transmitted to the computer by a user datagram protocol (UDP) in real time. Then, the system preprocesses, learns, and discriminates the received data on the computer and feeds the participant s decision back to the display, based on the discriminated result. The participant tries to input the character by repeating this series of steps. As a result of the experiment, the individual variation was observed in each participant, in the channel and timing of activating the brain. Therefore, the method by which the system automatically selects learning and classification data was introduced. When the system automatically selects data, we used a t-test analysis method. Data with a high significant difference was used by priority. Fig. 2. Spelling task (matrix form). Outline of the BCI system. Fig. 3. Spelling task (bit form). Additionally, we proposed the stimulus presentation method on task of bit form (Fig. 3) to replace the matrix form (Fig. 2) 7 used commonly by previous researches. The task flow of bit form is as follows. Step 1 To stabilize the participant s brain activity change, a rest period of 10 seconds was set. Step 2 The character in the character array was sequentially emphasized by 10 seconds according to the 5 bit codes. Only when the character which want to input was emphasized, the participant performed mental arithmetic according to the formula. One task was accomplished by highlighting for 5 times. One character was input per task. Then the classification accuracy was verified offline. As a result of this analysis, classification accuracy enhancement to 78.47% has been shown by support vector machine (SVM). Though additional examination is needed, the future potential of this system using NIRS was demonstrated. 1

2 本文は pp Validity of Method for Estimation of Stimulating Site Based on Cortical Structure in TMS Examined by fmri Masato Odagaki Non-member (Undergraduate Department of Industry, Maebashi Institute of Technology) Hiroshi Fukuda Non-member (Hiroshima City University) Hiroyuki Muranaka Non-member (Hiroshima City General Rehabilitation Center) Osamu Hiwaki Member (Hiroshima City University) Keywords : Transcranial Magnetic Stimulation, Functionarl Magnetic Resonance Image, Primary Motor Cortex, Stimulating Site 1. Introduction Transcranial magnetic stimulation (TMS) is a noninvasive method for stimulating brain nerve. The stimulation coil placed over the head generates the magnetic flux. The magnetic flux induces the electric fields in the brain for the nerve stimulation. Knowing the stimulated brain site in TMS is important to assess the brain function, however it is not easy to identify the stimulating point by TMS because the induced electric fields is broadly distributed over the brain. Although figure of eight coil can give the focal stimulation for the target area, the nerve excitation property on the fiber orientation for the induced electric field must be considered for accurate estimation of stimulated site. We have proposed a method for the estimation of stimulated site on the basis of the cortical structure. Brain activity measured with fmri during the motor task enables us to know the motor cortical area precisely. In this study we measured the cortical area activated during the finger motor task by fmri and compared the stimulating site innervating the finger muscles estimated by our estimation method. We verified the validity of our method for estimation of stimulating sites in TMS by comparison with the brain activity measured with fmri. 2. Materials and Methods The cortical neuron is stimulated by the electric field induced by the pulsed magnetic field in magnetic nerve stimulation. It is known that the induced electric field parallel to the straight axon is most effective for nerve excitation. Our method for estimation of the stimulating site in TMS is taking into consideration of the direction of the electric field and the cortical structure. The stimulating intensity which means the normal component of the electric field for each point on the cortical surface was calculated over the cortical surface. We estimated the maximum stimulating intensity on the cortical surface and determined the stimulating site. We used the estimation system for determing the stimulated brain site in TMS using multi-articular arm to locate the stimulating coil and 3D scanner for the measurement of subject s relative brain location. Four subjects participated in this experiment. The left side of the cerebral cortex was stimulated using a figure-of-eight coil. Motor evoked potentials (MEPs) were measured from the electrodes on the first dorsal interosseous (FDI) and abductor digiti minimi (ADM) muscles. The coil was located at the point where the maximum MEP at FDI or ADM was measured. After the experiment, our method for the estimation of stimulating site in TMS was performed. To identify the location of the cortical area innervating the index finger movement, the brain activity during the right finger motor task was measure with fmri. 3. Result and Discussion Figure 1 shows the estimated stimulating sites in TMS by our method (Stim) and the activated brain site related to the finger movement measured with fmri. The points on the maximum electric field intensity (MaxE), the center of gravity (CoG) and the point beneath the center of coil (CoC) were plotted as well in As shown in Fig. 1, there was good agreement in the localization of motor area for index finger movement between in our estimation method and fmri measurement. The estimation site in TMS and brain activity measured with fmri were located at the area which was anterior to the central sulcus in the left hemisphere. Table 1 listed the average distance between maximal active site measured with fmri in the finger movement task and stimulating site in TMS estimated by our method in all subjects. It was found that our method for the estimation of stimulated brain site in TMS has good accuracy. It was concluded that our method achivied to localize the cortical motor area, compared to the other methods in TMS. Central sulcus Stim fmri CoC CoG Max E Positions of maxial active site measured with fmri and estimated stimulating site in TMS (Index finger of Subject 1) Table 1. Distance between maximal active site measured with fmri and estimated stimulating site in TMS Distance (mm) (mean ±S.D.) Index finger Little finger (1) Stim 17±6 17±5 (2) MaxE 28±3 18±5 (3) CoG 32±8 30±7 (4) CoC 34±7 37±23-2-

3 本文は pp The Detection of Deception Using the Steady-State Visual Evoked Field Shota Katayama Non-member (Tokyo Denki University) Yusuke Itabashi Non-member (Tokyo Denki University) Keita Tanaka Member (Tokyo Denki University) Yoshinori Uchikawa Senior Member (Tokyo Denki University) Keywords : detection of deception, steady-state visual evoked field, magnetoencephalography A deception detection using polygraph tests has been used in criminal investigations. In particular, a technique known as guilty knowledge test (GKT) has been shown to be effective in both laboratory and field setting. This test is designed to determine whether a suspect possesses crime-specific knowledge that only the perpetrator of a crime, not uninvolved people, would be expected to have. However, the central mechanism underlying this technique continues to remain unclear, because the traditional peripheral autonomic measures such as electrodermal, heart rate and respiratory activities do not have sufficient response specificity. Recently, event-related brain potentials (ERPs) have been expected to fill the gap between peripheral and central process. The steady-state visual evoked potential/field (SSVEP/SSVEF) has a sinusoidal waveform at the frequency of the flickering stimulus and is substantially increased in amplitude when attention is focused upon the location of the stimulus. The SSVEP has advantages over the transient evoked potential as a measure of relative attention, because it is a robust oscillating signal that can be extracted rapidly from the ongoing electroencephalogram (EEG) using frequency-domain analysis methods. Most importantly, the SSVEP is an ongoing waveform that indexes attention-related changes in cortical facilitation continuously rather than intermittently as with transient evoked potentials. This study investigated that the detection of deception was related to the amplitudes of the steady-state visual evoked field (SSVEF) in magnetoencephalgraphy (MEG). We prepared four cards depicted symbols (triangle, square, circle, and rhombus), and subjects selected one card used as critical item and the other three cards used as non-critical items. For nine subjects, we presented stimuli which is flashing the symbol on 15Hz (Fig. 1), and were measured with a 122-channel magnetometer system (Fig. 2). We obtain then the source waveform of MEG and made Hilbert-transform (Fig. 3). As a result, the amplitude of SSVEF in critical item was larger than non-critical items images during the time interval s after the stimuli (p<0.05) (Fig. 4). Time 1.0s 2.0s 20Hz 1.0s 20Hz Stimuli and a trial sequence in this experiment (Flicker rates: 20Hz, Stimulus duration: 2s, Rest time: 1s, Total time: 3s). magnetically shielded room magnetic field strength (nam) visual sensory stimulus projector screen SQUID system (Neuromag 122 TM ) stimulus trigger MEG signal (122ch) stimulus computer measurement and analysis workstation Fig. 2. MEG measurement system. * stimulus critical non-critical time (ms) Fig. 3. Time course of the amplitudes resulting from Hilbert-transform of the 20Hz component in a source waveform under critical (black line) and non-critical (gray line) items. *p<0.05 Magnetic field strength(nam) critical item * non-critical items Fig. 4. The amplitude of SSVEF obtained from critical item (black line) and non-critical item (gray line). Time interval s after the stimuli. *p<

4 pp A Study on Distinction of Unfamiliar Information using NIRS Tetsuya Shimokawa Non-member (Tokyo University of Science, simokawa@ms.kuki.tus.ac.jp) Keywords: Near Infrared Spectroscopy (NIRS), Brain-computer interface, Neuromarketing, Single-trial classification Recent developments in neuroimaging technologies have given us a better understanding of the activation of sites in the brain. These developments have made it possible to apply neuroimaging technologies to decision-making in economic activities. In addition, several brain-computer interface (BCI) systems have been reported. A BCI system is used to read a user s decisions on the basis of brain activity and to then assist the user by performing actions based on his/her intentions. A number of noninvasive function-measuring technologies have been developed, such as functional magnetic resonance imaging (fmri), magnetoencephalography (MEG), electroencephalography (EEG), and near-infrared spectroscopy (NIRS). The current study explored the potential for the use of a brain activity in a situation that involves unfamiliar information. Typical applications are that a user encounters unfamiliar information while browsing a website, he/she can automatically obtain details of this information through a BCI without having to perform a web search. The construct of our system is as follows. Unfamiliar task Measuring equipment NIRS equipment is OEG-16 produced by Spectratec, Inc. Sensor interval is 3 cm and data sampling interval is 0.65 s. The measurement site is prefrontal cortex. Real-time data transfer The A/D sampling data are transferred from the NIRS equipment to a computer in real time using the User Datagram Protocol. The following unfamiliar tasks were performed in this study (Fig. 1). Unfamiliar task step.1:rest period X was displayed for 10 s to allow for stabilization of the oxyhb concentration. step.2:display of Image Image, which is familiar or unfamiliar for user, was displayed for 10 s. step.3:rest period X was displayed for 10 s to allow for stabilization of the oxyhb concentration. step.4:display of results of the search Search engine result page, which is about image displayed in step.2, was displayed for 10 s. Steps 1-4 above constituted 1 task. Fig. 2. Arithmetic mean of oxyhb A total of 26 subjects, all students at the University of Toyama, participated in the study. Each subject performed the task 60 times. But data of 2 subjects is removed from data for analysis. Because these subjects had pain during the experiment and had little understanding of unfamiliar task. Fig. 2 shows the plot for subject s oxyhb concentration. You can see that reaction is positive tendency. So, brain activity for unfamiliar information is positive reaction. In order to verify application of BCI, Experimental results was evaluated about distinguishing unfamiliar information by offline with leave-one-out cross validation. As a result, the average accuracy was about 80%. This paper described a possibility of identification of unfamiliar information using brain activity measured by NIRS. In future studies, it will be necessary for real-time system to examine the selection of data used in learning and discrimination. 4

5 pp A Development of NIRS-based Brain-Computer Interface for Robot Control Shinya Takano Non-member (University of Toyama, m @ems.u-toyama.ac.jp) Keywords: Near Infrared Spectroscopy (NIRS), Brain-Computer Interface, Support Vector Machine, Robot Contorol Recently, developments in neuroimaging technologies have facilitated a clearer understanding of the activation of sites in the brain. This technology is applied to brain-computer interface (BCI). A BCI is used to determine user intentions on the basis of brain activity and to display these intentions through the system. In addition, many brain-computer interface (BCI) systems based on functional magnetic resonance imaging (fmri) or electroencephalography (EEG) have been reported. However, these techniques have serious limitations in terms of size and scale for rehabilitation purpose, since it requires physical training in an open environment to help people live a healthy, useful, and active life after injury or illness. Previous BCIs have primarily used information on the brain activity related to the motor system. Moreover, sufficient research has not been conducted on BCI systems that are used to investigate activities in parts of the brain other than the motor area. Hence, in this study, a BCI that assists motor functions on the basis of brain activity in the prefrontal cortex was experimentally developed and its validity was confirmed. And we investigated the possibility of developing a BCI system based on NIRS. Fig. 1 shows an outline of the trial BCI system used in this study. BCI requires the construction of a system with an online learning model. Thus, the following system was constructed. ( 1 ) Measurement of brain information: The brain activity is measured using NIRS every 0.65 s. ( 2 ) Real-time data transfer: The A/D sampling data are transferred from the fnirs equipment to a computer in real time using the UDP protocol. ( 3 ) Preprocessing, learning, and classification of data: The computer preprocesses the data. Preprocessing that removes high-frequency signals via Savitzky-Golay smoothing filter is used to eliminate noise in the data. Next, the computer learns and classifies the data. In the experiment, the distinction accuracy of the Linear Discriminant Analysis, Hidden Markov Model, support vector machine and multi-support vector machine were verified. We propose the multi-support vector machine which is new clustering technology. Since the result of multi-support vector machine was good, the distinguishing of unfamiliar information takes place through the use of a multi-support vector machine. The subject tries to control the robot by repeating this flows. Fig. 2 shows a part of result. Fig. 2 (a) (b) shows a move result of a subject s two experiments. Fig. 2 (a) is the successful person s experimental result. Fig. 2 (b) is the subject who was successful once Fig. 2. Outline of the BCI system. Experiment result. of two experiments. 17 subjects both male and female, participated in this experiment. Moreover, the results, 16 people in 17 people reached the goal. Thus, Between two experiments, it is shown in this result that about 94% of the subject was able to control the robot. Fig. 2 (c) (d) compares random movement with a subject s movement. Successful people like Fig. 2 (c) reached the target earlier than random movements. Those who are not good as for a result also have like Fig. 2 (d) a person who is not different from random movement. Moreover, when the 1st time and the 2nd time compared, while the 1st time had the effective subject of 65% (11/17), the 2nd time was effective to the subject of about 88% (15/17). When a subject decided an intended action, the part of the brain activated could be specified in the developed BCI to some degree. In addition, movement assistance was shown to be possible through brain activity measurements in areas other than the motor cortex. Therefore, movement assistance can be provided by measuring the brain activity in the prefrontal cortex. 5