Working with EEG/ERP data. Sara Bögels Max Planck Institute for Psycholinguistics

Working with EEG/ERP data Sara Bögels Max Planck Institute for Psycholinguistics

Overview Methods Stimuli (cross-splicing) Task Electrode configuration Artifacts avoidance Pre-processing Filtering Time-locking Baselining Artifacts Averaging Analysis Time-windows Multiple electrodes Cluster-analysis Time-frequency analysis Source localization

Methods Stimuli (cross-splicing) Task Electrode configuration Artifacts avoidance

Auditory stimuli Necessary for prosody research More natural than visual presentation (RSVP) More difficult to manipulate / less controlled

Types of stimuli 1. Instructed speaker Freely choose materials 2. Corpus materials More natural Enough naturally occurring stimuli available? Can you manipulate the prosody? - Pauses: e.g., silence study Bögels, Kendrick, & Levinson (2015) - Use Praat to manipulate pitch/length: less natural, differences between manipulated and non-manipulated conditions? 3. Production study to generate materials Need good elicitation method Variability between and within speakers

Cross-splicing 1. Instructed speaker Cross-splicing Compare exact same acoustic tokens in different contexts Compare different tokens in the exact same context Fully crossed design

Cross-splicing Stimuli De chirurg adviseerde de vrouw te slapen... (intransitive) The surgeon advised the woman to sleep... De chirurg adviseerde # de vrouw te slapen... The surgeon advised # the woman to sleep... De chirurg adviseerde de vrouw te ondersteunen... (transitive) The surgeon advised the woman to support... De chirurg adviseerde # de vrouw te ondersteunen... The surgeon advised # the woman to support......voordat ze onder het mes zou moeten....before the start of the surgery. Assignment What would you record? How would you cross-splice? Try out with Praat!

Cross-splicing Only record natural sentences Record De chirurg adviseerde de vrouw te slapen... De chirurg adviseerde # de vrouw te ondersteunen... Create De chirurg adviseerde de vrouw te ondersteunen... De chirurg adviseerde # de vrouw te slapen...

Cross-splice all conditions? Recorded: 1a De chirurg adviseerde de vrouw te slapen... 1b De chirurg adviseerde de vrouw te slapen... 2a De chirurg adviseerde # de vrouw te ondersteunen... 2b De chirurg adviseerde # de vrouw te ondersteunen... Experimental: De chirurg adviseerde de vrouw te slapen... De chirurg adviseerde de vrouw te ondersteunen... De chirurg adviseerde # de vrouw te slapen... De chirurg adviseerde # de vrouw te ondersteunen... Bögels et al. (2010)

Cross-splicing No unnatural sentences have to be recorded Cross-splice all conditions Use similar phonemes at the point of cross-splicing Preferably record all stimuli in one session; to-be cross-spliced items close together Use PRAAT script (zero-crossings) Ask second opinion: click hearable etc. Perform acoustic analyses: difference between conditions e.g., length of pause, amount of lengthening, pitch height/range Works remarkably well! Questions?

Number of items 20 minimum - start with 30 (artifacts etc.) Per condition! - think about repetition - without repetition: 120 different items for 4 conditions

Task Assignment What different tasks can you think of in EEG prosody research? What are advantages and disadvantages of these tasks? Which do you like best?

Task 1. Prosody judgment task Leads attention to manipulation Not very natural 2. Comprehension task What kind of comprehension? 3. Minimal task (pay attention), e.g., memory EEG allows for minimal task Passive listening most natural? What is natural behavior? Can lead to differences in effects! (e.g., accents)

Electrode configuration Auditory areas General coverage: distribution for comparison Source-localization: larger array (e.g., 64)

Minimize artifacts Design/Instruction No contact lenses Sit still, no blinks/eye movements during trials (fixation cross) No visual stimuli during measurements Include time to blink in design (in between trials) Measure eye-electrodes

Filtering Time-locking Baselining Artifacts Averaging Pre-processing

Filtering High-pass filter (e.g., 0.05 Hz): remove very slow drifts Low-pass filter (e.g., 30 Hz): remove highfrequency noise

Time-locking & Baselining Time-locking At the onset of the event of interest Baselining at neutral interval before event: put to zero

Time-locking & Baselining Assignment How would you time-lock the CPS (elicited by prosodic boundary)? Where would you put the baseline?

Time-locking CPS A. Onset sentence (e.g., Steinhauer et al., 1999) B. Onset last stressed syllable before pause C. Onset pause (e.g., Kerkhofs et al., 2007) A. Jitter & removed trials C. Miss elements of boundary Bögels et al. (2010)

Baselining Compare different baselines e.g., Pauker et al. (2011) -500 to -150 ms, -500 to 0 msec, and -50 to 50 ms Be careful where you baseline

Dealing with artifacts 1. Remove trials with artifacts automatic manual 2. Correct for artifacts a) pre-specified method e.g., Gratton, Coles, & Donchin (1983) b) ICA e.g., Gross et al., 2012

Averaging Traditional Average within condition & participant One ERP per participant & condition Statistics over participants New Mixed-effects models for EEG: take into account variability in items and participants Grand average over participants to visualize

Time-windows Multiple electrodes Cluster-analysis Analysis

Multiple comparisons Multiple electrodes e.g., 32 or 64 Multiple time points: Sampling rate of 500 Hz Assignment How many comparisons can you make between 2 conditions in a 1 second window using 32 electrodes? How would you diminish the number of comparisons?

Multiple time points Choose time-windows Average over the window Pre-defined (previous research) Exploratory effects - visual inspection - time-course analyses: 50/100 ms windows N400: 300-500 ms Late positivity 1100-1400 ms Negativity: 300-400 ms

Multiple electrodes ANOVA with Hemisphere/ROI Left/Right hemisphere Anterior/(Middle/)Posterior Average over electrodes Separate midline analysis Anterior Left Right Posterior

Cluster-analysis No choices about time-windows and electrode groupings Procedure Maris & Oostenveld (2007) Paired T-tests per time-point & electrode Fz Cz Pz POz 1 2 3 4 5

Cluster-analysis No choices about time-windows and electrode groupings Procedure Maris & Oostenveld (2007) Paired T-tests per time-point & electrode All points with p <.05 are selected Neighbouring points are clustered Cluster statistic per cluster: sum of t-values Monte Carlo method: - 1000 random permutations of samples of 2 conditions - Largest cluster-statistic per randomizations enters distribution Observed cluster statistic is significant if proportion of distribution larger than that <.05 1 2 3 4 5 Fz Cz Pz POz

Time-frequency analysis EEG to time-frequency 5-7 Hz 30-40 Hz

Time-frequency analysis Differences to ERP Hypotheses: more exploratory Pre-processing - No filtering - No baseline necessary - Create time-frequency representation per trial => average - Choose sliding window size 500 ms time

Time-frequency: examples Magyari et al. (2014) Bögels et al. (2015)

Source-analysis Calculate source locations of effects Possible for both ERP and time-frequency data Different methods LCMV beamforming for ERP Van Veen & Buckley (1988) DICS beamforming for time-frequency Gross et al. (2001) Localize significant effects Define time-window (and frequency range) where effect is maximal Statistics: cluster-analysis over voxels

Source-localization Difficult for EEG Scull and scalp distort electrical currents => need (standard) head models Easier with MEG: magnetic current goes trough scalp Inverse problem for EEG and MEG: identify 3D-source with 2D surface data => multiple solutions

Source-analysis Different visualizations Surface Slices orthograpic (interactive)

References Bögels, S., Barr, D. J., Garrod, S., & Kessler, K. (2015). Conversational Interaction in the Scanner: Mentalizing during Language Processing as Revealed by MEG. Cerebral Cortex, 25(9), 3219-3234. Bögels, S., Kendrick, K. H., & Levinson, S. C. (2015). Never Say No How the Brain Interprets the Pregnant Pause in Conversation. PloS one, 10(12), e0145474. Bögels, S., Schriefers, H., Vonk, W., Chwilla, D. J., & Kerkhofs, R. (2010). The interplay between prosody and syntax in sentence processing: The case of subject-and object-control verbs. Journal of Cognitive Neuroscience, 22(5), 1036-1053. Gratton, G., Coles, M. G., & Donchin, E. (1983). A new method for off-line removal of ocular artifact. Electroencephalography and clinical neurophysiology, 55(4), 468-484. Gross, J., Baillet, S., Barnes, G. R., Henson, R. N., Hillebrand, A., Jensen, O., et al. (2012). Good-practice for conducting and reporting MEG research. NeuroImage. Gross, J., Kujala, J., Hämäläinen, M., Timmermann, L., Schnitzler, A., & Salmelin, R. (2001). Dynamic imaging of coherent sources: studying neural interactions in the human brain. Proceedings of the National Academy of Sciences, 98(2), 694-699. Kerkhofs, R., Vonk, W., Schriefers, H., & Chwilla, D. J. (2007). Discourse, syntax, and prosody: The brain reveals an immediate interaction. Journal of Cognitive Neuroscience, 19(9), 1421-1434. Magyari, L., Bastiaansen, M. C., de Ruiter, J. P., & Levinson, S. C. (2014). Early anticipation lies behind the speed of response in conversation. Journal of Cognitive Neuroscience, 26, 2530-2539. Maris, E., & Oostenveld, R. (2007). Nonparametric statistical testing of EEG-and MEG-data. Journal of Neuroscience Methods, 164(1), 177-190. Pauker, E., Itzhak, I., Baum, S. R., & Steinhauer, K. (2011). Effects of cooperating and conflicting prosody in spoken English garden path sentences: ERP evidence for the boundary deletion hypothesis. Journal of Cognitive Neuroscience, 23(10), 2731-2751. Steinhauer, K., Alter, K., & Friederici, A. D. (1999). Brain potentials indicate immediate use of prosodic cues in natural speech processing. Nature neuroscience, 2(2), 191-196. Van Veen, B. D., & Buckley, K. M. (1988). Beamforming: A versatile approach to spatial filtering. ASSP Magazine, IEEE, 5(2), 4-24.

Thank you! Questions?