Beyond Blind Averaging: Analyzing Event-Related Brain Dynamics Scott Makeig Institute for Neural Computation University of California San Diego La Jolla CA sccn.ucsd.edu Talk given at the EEG/MEG course preceding the Human Brain Mapping meeting in Toronto, June, 2005. 1
EEGLAB An Open-Source EEG/MEG Signal Processing Environment for Matlab sccn.ucsd.edu/eeglab Software for applying most of the techniques covered in this talk are freely available in the EEGLAB open source toolbox for Matlab. EEGLAB consists of several hundred Matlab functions, integrated under a graphic user interface by Arnaud Delorme (pictured). A growing list of plug-in capabilities have been contributed or published by other groups, making EEGLAB an attractive open source environment for exploratory signal processing of EEG, MEG, or other electrophysiological data. 2
Adequacy of Blind Response Averaging IF. If equivalent stimuli (passively) evoke the same macro field responses (with fixed latencies and polarities or phase) in all trials If all the REST of the EEG can be considered to be Gaussian noise sources that are not affected by the stimuli.. THEN The stimulus-locked average contains all the meaningful event-related EEG/MEG brain dynamics. The adequacy of the blind response averaging paradigm has been too little questioned in electrophysiology 3
Inadequacy of Blind Response Averaging EEG data ERP mean + EEG NOISEh Highly questionable assumptions:? The living brain produces passive responses?????????????????????? Ongoing EEG processes are not perturbed by events no transient phase locking no ERP contributions from ongoing EEG processes.? Evoked response processes are spatially segregated from ongoing EEG processes.? The true response baseline is flat.? Equivalent stimulus events evoke equivalent brain responses eventrelated brain dynamics are stationary from trial to trial. EEG1 EEG2 EEG4 Blind response averaging does not capture all the important features of eventrelated brain dynamics (!). Taking the results of response averaging as real often leads to both explicit and tacit confusions 4
Monkey LOOK see Monkey Do Do Monkey see Monkey do Currently, much ERP research (as well as much neuroscience in general), remains focused on bottom-up (afferent) brain processing of stimulus events As this figure (based on one published in Science, 2001) shows. However, anatomically, much or even most of the visual system is devoted to top-down (efferent) communication, whose role has until recently been all but ignored. 5
EEG? ERP EEG? ERP EEG? EEG? EEG? ERP C20-50 EEG? ERP EEG? ERP However, much ERP research (as well as much neuroscience in general), remains focused on bottom-up (afferent) brain processing of stimulus events As this figure shows. However, anatomically, much or even most of the visual system is devoted to top-down (efferent) communication, whose role has until recently been all but ignored. 6
Electrodes EEG Local Synchrony Cortex Domains of synchrony Local Synchrony Spatial Source Filtering Scalp sensors average the dynamics of cortical (and non-brain) sources Skin Skull EEG sources must be areas of partially synchronous local field potentials. Because of volume conduction, scalp electrodes (or MEG sensors) record the sum of potentials from different source areas. Spatial source filtering is therefore necessary to accurately measure the synchronized cortical activities. 7
Spatial Filtering by ICA Standard Beamforming Blind Beamforming by ICA Uses a forward head Does not require a forward head model (approximation) model can separate both applied first to separate cortical and non-cortical sources. cortical signals. (But a head model may be applied last). Assumes the signals from both Assumes the signals at cortical and non-brain sources are every defined brain voxel independent no source are uncorrelated. geometry constraint. Minimizes 2 nd -order Minimizes all orders of source (pairwise) correlations. interaction minimum mutual information. A comparison of Independent component analysis (ICA) with current beamforming approaches to spatial filtering for cortical activity. 8
ICA Blind EEG Source Separation Unmixes scalp channel mixing (spatial averaging)! EEG Cocktail Party Independent Component Analysis (ICA) (right) can be used to separate N sound sources summed in recordings at N microphones, without relying on a detailed phonological model of the sounds characteristics of each source this is so-called blind separation. ICA uses the presumption that the waveforms of the individual sound sources are independent over time. Applied to EEG data (left), ICA assumes that the EEG is predominantly composed of a number of domains of synchronous neural (or neuroglial) activity, each of which must, by simple biophysics, project to most of the recording scalp electrodes. If synchronous activity within these domains are predominantly independent of each other, ICA can separate the summed signals from these domains into records of their separation activities, given that the number of such domains making large contributions to the recorded signals are smaller than the number of recording sites. 9
ERP-Image Plotting µv, green is 0! " #$ %&' ("# )*"+,!-- ERP-image plotting (Jung et al., 1999; Makeig et al., 1999) reveals trial-to-trial dependence on external or internal sorting variables (here, subject reaction time, RT). Here, on the left, ERP-image plotting shows the P3 feature of the visual target ERP (lower trace) at scalp site Cz is actually time locked to the subject button press, and is therefore not well represented in the stimuluslocked average. The right image shows the effects of smoothing the ERP image on the left using a (vertical) moving average of ~20 trials. Note the complex relationship of the P2 feature to subject RT 10
Collections of single trials are regular, but in multiple ways so they appear noisy! The ERP Image Stim RT 43.7 500 Sorted Trials 400 300 200 100 21.8 0 21.8 0 100 200 300 400 500 600 Time (ms µv 18.2 18.2 1000 500 0 500 1000 1500 Time (ms) 43.7 Detail of an ERP-image plot 11
ERP image EEG_epoch EEG_epoch EEG_epoch EEG_epoch EEG_epoch EEG_epoch EEG_epoch ERP EEG_epoch time ERP EEG_epoch EEG_epoch EEG_epoch EEG_epoch RT Cz One ERP Many ERP-image projections ERP-image plotting theory. Instead of averaging all trials (here conceived as living in a multidimensional space of trial differences), creating one average ERP (red lines), A sorting variable (here, a sorting direction) is used to group trials along a one-dimensional continuum (straight orange arrow), producing an ERP image. However, many sorting directions are possible, even curving lines (curving orange arrow). Thus, there are many ERP-image visualizations of a set of event-related trials 12
What produces event-related potential averages (ERPs)? Inter-trial Coherence (ITC) ( phase-locking factor ) Significant consistency of local phase of a physiological waveform across successive trials. EVENTS EVENT EVENT EVENT given delay given frequency LOCAL PHASE LOCAL PHASE LOCAL PHASE PHASE LOCKING Inter-trial coherence (= Tallon-Baudry s phase-locking factor ) measures frequency-domain phase coherence between an electrophysiological channel (or independent component!) and a (virtual) indicator channel marking occurrences of stimuli. The result is a measure of the degree of phase consistency (or phase-locking ) at each frequency and latency relative to a class of events. 13
SINGLE TRIALS ERP-image Plot µv AVERAGE ERP INTER-TRIAL COHERENCE (phase resetting) NO AMPLITUDE INCREASE 400 SIM. TRIALS... P = 0.02 INTER-TRIAL COHERENCE P = 0.02 Here, a stimulation of a post- event increase in inter-trial coherence (ITC) was created. Although the effect is hard to see in a set of a few trials, an ERP image of the simulated data shows it clearly. Although ITC increases, power in the simulated frequency band does not. The post- stimulus ERP (top trace), therefore, does NOT represent increased amplitude at one frequency following the simulated events. 14
Pure ERP ITC An example of a phase/latency-sorted ERP-image plot of the activity of an independent component accounting for a large portion of the early visual ERP to single target letters presented in a working memory paradigm (Onton and Makeig, in press). This component process does not exhibit prolonged phaseresetting but rather something like a true or pure ERP. However, its eventrelated dynamics also include a prolonged energy decrease (middle trace) at the same frequencies and its evoked response is triphasic, not contributing to just one ERP peak 15
Phase -Sorted Trials Stimulus 1200 200 1200 10.25 Hz High Alpha, Non-target Lowest Alpha (10%) +10 0 µv -10 200 15 Ss +6 µv -6 0.4 ITC 0 ERP ITC p =.02-200 0 200 400 600 800 Time (ms) Makeig et al., Science 2002 An opposite extreme, from Makeig et al. (Science, 2002), showing partial phase-locking (ITC ~ 0.3) of an ongoing alpha process at the Pz electrode. This is indicated by the ITC trace (lower trace) and by the sigmoidal wave fronts, following stimulus onsets (vertical line) in the alpha phase-sorted ERP image. Note the relation of the ERP at the indicated (dotted line) latency to the individual trials in the phase-sorted ERP image 16
Event-Related Spectral Perturbation (ERSP) Frequency (Hz) 10 db Time (ms) Changes in the baseline-normalized mean power spectrum can be plotted in db in wideband time/frequency format I dubbed the Event-Related Spectral Perturbation (ERSP, Makeig, 1993). Here, the ERSP represents mean changes across the 31 scalp electrodes time locked to the subject button press (vertical solid line) following visual stimulus presentations (whose median latency is indicated by the vertical dotted line). At each time/frequency point, the scalp map of power changes is different, reflecting the mixed contributions to each scalp channel of the ERSP dynamics of multiple cortical source areas. (see Makeig et al., PLOS, 2004). 17
No single measure captures the event-related brain dynamics ERP Is a given ERP feature a true ERP, or phase resetting, or both (ITC)!? Does it coincide with an EEG power increase or decrease (ERSP)? ITC No amplitude effects (ERSP & ERP)! ERSP Does not show phase statistics (ITC). Is a given power increase also in the ERP, or not? But, are these measures enough?? No way! but first Event-related brain dynamics are complex. 18
ERP image of 9.6-Hz alpha power Load 7 Load 5 db Load 3 Onton & Makeig, in prep. EEG and MEG dynamics give clues as to the role of macrodynamics (producing EEG and MEG records) in brain function. Here, a cluster of lateral occipital independent component processes (from a group of subjects performing in a modified Sternberg visual letter working memory task), produce both the triphasic ERP (see 4-back single-subject slide) but also crisp changes in alpha power time locked to experimental events. Here, the ERP image plots the time course of alpha power in single trials (rather than potential). Not the sudden increase in alpha power following the auditory feedback (second curving black line) marking the end of the trial. This and other experiments associate alpha activity in sensory areas with a decrease in level of sensory attention. 19
Independent Alpha Processes An example from my laboratory of two independent components extracted from the same experimental session. Although the activity spectra and stimuluslocked ERPs of the two component processes are nearly identical, and both the scalp maps and equivalent dipole locations of the two processes are bilaterally symmetric, ICA determined that these were temporally independent processes in this session. Note: the scalp maps found by ICA to represent the projection of each source to all the channels very strongly resemble the projection of a single-equivalent dipole (in a simple spherical head model), with a residual variance, across all 253 scalp channels, of about 2% (doubtless the level of head model error)! Thus, the two process scalp maps are highly consistent with their possible generation in single patches of cortex. 20
Frontal Midline Theta Sources Onton et al., NeuroImage 05 A second example from the same Sternberg working memory task (from Onton et al., Neuroimage, 2005 available in press as of 6/05). Here, a cluster of frontal midline theta processes (from 19 of 22 subjects, pink dots) was identified. Their activity spectra (C) are strongly peaked in the theta range (about 6 Hz), while the activity of the overlying scalp channel (Fz) also includes other activity volume conducted from other areas of cortex. Only half of the theta power at Fz was contributed by the radially oriented sources located directly beneath it Yet (as shown in Onton et al., 2005), the fm cluster contributed ALL of the memory-related increase in theta power at Fz. 21
Independent Time-Frequency Modes (Medial Frontal Processes) Letter Onton et al., NeuroImage 05 In the same paper, we introduce a new mode of trial-by-trial decomposition, log spectral ICA. We use this method to demonstrate that the fm processes exhibit at least two modes of activity during presentation of letters to be memorized steady theta and its first harmonic (low beta) (left image) and isolated (and slightly higher-frequency) beta bursts (or their opposite, beta suppression, shown here). 22
Onton et al., NeuroImage 05 Plotting the weights on these two factors for all single trials from the fm process cluster allowed us to trace back and plot the actual time courses of the trials with extreme weights on one or another of the two spectral modes. This demonstrates the reality of the spectral decomposition, and further demonstrates the complexity of event-related activity, even from a set of wellfiltered cortical sources with distinct (mean) dynamics. 23
No single measure captures all the event-related brain dynamics Together, are the ERP, ERP image, ITC, & ERSP enough?? No! For one, they are still averages! TO DO: Explore the structure of (trials subjects source dynamics) Explore the effects of pre-event context and post-event expectation on event-related dynamics Cognition, perception, awareness, consciousness are all active! This is the fundamental shortcoming of average measures! Compare descriptive data models with generative cortical models And with invasive multiscale data! A deeper critique of blind trial averaging methods (either in time or time/frequency domains!) is that the brain subserves active cognition which cannot wait for results of averaging to act! In general, brain electrophysiology is in the midst of a fundamental advance made possible by the easy availability of intensive computational processing power For more information on our work, please visit the Swartz Center (SCCN) web pages: http://sccn.ucsd.edu And/or my personal web pages: http://sccn.ucsd.edu/~scott Scott Makeig La Jolla, CA 6/26/05 smakeig@ucsd.edu 24