NeuroImage xxx (2014) xxx xxx. Contents lists available at ScienceDirect. NeuroImage. journal homepage:

Transcription

1 YNIMG-11396; No. of pages: 11; 4C: 3, 4, 6, 7, 8, 9 NeuroImage xxx (2014) xxx xxx Contents lists available at ScienceDirect NeuroImage journal homepage: 1 Technical Note 2 Simultaneous recording of MEG, EEG and intracerebral EEG during visual 3 stimulation: From feasibility to single-trial analysis 4Q1 Anne-Sophie Dubarry a,b,c, Jean-Michel Badier a,c, Agnès Trébuchon-Da Fonseca a,c,d, Martine Gavaret a,c,d, 5 Romain Carron a,e, Fabrice Bartolomei a,c,d, Catherine Liégeois-Chauvel c,d,jeanrégis a,d,e, Patrick Chauvel a,c,d, 6 F.-Xavier Alario a,b, Christian Bénar a,c, 7 a Aix-Marseille Université, Faculté de Médecine La Timone, Marseille, France 8 b Aix-Marseille Université, CNRS, LPC UMR 7290, Marseille, France 9 c INSERM, UMR 1106, Institut de Neurosciences des Systèmes, Marseille, France 10 d APHM, Hôpital de la Timone, Service de Neurophysiologie Clinique, Marseille, France 11 e APHM, Hôpital de la Timone, Service de Neurochirurgie Fonctionnelle, Marseille, France 12 article info abstract 13 Article history: Electroencephalography (EEG), magnetoencephalography (MEG), and intracerebral stereotaxic EEG (SEEG) are Accepted 17 May 2014 the three neurophysiological recording techniques, which are thought to capture the same type of brain activity Available online xxxx Still, the relationships between non-invasive (EEG, MEG) and invasive (SEEG) signals remain to be further inves- 24 tigated. In early attempts at comparing SEEG with either EEG or MEG, the recordings were performed separately Keywords: for each modality. However such an approach presents substantial limitations in terms of signal analysis. The goal MEG 18 of this technical note is to investigate the feasibility of simultaneously recording these three signal modalities 27 EEG 19 SEEG (EEG, MEG and SEEG), and to provide strategies for analyzing this new kind of data. Intracerebral electrodes Simultaneous recordings were implanted in a patient with intractable epilepsy for presurgical evaluation purposes. This patient was pre Single-trial analysis sented with a visual stimulation paradigm while the three types of signals were simultaneously recorded. The 30 analysis started with a characterization of the MEG artifact caused by the SEEG equipment. Next, the average 31 evoked activities were computed at the sensor level, and cortical source activations were estimated for both 32 the EEG and MEG recordings; these were shown to be compatible with the spatiotemporal dynamics of the 33 SEEG signals. In the average time frequency domain, concordant patterns between the MEG/EEG and SEEG re- 34 cordings were found below the 40 Hz level. Finally, a fine-grained coupling between the amplitudes of the 35 three recording modalities was detected in the time domain, at the level of single evoked responses. Importantly, 36 these correlations have shown a high level of spatial and temporal specificity. These findings provide a case for 37 the ability of trimodal recordings (EEG, MEG, and SEEG) to reach a greater level of specificity in the investigation 38 of brain signals and functions Elsevier Inc. All rights reserved Introduction 46 Scalp electroencephalography (EEG) and magnetoencephalography 47 (MEG) are standard techniques that are widely used for the study of 48 physiological or pathological activities in large-scale brain networks, 49 with a millisecond time resolution (Lopes da Silva and Van Rotterdam, ). These techniques consist of non-invasive surface recordings 51 and thus require solving a difficult inverse problem in order to infer 52 the activity at the cortical level (Baillet et al., 2001). In contrast, intrace- 53 rebral EEG (stereotaxic EEG, SEEG) records electrical activity directly 54 from specific brain areas with great anatomical accuracy by means of 55 implanted multi-lead electrodes. This invasive technique is used in the 56 pre-surgical evaluation of drug-resistant epilepsies (Talairach and Bancaud, 1973). Beyond its clinical significance, the recorded signal 57 can be analyzed in parallel for research purposes (Barbeau et al., 2008; 58 Engel et al., 2005). 59 The three recording modalities (EEG, MEG, and SEEG) are thought to 60 capture signals produced by the same type of neural activity, arising 61 from post-synaptic activity of pyramidal cells (Pfurtscheller and Lopes 62 da Silva, 1999). Still, the signals have diverse properties, and the rela- 63 tionship between non-invasive (EEG, MEG) and depth recordings 64 (SEEG) remains largely unknown. Characterizing the correspondence 65 between scalp and intracerebral recordings would provide crucial in- 66 sights to better understand the cortical source configuration of the sig- 67 nals recorded in EEG and MEG techniques. 68 In early attempts at comparing SEEG with either EEG or MEG, the re- 69 cordings were performed separately for each modality. While valuable, 70 this approach presents substantial limitations. As brain activity fluctu- 71 ates over time, separate recordings cannot ensure that the exact same 72 Corresponding author at: Institut de Neuroscience des Systèmes, INSERM UMR 1106, 27 Bd Jean Moulin, Marseille Cedex 05, France / 2014 Elsevier Inc. All rights reserved.

2 2 A.-S. Dubarry et al. / NeuroImage xxx (2014) xxx xxx 73 activity is seen on the two recordings. Simultaneous recording would 74 allow the comparison of average responses from the very same neural 75 events across the different modalities. Most importantly, it is the only 76 method that allows the combined analysis of signal fluctuations evoked 77 by individual stimuli, referred to as single trials. This is a prerequisite 78 for an optimal use of multimodal recording, where fine-grained co- 79 variation across modalities can be exploited as a new and major source 80 of information on the links between modalities (Bénar et al., 2007). 81 A few recent studies have reported simultaneous recordings of MEG- 82 SEEG or EEG-SEEG in epilepsy (Kakisaka et al., 2012; Pacia and Ebersole, ; Santiuste et al., 2008) or in cognition (Dalal et al., 2009), but 84 neither the quality of the non-invasive signals nor the possibility of re- 85 covering evoked activity at the single trial level has been addressed. 86 Moreover, the complementary information provided by MEG and EEG 87 (Dehghani et al., 2010) has not been considered in these studies. 88 The enterprise of combining MEG, EEG and SEEG recordings within a 89 single trimodal session is not without technical challenges. Besides 90 physical constraints, such as fitting the bulky electrode connections 91 within the MEG dewar, the recording session requires a tight organiza- 92 tion in order to ensure the smoothness of the operations and to reduce 93 the amount of time during which the patient is in the MEG laboratory 94 (and therefore not in the monitoring unit). Despite these precautions, 95 the quality of non-invasive recordings can be affected by the presence 96 of the SEEG equipment. This can in turn compromise single-trial analy- 97 ses, which do not compensate for low signal-to-noise ratio through 98 averaging. 99 The goal of this technical note is to report on the feasibility of simul- 100 taneously recording three signal modalities (EEG, MEG and SEEG), and 101 to provide strategies for analyzing this kind of data. To our knowledge, 102 this is the first report of such a trimodal recording. The evidence comes 103 from a patient case study involving a visual stimulation paradigm. 104 Material and methods 105 Patient 106 The recordings were performed on a 19-year-old female patient 107 with intractable epilepsy undergoing an intracerebral EEG pre-surgery 108 evaluation. The locations of implantation were strictly imposed by clin- 109 ical indications. The trimodal recording session was performed on the 110 last (10th) day of clinical evaluation. Several weeks after SEEG de- 111 implantation, the patient was re-tested on the same task in a separate 112 session during which only MEG signals were recorded. This was 113 intended to evaluate the impact of the SEEG equipment on the spatial 114 distributions (topographies) and on the time courses of the evoked 115 field. We compared the first session (with SEEG) and the second session 116 (without SEEG). The patient gave a written consent for both sessions. 117 This research has been approved by the relevant Institutional Review 118 Board (i.e. the Comité de Protection des Personnes, CPP ). 119 Stimulation paradigm 120 The paradigm consisted in the presentation of a full screen reversing 121 checkerboard pattern during 5 min. The checkerboard was composed of by 8 black and white squares covering the whole screen with a small 123 central yellow fixation point. The patient was passively watching the 124 screen in a supine position at a distance such that every square unit rep- 125 resented an angle of 1.5. The pattern reversed every second such that a 126 total of 300 checkerboard reversals were presented. A photocell was 127 flashed at each reversal to allow precise timing. This stimulation experi- 128 ment was developed using Presentation software ( 129 Recording set-up 130 The data were acquired at La Timone hospital in Marseille on a 4D 131 Neuroimaging MEG/EEG system at a sampling rate of 2035 Hz. The recording included 20 s of background activity followed by 5 min of stimulation and further background activity to reach a total duration of 10 min. A total of 248 MEG magnetometers were recorded with an online correction based on reference channels. The electrocardiographic activity was recorded on a bipolar EEG channel. Scalp EEG was recorded from twenty-three GRASS silver-chloride EEG electrodes positioned on a10 20 montage slightly modified to avoid any overlap with the implantation sites. A right mastoid reference was used for both EEG and SEEG. Intracerebral EEG electrodes were implanted stereotaxically orthogonal to the midline vertical plane. The location of the electrodes was bilateral with predominance in the left hemisphere (11/13 leftsided electrodes) exploring the temporo-parieto-occipital junction. The electrodes had a diameter of 0.8 mm, and contained 10 to 15 recording sites (also referred to as contacts ), each 2 mm long and separated from each other by 1.5 mm (Alcis, Besançon, France). This resulted in a total number of 190 contacts. The electrodes were labeled (from the most posterior to the most frontal): GL, CU, FCA, PP, PFG, PFG (right), OT, OT,GC, PI, GPH, B, TB and contact numbers start from the most mesial one to the most lateral one (for an overview of the implantation sites see Figs. 1c and4). The recording equipment used in the present study constrained the number of recorded SEEG contacts to 72. The neurologists selected contacts based on the patient epileptic activity and the planned functional mapping of the primary visual areas. Anatomical co-registration Co-registration of the EEG electrodes was performed manually based on pictures of the patient taken during the placement of these electrodes. Anatomical MRI pre-implantation, MRI intra-implantation and CT scan intra-implantation were all co-registered using MedInria software ( index.php). Each electrode contact was identified on the CT scan performed during the implantation and all electrodes were reconstructed in 3D using Brainvisa ( combined with our own code in MATLAB 2012a (Mathworks, Naticks, MA). The co-registration between the MRI and MEG coordinate systems was based on three fiducials: nasion, left and right pre-auricular points for the trimodal recording and on the head shape for the separate session. The quality of the coregistration was checked using the digitization of the facial mask. Spectral analysis and signal pre-processing Based on the observation of the continuous signals, we identified two groups of 8 MEG sensors each near the SEEG implantation site that presented two different types of artifacts: low and high-frequency noise. Another set of 8 sensors on the contralateral side was taken as a reference (see Fig. 1). In order to characterize the MEG artifacts, the power spectral density of each of the three groups (averaged over sensors) was computed over 10 s of background data (no stimulation) using Welch's method (Welch, 1967). The continuous data of all channels was band pass filtered with a fourth-order forward backward Butterworth filter (Anywave software, MEG laboratory, INS, We used a 1 40 Hz bandpass for time domain analysis and a1 120 Hz bandpass for time frequency analysis. The scalp EEG data was additionally filtered with a notch filter between 45 Hz and 55 Hz in order to remove the 50 Hz contamination. Artifacts and channel rejection, segmentation into epochs, averages and source localizations were performed with Brainstorm Matlab toolbox (Tadel et al., 2011). For visualizing the averaged evoked activity we used the 40 Hz low pass filter implemented in Brainstorm. The continuous data were inspected in order to identify artifacts, either physiological (blinks, epileptiform discharges) or instrumental (bad or noisy sensors). Blinks and epileptiform discharges were detected manually on continuous data (respectively on MEG and SEEG signals). Epochs free of artifact were extracted around the checkerboard reversal, from 200 ms to 500 ms for the time domain analysis and

3 A.-S. Dubarry et al. / NeuroImage xxx (2014) xxx xxx 3 Fig. 1. Recording set-up, constraints, and major artifacts during the simultaneous recording of SEEG, EEG and MEG signals. (a) The patient was lying in the MEG bed. Scalp EEG electrodes were directly connected to the acquisition device. Depth electrodes were equipped with shorter cables than EEG and required the use of specific connectors (see red arrow). These were in contact with the patient's shoulder during the recording. They are suspected to have produced a movement artifact on the raw MEG signals. (b) The topography of the low frequency MEG component (low pass filtered at 40 Hz) shows a clear activity on the sensors nearest to the patient's shoulder. The corresponding time course of the raw MEG data reveals a slow modulation (around 1 Hz) likely related to breathing. (c) A lateral view of the patient's head during the placement of scalp EEG electrodes (clearly visible) shows the bolts that held the depth electrodes in place (green arrows). (d) The 2D layout view of the MEG high frequency component (high pass filtered at 5 Hz) reveals a group of adjacent sensors with strong high frequency noise (blue circle). These are above the depth electrodes bolts. The signals had more low frequency noise in the sensors that are nearest to the patient's shoulder (red circle) compared to a reference group of contralateral sensors (green circle). (e) Average spectra of the three groups of 8 sensors indicated by colored circle both for the trimodal and the separate (with MEG only) recordings. 194 from 500 ms to 800 ms for the time frequency analysis (in order to 195 take into account the edge effects). 196 Time domain averages at sensor level 197 Signal averages were computed separately for each modality of the 198 trimodal session as well as for the MEG data of the separate session. A 199 baseline correction was performed on a pre-stimulus window 200 [ 200 ms; 0 ms]. 201 To quantify the overall similarity of the averaged signals across the 202 three modalities, we computed the global field power (GFP) (Lehmann 203 and Skrandies, 1980) of the average signal of each modality separately 204 and calculated the Pearson correlations of their time courses. Since the 205 artifact rejection procedure did not leave the same number of trials in 206 the two sessions, we randomly selected from the largest dataset the 207 same number of trials as in the smallest dataset to compute the GFP 208 and the correlation. We repeated this procedure 500 times and, after ob- 209 serving that the distribution of the correlation coefficient was Gaussian, 210 reported the average of their values. average level (SNR average, Eq. (1)), and another one at the single trial 217 level (SNR trials Eq. (2)) SNR average ¼ 20 log 10 SNR trials ¼ 20 log 10 A max σ average A max σ trials!! ð1þ ð2þ with A max the maximum amplitude of the averaged evoked response 224 between 50 ms to 100 ms after stimulation, and σ the estimated standard deviation, computed either on the baseline of the averaged signal 225 (σ average )oracrosstrials(σ trials ). The values obtained at all MEG sensors 226 were imported into Brainstorm in order to visualize them on a topo- 227 graphic view (interpolated with 10 contour lines) (Fig. 2). 228 Source localization Signal to noise ratio 212 We quantified the MEG signal quality degradation introduced by the 213 EEG/SEEG equipment during the trimodal session by computing the sig- 214 nal to noise ratio (SNR) of the evoked responses separately for each ses- 215 sion. For MEG sensors, two SNR measures were computed with our own 216 code in MATLAB. One SNR aimed at evaluating the signal quality at the The 3D reconstruction of the patient's cortex was performed with BrainVisa (Rivière et al., 2003) and down-sampled to 15,000 vertices. The Minimum Norm Estimate source localization method was used (Hämäläinen and Ilmoniemi, 1994), for which the regularized linear inverse operator is given by M ¼ R 0 G T GR 0 G T þ λ 2 1 C ð3þ

4 4 A.-S. Dubarry et al. / NeuroImage xxx (2014) xxx xxx Fig. 2. MEG maps of signal to noise ratio. Left/right column: separate/trimodal session. Up/down row: SNR average /SNR trials. There is a decrease of SNR on the left side presumably due to the presence of SEEG connectors, and this effect is more prominent at the single trial level (black arrow). The maps of the two sessions should not be compared sensor by sensor because the head position has changed between the two sessions. 236 where G is the lead field matrix, R is the source covariance matrix, λ is the regularization parameter set to λ 2 = SNR-1 with signal to noise ratio de- 237 fined as the ratio of variances computed on the whitened noise, and C is 238 the noise covariance matrix. The MEG lead fields were computed using 239 an overlapping-sphere head model (Huang et al., 1999) based on the pa- 240 tient's individual anatomy, whereas the EEG lead fields were computed 241 using a Boundary Element Model (Gramfort et al., 2010). The noise co- 242 variance C was computed from 30 s of recording post stimulation with 243 the patient inside of the MEG array with no stimulation. It was regularized 244 with a factor of 0.1. The SNR parameter was set to 2 for the MEG during 245 simultaneous recordings and to 3 for the MEG only. No constraint on 246 the orientation of the dipoles was used. We corrected the bias of the 247 method towards superficial currents by adjusting the matrix R, 248 performing a depth weighting (order 0.5, max amount 10). Further de- 249 tails are available at Time frequency analysis 251 The EEGLAB Matlab toolbox (Delorme and Makeig, 2004) was used 252 to compute the time frequency analysis of the three modalities (bipolar 253 SEEG, EEG and MEG). We used a seven cycle Morlet wavelet to compute 254 the event related spectral perturbation (ERSP) (Makeig, 1993). The gain 255 model described by Delorme and Makeig (2004) was used to perform 256 the baseline correction over the pre-stimulus period ( 500 ms to 0). 257 Maps were thresholded using False Discovery Rate (FDR) correction 258 (q b 0.05) (Genovese et al., 2002). 259 In the three modalities, the time frequency planes of the posterior 260 sensors were reviewed visually in order to identify representative pat- 261 terns. For each pattern we selected the sensors showing a local peak, ei- 262 ther positive or negative. 263 Single trial correlation across modalities 264 At each time point, we computed a robust linear regression (Holland 265 and Welsch, 1977) across trials between SEEG contacts and non- 266 invasive data. The details of this procedure are as follows. For each time t [ 200, 500 ms] and each SEEG contact x we esti- 267 mated β 1 in 268 data depth ðk; t; xþ ¼ β 1 ðt; xþdata surface ðk; tþþβ 0 ðt; xþþε ð4þ using robust regression (Matlab function robustfit). With k = 1 trials. 270 Data vectors data depth and data surface had previously been standardized by removing the mean and dividing by the standard deviation. 271 For each surface time series, we defined the inter-trial correlation 272 index (Itcor) as being the t-test of significance at each time point t and 273 each SEEG location x: 274 Itcorðt; xþ ¼ jtstatðt; xþj ¼ β 1ðt; xþ σ β1t;x ð Þ with σ estimated standard deviation. 276 The threshold of significance of the Itcor index was determined using the False Discovery Rate (Genovese et al., 2002) with a level 277 of q = This addresses the issue of multiple comparisons jointly 278 in time (one test per time sample) and in space (one test per pair of 279 surface and intracerebral contacts). Additionally, a threshold on the 280 length of significant time windows was set to 10 ms in order to fur- 281 ther reduce the number of false detections. 282 To avoid capturing spurious correlations driven by the use of a com- 283 mon reference between EEG and SEEG, we computed a bipolar montage 284 on the SEEG signals. The bipolar montage construction consisted in 285 subtracting all pairs of recorded and not rejected consecutive contacts 286 within each electrode. 287 The Itcor was estimated over independent components extracted 288 from ICA (Bell and Sejnowski, 1989) performed on the non-invasive sig- 289 nals, in order to increase the signal-to-noise ratio (critical for single trial 290 analysis). The EEGLAB Matlab toolbox (Delorme and Makeig, 2004) was 291 used to compute ICA separately on the MEG and EEG data. As a first step, 292 we selected visually for each modality (EEG and MEG) the ICA compo- 293 nent corresponding to the visual evoked response. This component 294 was selected on the basis of its temporal mean timecourse and its 295 ð5þ

5 A.-S. Dubarry et al. / NeuroImage xxx (2014) xxx xxx topography across sensors. We confirmed the visual selection by 297 performing a t-test for each component at each time point. This allowed 298 assessing whether the evoked response on a given component differed 299 significantly from zero. In a second step, we ran the single trial correlation 300 analysis on all ICA components as well as all sensors in an exploratory 301 manner. There, the FDR correction accounted for multiple comparisons 302 not only across time and SEEG contacts but also across sensors/ica 303 components. 304 The same approach was used to perform the single-trial correlation 305 analysis in the time frequency domain. We computed the Morlet trans- 306 form (see Time frequency analysis section) for each trial and summed 307 the activities for bands of interest: beta Hz, gamma Hz, 308 gamma Hz, and gamma Hz. 309 Results 310 Preprocessing and spectral characterization of the recordings 311 The artifact rejection left 242 trials spanning [ 200 ms; +500 ms] 312 for the first session and 161 trials of similar length for the separate 313 MEG session (without simultaneous SEEG nor EEG). One flat and two 314 noisy MEG channels (3/248) were excluded from the analysis of both 315 sessions. Five EEG (5/23) and five SEEG channels (5/72) were rejected 316 from the trimodal analysis because they were either not showing any 317 signal (5 EEG and 3 SEEG contacts) or were noisy (2 SEEG contacts). 318 We compared the MEG signals recorded during two sessions (with 319 and without SEEG and EEG equipment). During trimodal recording, a 320 left posterior group of sensors located above the patient's shoulder 321 showed a low frequency artifact (b1 Hz) directly observable on the 322 raw data, presumably following the patient's breathing rhythm. In con- 323 trast, a left central group of sensors, above the SEEG implantation site, 324 showed a high frequency component (N15 Hz). The separate recording 325 from the MEG-only session did not show such artifacts. The contralater- 326 al sensors to the implantation site showed similar amplitudes and spec- 327 tra across sessions (Fig. 1). 328 Evaluation of the quality of MEG signals 329 The spatial pattern of the SNR values (Fig. 2) was rather symmetric 330 for the separate session. In contrast, for the trimodal session we ob- 331 served a decrease of SNR in the left posterior region presumably due 332 to the presence of SEEG connectors. This effect was more prominent at 333 the single trial level. 334 Still, the maximum values of the SNR on the right side were of the 335 same order. The SNR average was 30.9 db versus 28.9 db respectively for 336 trimodal and separate sessions. The SNR trials was 2.4 db versus 3 db 337 respectively for trimodal and separate sessions. 338 Concordance of time-domain average responses 339 We compared the responses evoked by the visual stimulus (a revers- 340 ing checkerboard) in the three recording modalities. The average 341 evoked responses of the three modalities showed main peaks at the 342 same latency (80 ms), as presented on Fig. 3 (left panel). In addition, 343 an early response was observed on MEG signals at 35 ms (both ses- 344 sions). At the same latency, a peak of low amplitude (relative to the 345 main peak) was also observed on the medial contacts of the posterior 346 SEEG electrodes (GL, FCA, OT ) as indicated by a star on Fig. 3a, but 347 no peak was observed on the EEG signals. The average evoked response 348 obtained during the separate MEG-only session (Fig. 3d) showed a 349 waveform and a spatial pattern similar to those obtained during the si- 350 multaneous recording. 351 The overall similarity of the signals across the three modalities was 352 quantified as the temporal correlation of the global field power (GFP) 353 of each signal averaged separately. We compensated for the different 354 numbers of trials between the two MEG sessions as described in the Material and methods section. Within the simultaneous session, the correlation coefficient between SEEG and EEG was It was 0.86 between SEEG and MEG, and 0.86 between MEG and EEG. The MEG signals of the separate session showed a correlation of 0.70 with the MEG signals (of the simultaneous session), 0.75 with the SEEG signals and 0.73 with EEG signals. Concordance of reconstructed sources To visualize the activity at the cortical level, we computed the source localization of the MEG and EEG averaged evoked responses. In Fig. 4 we present source activations at three latencies (35, 80 and 120 ms), along with the corresponding SEEG potentials. These latencies were chosen for being representative of the spatiotemporal dynamics of the evoked response. SEEG activations were mainly observed on four occipito-temporal electrodes exploring the lingual gyrus (GL and FCA ), the cuneus (Cu ) and the fusiform gyrus (OT ). The activations progressed within these electrodes towards the lateral occipital cortex. In EEG, the first latency revealed no substantial activation. At the later latencies (80 and 120 ms) a broad posterior temporo-parieto-occipital region was activated, as well as the left lateral occipital region. In MEG, the reconstructed activations from both sessions were similar for the three latencies. Firstly (35 ms), a small activation was observed in the medial part of left and right occipital poles. Then (80 ms), the activation spread through the bilateral ventral pathway and the first steps of the dorsal pathway in the lateral occipital cortex. Finally (120 ms), the polar activity disappeared while the lateral activity remained present. Time frequency analysis The artifact rejection procedure left 217 trials for time frequency analysis in the trimodal session and 140 in the separate session. The rejected channels were the same as in the time domain analysis. The visual inspection of the time frequency planes across all modalities (in both sessions) revealed four main patterns (Fig. 5). The first pattern was an increase of energy in the Hz band before 100 ms that was visible both in MEG and SEEG. In EEG a similar pattern was observed at a slightly higher frequency (around 40 Hz). The second pattern was a decrease of energy around 20 Hz. This pattern was clearly visible in the SEEG signals and in the MEG signals of the separate session. It was present but less visible on the EEG and MEG sensors of the trimodal recording. The third pattern was a broadband increase of energy in the high gamma range (N50 Hz) clearly visible on SEEG contacts, but scarce and inconsistent for the non-invasive data. The fourth pattern was an increase of energy in the Hz band at around 300 ms and was visible in the MEG and EEG signals of the trimodal session. Correlations between modalities at the single-trial level We performed the inter-trial correlation analysis (Itcor) on the selected ICA components. Within each modality, the component that was visually selected corresponded to the maximum t-value in the t- test analysis across trials for each time point and each component (IC #9 for EEG and IC #26 for MEG). The inter-trial correlation (Itcor) was computed between these two ICA components and each SEEG contact. The MEG ICA component showed a significant correlation with seven SEEG contacts from three bilateral posterior electrodes (GL 1 togl 3, CU 4, CU 6, OT8, and OT11) (q b 0.05 FDR corrected). Notably, the correlation occurred earlier (starting around 80 ms) for the most posterior contacts (on electrode GL), and later (starting around 150 ms) for the most anterior contacts (on electrode OT). This pattern was consistent with peak activity progression as seen on the source localization data. The EEG ICA component showed a significant correlation with two contacts from two SEEG electrodes (TB 2, Cu 4) (q b 0.05 FDR corrected). On all raster plots (Fig. 6), the column corresponding to the main evoked

6 6 A.-S. Dubarry et al. / NeuroImage xxx (2014) xxx xxx Fig. 3. Concordance of evoked responses in simultaneously recorded SEEG, EEG and MEG signals, and in MEG signals after de-implantation. The left column shows the average time series, highlighting three time stamps (35, 80, and 120 ms) used in Fig. 4. These time points were selected to be representative of the spatiotemporal dynamics of the evoked response. The right column shows the spatial pattern of activity at the peak (80 ms). (a) Depth electrodes: monopolar evoked potentials from medial contacts and lateral contacts of two electrodes (FCA and GL ) located in the visual areas as shown on the right column (in absolute values). (b) Scalp EEG: evoked potentials for 5 posterior electrodes (shown on the right scalp rendering as black dots), and topography shown on the scalp (scalp reconstruction was performed from the intra-implantation MRI; the lateral view shows the actual implantation sites of depth electrodes). The gray color corresponds to areas with either no electrode or an electrode that was rejected from this analysis. (c) MEG: evoked fields during the trimodal recording session, and topography represented on the MEG helmet. (d) MEG signals recorded during a separate unimodal session, without SEEG or EEG. In (a c) N = 242 trials, in (d) N = 161 trials. Asterisks indicate the early response observed on SEEG and MEG signals. 415 response clearly appeared. In SEEG, two additional columns were visible 416 following the main peak (indicated by arrows in Fig. 6a), but only the 417 last one was clearly visible on EEG and MEG. The increase in correlation 418 occurred at the time of the secondary peaks. By construction, these cor- 419 relations reveal the co-variation of the signals around their respective 420 means and not only the increase of signal amplitude at the evoked po- 421 tential (see Material and methods for details). 422 The correlations in the time frequency domain were computed in frequency ranges (beta, gamma1, gamma2, gamma3) between the se- 424 lected ICA components (based on the visual response as described in 425 Single trial correlation across modalities section) and the SEEG contacts. 426 No significant correlation was observed. 427 As a final step we computed the single trial correlations exhaustively 428 over all sensors and ICA components both in temporal and time frequency domain. The results are shown as supplementary material 429 on Fig. S1. In the time domain, correlations were observed in EEG both 430 on sensors (statistically significant on 10 electrodes) and ICA compo- 431 nents (statistically significant on 4 ICA components, including that se- 432 lected based on the visual response, IC#9). They mainly occurred after ms post stimulation. The two ICA components with the most signif- 434 icant correlations (IC#8 & IC#13) showed a posterior spatial pattern 435 (Fig. S1a). Correlations between SEEG and MEG were only observed on 436 the ICA components, specifically on two of them: the component previ- 437 ously selected based on the visual response (IC#26), and another compo- 438 nent also showing a posterior spatial pattern compatible with a brain 439 response in the visual areas (IC#14; Fig. S1b). Finally, the computation 440 of the time frequency correlations per frequency band for EEG and 441 MEG data did not show any significant correlations in the gamma 442

7 A.-S. Dubarry et al. / NeuroImage xxx (2014) xxx xxx 7 Fig. 4. Concordance of reconstructed cortical activations for the three modalities at three notable timestamps of the evoked response (corresponding to those shown on Fig. 3). (a) Activity recorded from depth electrode contacts (absolute values) mapped on a 3D representation of all electrodes within the cortex. (b c) Cortical source localizations for EEG and MEG signals, respectively. (d) Cortical source localization for the separate MEG session of the same patient without SEEG or EEG. For each modality, the maximum color scale value was set to the maximum value of activation over the whole time window. The threshold on source estimates was set to 10% of the maximum value. 443 bands. Two significant correlations appeared in the beta band for ICA 444 components extracted from EEG data (Fig. S1c). 445 Discussion 446 We recorded SEEG, EEG and MEG in a patient within a single 447 trimodal acquisition. We described the signals obtained in the three mo- 448 dalities both in time and time frequency domains. We characterized 449 the artifact observed on the MEG signals due to the presence of SEEG/ 450 EEG equipment. Most importantly, we provided strategies aimed at re- 451 vealing the presence (or absence) of correlations between depth and 452 non-invasive signals at the single trial level. 453 Signal quality 454 As expected, the presence of the SEEG equipment resulted in charac- 455 teristic artifacts in the raw MEG signals. A low frequency component 456 was presumably caused by the presence of the SEEG connectors in con- 457 tact with the patient's shoulder, which moved with breathing. A high 458 frequency component was more likely due to the presence of the bolts holding the depth SEEG electrodes in place on the scalp of the patient. 459 This is consistent with previous studies that described artifacts caused 460 by metallic parts within the MEG array (Hämäläinen et al., 1993; 461 Hillebrand et al., 2013; Volegov et al., 2004). 462 Ball and colleagues investigated the signal quality degradation on si- 463 multaneous recordings of EEG and intracerebral EEG (Ball et al., 2009). 464 However they focused a specific type of physiological artifact (eye 465 blinks). In our recordings, the SEEG induced artifacts impacted the sig- 466 nal to noise ratio of the MEG signals in the posterior left region, especial- 467 ly at the single trial level. In order to improve the signal quality (for 468 general MEG guidelines see Gross et al., 2013), future efforts should 469 Q2 be made when positioning the patient and the equipment in the MEG 470 system. The SEEG connectors should be placed as far as possible from 471 the MEG array and as far as possible from the region(s) of interest. 472 Also, every effort should be made to avoid contact them with the pa- 473 tient's body. 474 In spite of these constraints, non-invasive signals (both MEG and 475 EEG) were interpretable. Previous clinical studies have demonstrated 476 that non-invasive signals (either MEG or EEG) recorded along with 477 SEEG could be processed and interpreted. However, these studies only 478

8 8 A.-S. Dubarry et al. / NeuroImage xxx (2014) xxx xxx Fig. 5. Time frequency analysis of the visual response. (a) SEEG bipolar channel located on two posterior electrodes. (b) Posterior EEG electrodes, (c) posterior MEG sensors in the trimodal recording, and (d) posterior MEG sensors in the separate session. Circle one indicates the Hz increase of energy pattern. Circle two indicates a decrease of energy around 20 Hz visible in the three modalities as well as in the separate session after 100 ms. Circle three indicates a broadband increase of energy above 50 Hz visible on SEEG. Circle four indicates increase of energy in the Hz band at around 300 ms visible in MEG and EEG signals of the trimodal session. 479 focused on epileptiform discharges (Kakisaka et al., 2012; Santiuste 480 et al., 2008) and epileptic oscillations (Rampp et al., 2010), which are ac- 481 tivities that have a high signal-to-noise ratio within their respective fre- 482 quency bands (Zelmann et al., 2012). 483 Evoked responses early response observed at 35 ms on SEEG and MEG but not on EEG, pre- 489 sumably because of the underlying source orientation (Ahlfors et al., ; Merlet et al., 1997). The net benefit of simultaneous compared to 491 separate recordings (Godey et al., 2001) is readily apparent in the correla- 492 tion coefficients between non-invasive and depth signals; these were 493 substantially larger within the trimodal session than across sessions The evoked responses that we obtained were comparable to those 485 generally observed for EEG, MEG, or SEEG recorded separately (Barbeau 486 et al., 2008; Hatanaka et al., 1997; Nakamura et al., 1997). In our study, 487 the average waveforms were consistent between non-invasive (MEG, 488 EEG) and depth (SEEG) recordings. The only notable difference was an Identification of time frequency patterns The visibility of specific frequency bands in the three modalities has largely been discussed on separate recordings (Lachaux et al., 2005; Michel and Murray, 2009; Vidal et al., 2010). While MEG studies

9 9 N C O R R E C T E D P R O O F A.-S. Dubarry et al. / NeuroImage xxx (2014) xxx xxx U Fig. 6. Single-trial analysis. (a c) Raster plots (left column) and spatial distribution of the corresponding activity (right column). (a) Depth bipolar channel showing the highest amplitude activation during the peak of the evoked response (GL 2 GL 1). Activities consistent across trials are visible as columns (indicated by black arrows) (b) EEG ICA component capturing an evoked response whose temporal and spatial characteristics reflected the evoked response. (c) MEG ICA component with similar properties. (d e) Left column: sample by sample inter-trial correlation (Itcor) between activity at several depth electrode contacts and, respectively, the selected EEG and MEG ICA components. Only the SEEG contacts showing significant correlation are shown (q b 0.05 FDR corrected for comparison across SEEG contacts and time samples). Right column: scatter plots of values at the peak of correlation, shown as arrow heads on the traces (Adjamian et al., 2004; Hoogenboom et al., 2006; Schwarzkopf et al., 2012) have shown an increase in the gamma band during visual paradigms, our data did not reveal such activity neither in the trimodal nor in the separate session. However the gamma activity was clearly present in the SEEG data. This indicates that strong oscillatory activity in the cortex can go unnoticed on non-invasive data. The absence of high gamma on MEG and EEG could arise from the fact that our stimulation protocol only induced the synchronization of small areas of cortex in this frequency band (Schwarzkopf et al., 2012). Alternatively, other signal processing methods could help enhance the detection of gammaband activity (e.g. beamforming Dalal et al., 2009, 2013). Reconstructed sources 510 The cortical sources estimated from the non-invasive signals revealed sources that were consistent with the location of the SEEG contacts that detected the evoked potential, similarly to Dalal et al. (2009). In addition, we showed that the temporal evolution of SEEG activity, starting from primary areas and propagating to secondary areas, could be retrieved from the non-invasive recordings. This spatiotemporal pattern is consistent with the previous findings (Hagler et al., 2009). Importantly, our results demonstrate that, within the trimodal recording, the precise spatiotemporal dynamics of evoked neural processes

10 10 A.-S. Dubarry et al. / NeuroImage xxx (2014) xxx xxx 520 (e.g. Sergent et al., 2005) can be revealed by the source reconstruction 521 procedure. topic (Malmivuo et al., 1997); our results suggest that the two modalities are best used in combination (Zijlmans et al., 2002) Single trial correlations 523 The full benefit of simultaneous recordings, however, only comes 524 from the single-trial analyses. This has been shown in the field of EEG- 525 fmri, where trial-to-trial signal fluctuations were a fruitful source of in- 526 formation on the coupling between modalities (Bénar et al., 2007; 527 Debener et al., 2005). 528 At the level of sensors, results were either not significant (MEG) or 529 without clear spatial or temporal pattern (EEG). In contrast, indepen- 530 dent component analysis allowed retrieving meaningful response fluc- 531 tuations at the level of single-trial (Jung et al., 2001). Most importantly, 532 it allowed us to observe significant correlations between the amplitudes 533 of non-invasive and depth signals. This confirms the usefulness of ICA 534 for capturing single trials correlations on multimodal data as done pre- 535 viously in the field of simultaneous EEG-fMRI (Debener et al., 2005; 536 Eichele et al., 2005). Moreover, the correlation patterns revealed a 537 high level of specificity in space and time: only a few SEEG contacts 538 and time points were statistically significant. Most interestingly, MEG 539 correlations were observed rapidly after the stimulation (i.e. within ms post-stimulation) on SEEG contacts located in the visual cortex, 541 although not precisely at the peak of the evoked response. 542 Concerning the small number of SEEG contacts presenting correla- 543 tions, a factor to take into account is the relatively low spatial sampling 544 of SEEG, both across the brain and within a given region. Many of the 545 electrodes that we sampled were not located in visual areas. The ab- 546 sence of correlation on their contacts is not surprising, and is in fact 547 reassuring regarding the selectivity of the correlation measure. In 548 MEG, the significant correlations that we observed on the ICA corre- 549 sponding to the visual response reproduced the spatiotemporal pat- 550 terns seen at the source level. However, our analysis did not consider 551 the possible impact of source orientations on bipolar SEEG (for discus- 552 sion see Lachaux et al., 2004, p. 615) or on MEG and EEG. Variations in 553 the local cortical orientation may promote or dampen the relationship 554 with the non-invasive signals. This effect might vary from MEG to EEG 555 signals because of the different sensitivities to source orientations of 556 the two recording techniques: MEG being less sensitive to radial sources 557 than EEG (Ahlfors et al., 2010; Merlet et al., 1997). 558 The fact that correlations occurred later than the main peak of 559 evoked activity (Fig. 6) is somewhat unexpected. A tentative explana- 560 tion can be built on the fact that the three modalities have different vol- 561 umes of sensitivity. While intracranial electrodes measure a focal 562 activity, especially in a bipolar montage (Lachaux et al., 2004), MEG 563 and EEG record a mixture of synchronous activities over a broad area 564 (Cosandier-Rimélé et al., 2012). If a focal activation such as the earliest 565 peak of the visual response has spatial variability across trials (arising, 566 for example, from small differences in eye fixation), this would have a 567 more direct impact on SEEG amplitude than on MEG/EEG amplitude. 568 In contrast, a less focal activation (for example, the one observed slightly 569 after the peak) might be more homogenous across trials both in depth 570 and surface sensors. While only tentative, this account could explain 571 why the correlations are not strongest at the peak. This proposal is com- 572 patible with previous findings by Cohen et al. (2008), who reported a 573 modest amplitude coupling along with consistent phase coupling be- 574 tween scalp and depth EEG signals. 575 A limitation of our study is that only 23 electrodes were placed on 576 the scalp because of physical and clinical constraints. This could nega- 577 tively impact detectability at the EEG sensor level, as well as the separa- 578 bility of ICA components. Despite this limitation, EEG presented more 579 significant correlations with depth signals than MEG. Still, it is worth 580 noting that the correlated SEEG contacts with MEG and EEG were differ- 581 ent; those correlating with MEG being located within the primary visual 582 areas. The differential capacities of EEG and MEG are a controversial Conclusion and perspectives This technical note reports a procedure for recording MEG, EEG, and SEEG signals within a single session. Our results show the feasibility of such trimodal recordings, and the tractable SNR of the signals. For the reported case, the data revealed clear and meaningful correlations between certain signal components, as well as a notable degree of specificity. The main limitation of this study is that a single case was explored. Thus, it is premature to translate the findings to other cases or the general population. In addition, the complexity of this procedure is not negligible. For this reason, the current findings are not intended to provide arguments against unimodal recordings, as these are to be preferred in most contexts. Still, trimodal recordings shall provide unique information on the coupling between signals while controlling for the fluctuations of brain activity across trials or recording sessions. There are numerous sources of such confounding fluctuations when signals are not recorded concurrently: spontaneous changes of the brain state, attention levels, learning, consequences of pathological activity, etc. In this context, trimodal recordings could be used to address several issues. One is to investigate the visibility of gamma oscillations (Michel and Murray, 2009) or epileptic discharges (Alarcon et al., 1994) across the three modalities. This is directly relevant for unimodal research. A better understanding of how and to what extent surface signals capture activities directly seen within the cortex should constrain the interpretation of surface empirical phenomena. Another perspective is to fuse the three modalities into a common model, such as in joint source-localization procedures or joint connectivity analysis. In sum, this preliminary study shows that, despite its complexities, trimodal recordings have the potential to address specific neurophysiological questions from a new perspective. Supplementary data to this article can be found online at doi.org/ /j.neuroimage Acknowledgments This research was supported by the European Research Council under the European Community's Seventh Framework Program (FP7/ Grant Agreement No ) and by the Multimodel ANR grant (2010 BLAN ). The authors thank Teresa Cvetkov for proofreading the manuscript. References Adjamian, P., Holliday, I.E., Barnes, G.R., Hillebrand, A., Hadjipapas, A., Singh, K.D., Induced visual illusions and gamma oscillations in human primary visual cortex. Eur. J. Neurosci. 20, Ahlfors, S.P., Han, J., Belliveau, J.W., Hämäläinen, M.S., Sensitivity of MEG and EEG to source orientation. Brain Topogr. 23, Alarcon, G., Guy, C.N., Binnie, C.D., Walker, S.R., Elwes, R.D., Polkey, C.E., Intracerebral propagation of interictal activity in partial epilepsy: implications for source localisation. J. Neurol. Neurosurg. Psychiatry 57, Baillet, S., Mosher, J., Leahy, R., Electromagnetic brain mapping. IEEE Signal Process. Mag. 18, Ball, T., Kern, M., Mutschler, I., Aertsen, A., Schulze-Bonhage, A., Signal quality of simultaneously recorded invasive and non-invasive EEG. Neuroimage 46, Barbeau, E.J., Taylor, M.J., Regis, J., Marquis, P., Chauvel, P., Liégeois-Chauvel, C., Spatio-temporal dynamics of face recognition. Cereb. Cortex 18, Bell, A.J., Sejnowski, T.J., An information-maximisation approach to blind separation and blind deconvolution. 1034, Bénar, C.-G., Schön, D., Grimault, S., Nazarian, B., Burle, B., Roth, M., Badier, J.-M., Marquis, P., Liegeois-Chauvel, C., Anton, J.-L., Single-trial analysis of oddball eventrelated potentials in simultaneous EEG-fMRI. Hum. Brain Mapp. 28, Cohen, M.X., Ridderinkhof, K.R., Haupt, S., Elger, C.E., Fell, J., Medial frontal cortex and response conflict: evidence from human intracranial EEG and medial frontal cortex lesion. Brain Res. 1238,