5/2/2013. Real-time fmri: Methods and applications

Transcription

1 Real-time fmri: Methods and applications Frank Scharnowski Non-invasive imaging technique PET Magnetic Resonance Imaging (MRI) NIRS EEG MEG Functional MRI Structural MRI Perfusion MRI Diffusion MRI MR angiography Real-time fmri First described in 1995 (Cox, Magnetic Resonance Imaging)... faster algorithms... faster computers... better image quality First real implementation in 2003 (Weiskopf, NeuroImage)... data analysis kept up with data acquisition! 1

2 Conventional fmri fmri image acquisition 1-3 seconds fmri data analysis hours, days, weeks,... Real-time fmri fmri image acquisition 1-3 seconds Has to keep up with... Real-time fmri data analysis hours, days, weeks, seconds 2

3 Conventional fmri analysis Real-time fmri 3 hours 2 seconds Real-time fmri signal (image) quality artefact control online statistical analysis data acquisition image reconstruction head motion magnetic field inhomogeneities Respiration GLM, pattern recognition,... magnetic field strength image resolution (spatial & temporal) instruct to not move the head padding the head robust and fast rigidbody motion correction focus on areas which do not exhibit strong susceptibility effects instruct to breath regularly monitor breathing during the experiment Magnetisation increases with the field strength, but cumulative sliding window 3

4 Real-time fmri signal (image) quality artefact control online statistical analysis data acquisition image reconstruction head motion magnetic field inhomogeneities Respiration GLM, pattern recognition,... magnetic field strength image resolution (spatial & temporal) instruct to not focus on instruct to move the areas which breath head do not exhibit regularly strong susceptibility padding the effects head QUICK and DIRTY: monitor breathing Compromise between spatial resolution, and speed. robust and fast rigidbody motion correction during the experiment cumulative sliding window Real-time fmri signal (image) quality artefact control online statistical analysis data acquisition image reconstruction head motion magnetic field inhomogeneities Respiration GLM, pattern recognition,... magnetic field strength image resolution (spatial & temporal) instruct to not move the head padding the head robust and fast rigidbody motion correction focus on areas which do not exhibit strong susceptibility effects instruct to breath regularly monitor breathing during the experiment cumulative sliding window 4

5 Real-time fmri signal (image) quality artefact control online statistical analysis data acquisition image reconstruction head motion magnetic field inhomogeneities Respiration GLM, pattern recognition,... magnetic field strength image resolution (spatial & temporal) instruct to not move the head padding the head robust and fast rigidbody motion correction focus on areas which do not exhibit strong susceptibility effects instruct to breath regularly monitor breathing during the experiment cumulative sliding window Real-time fmri signal (image) quality artefact control online statistical analysis data acquisition image reconstruction head motion magnetic field inhomogeneities Respiration GLM, pattern recognition,... magnetic field strength image resolution (spatial & temporal) instruct to not move the head padding the head robust and fast rigidbody motion correction focus on areas which do not exhibit strong susceptibility effects instruct to breath regularly monitor breathing during the experiment cumulative sliding window 5

6 Real-time fmri signal (image) quality artefact control online statistical analysis data acquisition image reconstruction head motion magnetic field inhomogeneities Respiration GLM, pattern recognition,... magnetic field strength image resolution (spatial & temporal) instruct to not move the head padding the head robust and fast rigidbody motion correction focus on areas which do not exhibit strong susceptibility effects instruct to breath regularly monitor breathing during the experiment sliding window cumulative Real-time fmri signal (image) quality artefact control online statistical analysis... less vulnerable to linear drifts... data acquisition image reconstruction head motion magnetic field inhomogeneities Respiration GLM, pattern recognition,... magnetic field strength image resolution (spatial & temporal)... closer to offline analysis... instruct to not move the head padding the head robust and fast rigidbody motion correction focus on areas which do not exhibit strong susceptibility effects instruct to breath regularly monitor breathing during the experiment sliding window cumulative 6

7 Real-time fmri signal (image) quality artefact control online statistical analysis data acquisition image reconstruction head motion magnetic field inhomogeneities Respiration GLM, pattern recognition,... magnetic field strength image resolution (spatial & temporal) instruct to not move the head padding the head robust and fast rigidbody motion correction focus on areas which do not exhibit strong susceptibility effects instruct to breath regularly monitor breathing during the experiment sliding window cumulative Real-time fmri Methods What is special about real-time fmri?... data analysis keeps up with image acquisition! How is real-time fmri accomplished?... Fast image acquisition!... Robust artefact control!... Purpose-made statistical analysis algorithms! 7

8 We have got the tools What can they be used for? Real-time fmri 8

9 Real-time fmri 9

10 Intra-operative real-time (f)mri The usefulness of pre-operative structural MRI is well established (e.g. planning tumor resection, navigating during the operation). BUT: During the operation, brain tissue shifts in position (due to craniotomy & tissue resection). Intra-operative real-time (f)mri The usefulness of pre-operative structural MRI is well established (e.g. planning tumor resection, navigating during the operation). BUT: During the operation, brain tissue shifts in position (due to craniotomy & tissue resection). Intra-operative structural MRI allows to guide surgery while taking shifts in tissue position into account. 10

11 Intra-operative real-time fmri BUT: functional brain networks can change (acute plasticity due to tissue resection). Intra-operative real-time fmri allows to track functional changes and thereby to avoid resection of functionally relevant areas. Real-time fmri 11

12 Real-time fmri for Teaching. Real-time fmri allows to cover all essential parts of an fmri experiment with a relatively short hands-on course,... immediately observe the effects of experimental design, hemodynamic delay, head motion artifacts, specific pulse sequences, and different statistical analysis, provide a coherent picture of all the different parts of an fmri experiment. Real-time fmri 12

13 Real-time fmri for Quality Assurance. Real-time fmri allows to monitor the progress of an fmri experiment,... detect problems as early as possible,... increase reliability and facilitate troubleshooting. Example: Real-time quality assurance system at the Wellome Trust Centre for Neuroimaging (developed by Bob Turner, Nik Weiskopf, Oliver Josephs, Chloe Hutton)... Quality Assurance System Continuously updated display of the current images. High-intensitiy (4x) windowing to make ghosting visible. Difference image: current previous image. Time course plot of the variance for the whole volume. Mean signal variance over the previous 10 images (sliding window variance map). Time series of RF noise for each slice (spikes). Time [vols] 13

14 Quality Assurance System RF spike artifact due to malfunction of stimulus equipment: Spikes not visible in the images. Instability! Head Motion? Technical Problem? RF noise plot indicates two spikes seen as high-intensity peaks in autoscaled plot (lower-level RF noise is suppressed). Reveals artifacts and their location due to head motion, cardiorespiratory activity, and scanner instability. Real-time fmri 14

15 Brain-Computer-Interface Non-muscular channel for sending messages and commands to the external world. Measure brain activity, process the brain signal, and translate it into a control command. Invasive Brain-Computer-Interface Hochberg, 2006, Nature. 15

16 Non-invasive Brain-Computer-Interface: EEG Controlling a wheel chair. Non-invasive Brain-Computer-Interface: EEG Letter spelling. Modulate EEG-activity select letters on a computer screen Time-consuming language support programme Excessive training necessary positive reinforcement Birbaumer, 1999, Nature. 16

17 Non-invasive Brain-Computer-Interface: EEG Letter spelling. Non-invasive Brain-Computer-Interface: fmri Limitations of BCIs based on EEG: low spatial resolution no whole brain coverage excessive training necessary Advantages of fmri: high spatial resolution (target specific areas) deep brain structures accessible easy to learn...! 17

18 Non-invasive Brain-Computer-Interface: fmri Non-invasive Brain-Computer-Interface: fmri Spatial navigation by thought. Yoo, 2004, Neuroreport. 18

19 Non-invasive Brain-Computer-Interface: fmri How to increase the information transfer? 4 choices every few seconds... Presentation rate. Information transfer per presentation. More choices? Classification accuracy drops! Non-invasive Brain-Computer-Interface: fmri How to increase the information transfer: spatial-temporal pattern! 2 patterns X 2 times 4 choices Sorger, 2009, Progress in Brain Research. 19

20 Non-invasive Brain-Computer-Interface: fmri How to increase the information transfer: spatial-temporal pattern! motor imagery ROI mental calculation ROI Sorger, 2009, Progress in Brain Research. > 95% accuracy! Non-invasive Brain-Computer-Interface: fmri Why to use BCIs based on fmri at all? fmri-bcis are slower than EEG-based BCIs. fmri-bcis are very expensive. fmri-bcis are not portable. Vegetative state patients might be able to communicate with fmri-based BCIs, i.e. they might benefit from the ease of learning to control a real-time fmri-based BCI. Owen, 2006, Science. 20

21 Real-time fmri Applications What are the applications of real-time fmri?... Intra-operative real-time fmri to guide surgery... Quality assurance to improve the reliability of conventional fmri experiments.... Interactive teaching.... Brain-Computer-Interfaces.... Neurofeedback to learn voluntary control over brain activity. Real-time fmri Applications Why is intra-operative real-time fmri necessary?... During the surgery, brain tissue shifts in position.... Functional brain networks can change during surgery due to acute plasticity. What are the pros & cons of a real-time fmri BCI?... Easy to learn.... Multiple focal brain areas can be targeted simultaneously (including deep brain areas).... Slow information transfer rate.... Expensive and not portable.... No direct measure of neural activity. 21

22 Real-time fmri Real-time fmri for Neurofeedback. to provide feedback about one s own brain activity. can be used by a person to learn voluntary control over brain activity. Feedback is important for learning. But we don t have conscious access to our own brain activity. 22

23 Real-time fmri for Neurofeedback. to provide feedback about one s own brain activity. can be used by a person to learn voluntary control over brain activity. How does voluntary control of brain activity affect perception/behaviour? Brain activity as the independent variable! (Brain activity dependent on behaviour) -1 Correlational techniques: EEG fmri MEG Brain activity... causes measure manipulate... perception / behaviour. Causal techniques: TMS Lesions Pharmacology Neurofeedback manipulate Brain activity... causes measure... perception / behaviour. 23

24 Neurofeedback general setup and data flow. Neurofeedback general setup and data flow. 24

25 Real-time analysis software: Turbo-BrainVoyager Few slices: FAST! Real-time analysis software: Turbo-BrainVoyager Experimental Paradigm ROI time courses Statistical Maps Motion Correction 25

26 Neurofeedback Neurofeedback target areas so far. 26

27 Neurofeedback exemplary data: SMA vs. PG Supplementary Motor Area (SMA) Parahippocampal Gyrus (PG) - movement planning - motor imagery - bimanual coordination - memorized sequential movements - processing of scenes - spatial navigation - memory (e.g. spontaneous PG activity during word encoding predicts their recognition) Does self-regulation of the SMA affect reaction times in a bimanual motor task? Does self-regulation of the PG affect incidental verbal encoding and memory? Sessions (~1.5 hours each) Experimental Design 1 High-resolution structural. Functional localizers. 2 ~3 runs per session (run duration ~8min). 3 4 Neurofeedback training. 5 6 Transfer run. baseline (count backwards) up-regulate down-regulate 7 8 Behavioral testing. (while self-regulating) 27

28 Neurofeedback differential feedback signal. Feedback Signal = difference of the mean BOLD signal from SMA and PG SMA PG... to reduce the signal to one dimension for easy interpretation.... to cancel out any non-specific global effects that could be used to manipulate the signal, such as arousal, breathing, or heart rate. The Feedback Display: normalized differential feedback signal baseline up-regulate down-regulate Neurofeedback learning self-regulation. All 7 participants achieved significant control over the differential signal. 28

29 Neurofeedback behavioural tasks during self-regulation: SMA reaction time test Participants had to carry out one of two acoustically triggered complex finger sequences. & Hypothesis: Faster during SMA up-regulation. Results: All 7 participants were faster in the SMA up / PG down condition. Reaction times correlated negatively with SMA activity in all 7 participants. signal change [%] Neurofeedback behavioural tasks during self-regulation: PG word memory test SALT Unexpected Word Recognition Test (6 words + 1 pseudo-word per block) X VANILLE SALT SALT CAVIAR Sure Sure Sure Sure Unsure Unsure Unsure Unsure Hypothesis: Better recognition when up-regulating the PG during word encoding.... New New New New Performance deteriorates when up-regulating PG (in all subjects). PG down (SMA up) PG up (SMA down) Dual task interference between self-regulation and behavioural testing (competition for the same resources). 29

30 Conclusions SMA vs. PG Voluntary self-regulation of SMA & PG can be learned with neurofeedback, and has causal effects on motor behaviour and memory. Neurofeedback target areas so far. 30

31 Neurofeedback exemplary data: Visual Cortex Spontaneous activity in the visual cortex at rest: Even in the absence of external stimulation, fluctuations in spontaneous brain activity predict whether or not a stimulus will be perceived. (Fox & Raichle, 2007; Hesselmann et al., 2008; Ress et al., 2000) X Study Objectives: Hypothesis: Train participants to explicitly clamp visual cortex activity at high or low levels using neurofeedback. When the level of ongoing activity in early visual cortex is increased, participants will become better at detecting a visual stimulus. Neurofeedback Setup 31

32 Experimental Design Neurofeedback The Feedback Signal: visual cortex activation visual cortex Feedback Signal visual cortex baseline baseline The Feedback Display: visual cortex activity [a.u.] time 32

33 Neurofeedback Training Visual ROI Self-regulation of visual cortex can be learned (not easy), and is maintained despite distracting behavioral testing. Feedback is crucial for learning. V1-, V2, V3-voxel components of the ROI. All V1-, V2, V3-voxels of the corresponding quadrant. Learning is specific to the visual ROI. Behavioral Results Visual ROI Behavior follows BOLD. This improvement was specific to the visual field corresponding to the self-regulated ROI, i.e. no improvements were found for the ipsilateral visual field. 33

34 Signal Change predicts Behavior significant positive correlation between the degree to which an individual could increase activity in the visual ROI and the improvement in visual sensitivity (Pearson correlation r(14) = 0.49, p < 0.05 one-tailed, N = 16). How is learning mediated? Why do some learn and others fail? Cognitive strategies... 34

35 How is learning mediated? Why do some learn and others fail? Attention Rating: Visual Imagery Rating: Learners, non-learners and controls showed indistinguishable attentional effort and vividness of mental imagery. Physiological Measures Heart Rate: Respiration: 35

36 Neural substrates of the learning effect: PPI (psychophysical interaction) analysis to reveal brain areas whose connectivity to the visual ROI changes depending on whether participants were up-regulating or not: During up-regulation, the visual ROI is more correlated with the contralateral superior parietal lobe (SPL) which is involved in directing covert visual-spatial attention and cognitive control. Neural substrates of the learning effect: In the learners, connectivity between the SPL and the visual ROI increases with training: Increasing attentional and cognitive control. The absence of such connectivity in the non-learners and the controls might explain their failure. 36

37 Conclusions visual cortex Voluntary self-regulation of early visual cortex can be learned with neurofeedback, and has a causal effect on the detectability of visual stimuli. Uuufff 37

38 Neurofeedback Uuufff Aha! 38

39 Neurofeedback is intuitive! or what would you do when you get a VCR without a manual? Pressing buttons and seeing what happens! Ways to press some buttons: 39

40 Learned self-regulation of brain activity is not a permanent condition but can be voluntarily switched on/off by the participant! 40

41 LEARN! Neurofeedback learning is not an easy task. It requires the cooperation of the participant. Without being motivated and without being focused, a participant will not be able to learn self-regulation of brain activity. You cannot force a participant to learn self-regulation of brain activity. Real-time fmri Methods What is real-time fmri-based neurofeedback good for? to learn voluntary control over spatially localized brain activity!... to address causal links between brain activity and mental functions! How to cancel out that self-regulation is artefactual (non-neuronal)? use differential feedback!... carefully analyse physiological noise after the real-time experiment, i.e. respiration, heart rate, head motion! 41

42 Real-time fmri Methods What is special about real-time fmri?... data analysis keeps up with image acquisition! How is real-time fmri accomplished?... Fast image acquisition!... Robust artefact control!... Purpose-made statistical analysis algorithms! Real-time fmri Applications What are the applications of real-time fmri?... Intra-operative real-time fmri to guide surgery... Quality assurance to improve the reliability of conventional fmri experiments.... Interactive teaching.... Brain-Computer-Interfaces.... Neurofeedback to learn voluntary control over brain activity. 42

43 Real-time fmri Applications Why is intra-operative real-time fmri necessary?... During the surgery, brain tissue shifts in position.... Functional brain networks can change during surgery due to acute plasticity. What are the pros & cons of a real-time fmri BCI?... Easy to learn.... Multiple focal brain areas can be targeted simultaneously (including deep brain areas).... Slow information transfer rate.... Expensive and not portable.... No direct measure of neural activity. Further reading (review articles) Sulzer J., Haller S., Scharnowski F., Weiskopf N., Birbaumer N., Blefari M., Brühl A., Cohen L., decharms R.C., Gassert R., Göbel R., Herwig U., LaConte S., Linden D., Luft A., Seifritz E., Sitaram R. (2013): Realtime fmri neurofeedback: progress and challenges. Neuroimage, 76(1), decharms (2008): Applications of real-time fmri. Nature Reviews Neuroscience. LaConte (2010): Decoding fmri brain states in real-time. NeuroImage. Weiskopf (2007): Real-time functional magnetic resonance imaging: Methods and applications. Magnetic Resonance Imaging. 43