Computational approaches for understanding the human brain.

Transcription

1 Computational approaches for understanding the human brain. John P. O Doherty Caltech Brain Imaging Center

2 Approach to understanding the brain and behavior Behavior Psychology Economics Computation Molecules, neurons, circuits and brain networks Neurobiology

3 Computational fmri N-armed bandit task?

4 Reinforcement-learning models Computational model: Reinforcement learning Prediction error: d(t) = R(t) - V(t)a Value update: V(t+1)a = V(t)a + ad(t) Choose (soft) max (Va, Vb, Vc) V d From Schultz, Montague and Dayan, 1997

5 Introduction to computational fmri n Objective: find brain activity correlating with variables from a computational model. n Provides account of how a particular cognitive function is being implemented in the brain, as opposed to identifying what brain areas are activated. n Can perform model-comparison: which model out of several models provides a better account of (a) behavior and (b) activity in a given brain area or across the brain?

6 Model-based procedure MODEL-BASED PROCEDURE: Find best-fitting model-parameters to behavior A t,a t+1,a t+2.a n Subject (Driver!) Model parameters α,β,γ etc. Take a left you idiot! Model (Back seat driver!) Minimize disagreement e.g. find model parameters that give the observed data the greatest probability under the model. {P a1 t, P aj t,}, {P a1 t+1, P aj t+1,},.., {P a1 n, P aj n,}

7 Model-based procedure MODEL-BASED PROCEDURE: Generate model timeseries

8 Model-based procedure Regress model-timeseries against fmri data

9 MODEL-BASED PROCEDURE: Compare different models Model A Model B Model C Compare likelihood of models given the data Need to penalize for model complexity èbayes information criterion (Adjusts likelihoods for # parameters) è Out of sample test / cross-validation è Bayesian model selection (Stephan et al., 2009)

10 Reward prediction errors in the mid-brain A. B. z = -12 z = -13 C. z = -14 A R L z = -15 P W. Pauli et al., Journal of Neuroscience, 2015

11 Reward prediction errors O Doherty et al., Neuron, 2003 O Doherty et al., Science, 2004 Kim et al., Plos Biology 2006

12 Value signals Expected value (action_probability) A L vmpfc vmpfc p<0.01 p<0.001 Daw et al. Nature, 2006 % signal change probability of chosen action Expected value B Kim et al. Plos Biology, 2006

13 Observational reward-learning With Jeff Cooper, Simon Dunne, Teresa Furey

14 Observational reward-learning Cooper, Dunne, Finnegan and O Doherty, JOCN 2011

15 Strategic interactions Hampton, Bossaerts and O Doherty, PNAS, 2008

16 Expected value Hampton, Bossaerts and O Doherty, PNAS, 2008

17 Integration with other modalities

18 Integrating computational fmri with TMS Figure 3. Representation of the influence update and TMS inhibition A. Representation of the influence update in the vertex condition. (FWE Cluster-corrected p = 0.05, cluster forming threshold selected as stimulation site. B. In line with the a priori hypothesis, we find a reduced representation of the influence update in the (FWE p < 0.05, small volume corrected within a 15mm sphere around rtpj peak shown in A ). On the right, Betas extracted illustrate the localized TMS-induced reduction of the influence update representation. B! A!! p!<!0.005! p!<!0.001! Hampton!peak!! FMRI x!=!39! Neural β B! vertex TMS TMS rtpj vertex rtpj With Christopher Hill; Christian Ruff; Shinsuke Suzuki

19 =.44, p = 0.017), supporting the notion that anticipating your opponent update in response to your actions plays a eaks down in the TMS-rTPJ condition (p = 0.016), suggesting that the parameter can no longer fulfill the same!! Causal role of psts/tpj in higher order learning 5. Value-driven representations in the vmpfc and rtpj-vmpfc connectivity. esentation of the expected value of the chosen option in the TMS-vertex group. We find significant activation in the vmpfc n 0, -32, 13). In the TMS-rTPJ condition, we do not find evidence for a value-driven signal in our search volume, even at a very a plot on the right of the B! panel (extracted from the cluster within our search volume at p = from A) to visualize th tation could be driven by altered processing of the influence update in the rtpj. To explore this possibility, we perform a κ parameter dngencies from 15mm spheres around 0.08 the individual rtpj peak representation of the influence with x!=!6!the influence-update parametricy!=m *! We find evidence for differential connectivity in the vmpfc between both psychophysiological interactions in our design matrix. vertex FWETMS corrected p < 0.05, SVC mm sphere on 0, -32, 13). This effect is visualized by the beta parameters plotted on the right TMS rtpj ur search volume at p = in B). C. Exploratory analysis reveals a positive relationship (Robust regression, p = 0.02) betwe ndition and the Kappa parameter, suggesting that alteration in rtpj-vmpfc connectivity might underlie the behavioral effect of 0.04 θ value Influence parameter x!=!6! Employee shirked at t-1 B! κ parameter C! TMS vertex TMS rtpj y!= 5 C! Points Points x!=!92! B! TMS vertex y!=!913! TMS rtpj x!=!92! TMS vertex TMS rtpj y!=!913! eural β (PPI) A! 0 Neural β *! 0.02 Influence parameter!! A! With Christopher Hill; Christian Ruff; Shinsuke Suzuki!

20 Combining lesions with computational fmri Controls - Amyg Expected Value p<0.01 p<0.001 Normalized Betas Controls SM Matched AP SM mpfc AN Hampton, R Adolphs, JM Tyszka, JP O'Doherty - Neuron, 2007

21 Computational EEG with computational fmri (a) time: ms (b) time: ms (c) time: ms anterior x = 24 y = 46 central posterior dorsal x = 6 y = 62 ventral left medial right left medial right left medial right Figure 5. Results of the analysis of the chosen value (a c). Same data as in figure 4, but aggregated over time windows and plotted on scalp maps. Top row shows transverse view, bottom row coronal view. Bigger dots mean more significant time bins in the time window. (Online version in colour.) Figure 6. fmri-informed source localization of chosen value with threshold set at p, (unc). (Online version in colour.) Larsen and O Doherty, Phil Trans Royal Society B, 2014

22 Conclusions

23 Conclusions n Computational fmri has proved very fruitful particularly in the domain of learning and decision-making, and is now beginning to make an impact in the social neuroscience domain. n While a powerful approach, as with any other method, has important limitations and depends on assumptions which may not always be valid. n Increasing utility to combining the approach across modalities: e.g. connectivity/network analyses, multivariate analyses, computational EEG/MEG, TMS, lesion studies, and neurophysiology.

24 Computational approaches for understanding the human brain. Thanks to the O Doherty lab