Decision neuroscience seeks neural models for how we identify, evaluate and choose

Transcription

1 VmPFC function: The value proposition Lesley K Fellows and Scott A Huettel Decision neuroscience seeks neural models for how we identify, evaluate and choose options, goals, and actions. These processes which have long histories of study within economics and psychology are assumed to reflect decision-makers attempts to maximize the subjective value they obtain from their actions. But what do we mean by value? Value presents conundrums for theoreticians and empiricists alike. It is defined operationally, based on the decisions that a given person makes. When we choose one consumer good over its competitor, it is because the former good was of higher value (or utility, in the parlance of economics) 1. When sets of decisions are examined, it may be clear that one feature (e.g., taste of snack foods) exerts more influence on an individual s decisions than another feature (e.g., healthfulness) thus contributing more to subjective value. The subjectivity of value means that it must be measured indirectly, usually inferred from a pattern of choices or quantified by how hard someone will work for a given outcome 2. Considered in such ways, value seems tautological, an abstraction that provides a label for the motivations of a specific decision-maker who considers specific features of a decision scenario at a specific moment in time. Yet, value need not be only operationally defined. Research in decision neuroscience has identified physical markers for value in specific brain regions, notably vmpfc 3,4 but also including striatum, thalamus, and more dorsal portions of the medial frontal lobe 5. Activity of neurons in vmpfc is sensitive to variables that are known to influence economic behavior, such as decision framing, delay, and risk 6,7. Like subjective value itself, signals

2 measured within vmpfc track external information (e.g., other available options), internal states (e.g., hunger and satiety), and outcomes obtained by others (e.g., regret, envy), among many factors 8,9. Value signals in vmpfc have been identified using both electrophysiological and functional neuroimaging methods, for both positive and negative outcomes, and across a very wide range of decisions 3,5,9,10. Essentially any situation that involves decisions among competing goals can elicit vmpfc activation: economic risk and benefit in a stock market pick, sexual attraction in social networks, or larger or smaller food rewards. Conversely, lesions to vmpfc make value-based decisions more inconsistent 11, and affect the ability to learn from value information, especially under dynamic conditions such as when stimulus-reward contingencies are probabilistic or reversing 12,13. This flexibility has led to the conjecture that signals in vmpfc serve as a common currency that allows comparison across disparate options 3,6. Supporting the common currency idea, subjective value signals have been found in vmpfc across a diverse range of tasks and stimuli, ranging from food or social stimuli with which experimental subjects have extensive prior experience to abstract, even entirely novel stimuli for which value must be learned or inferred de novo in the context of the experiment 14. These subjective value signals may influence or be influenced by signals elsewhere in the brain that track other quantities (e.g., physical salience, selective attention, or arousal), but can be distinguished from those signals 2,4. Thus, a substantial body of converging evidence supports the claim that vmpfc represents subjective value. This simple claim has considerable explanatory power and elegantly links neural signals to concepts from economics and psychology. As such, it provides the best current account for a (single) functional role of vmpfc. Yet, this simple and straightforward

3 story vmpfc encodes subjective value is not yet complete. It should be considered a starting point that raises more questions about vmpfc function than it answers, providing direction for future work. First, there is uncertainty about anatomical specificity: different methods have pointed to different sub-regions within the broader ventral frontal lobe as important, and more work is needed to reconcile the findings across methods in humans, and across species 7,15. Amongst other issues, the common currency view of vmpfc in humans might be an artifact of the coarse level of resolution of human neuroscience methods: population activity may reflect domain-general value, but individual neurons could respond only to specific stimulus classes 6,16. Second, there is a need to get clearer about the behavioral construct of subjective value itself, at both conceptual and experimental levels. Decision neuroscience draws on wellelaborated formalisms from reinforcement learning theory, economics, and ecology 1,17,18, and has applied these approaches in computational models to useful effect. Even so, the field needs models that can address broader classes of decision behavior including those addressed by the other contributors to this Viewpoint. Suppose that value signals are observed all the way from concrete assessment of physical goods to an abstract sense of rightness in recalling a particular memory. Would that mean that decision neuroscience has identified the appropriate concept? Or, would value become too broad to be useful? Clearly, vmpfc is not needed simply to orient an organism towards reward, or away from harm. Rather, it seems to be particularly engaged when value assessment is somehow hard : when it requires integration, interpretation, or rapid updating 9,12,15. For sustained

4 progress, research will need to move from such descriptive accounts towards specific brain mechanisms, efforts already underway. Finally, vmpfc needs to be understood within the broader circuits that underlie motivated behavior. Progress has been made in understanding how vmpfc interacts with ventral striatum and amygdala in relation to reward learning and fear conditioning, and with hippocampus in relation to memory retrieval 19,20. Yet, less is known about its interplay with other prefrontal and subcortical regions, although these are likely to be of high relevance for understanding the mechanisms by which vmpfc supports context-sensitive value assessment. How do vmpfc signals influence behavior, and does the underlying mechanism vary across decision problems? Emerging views on this question include a role in shaping dopaminergic responses to value-predictive cues (e.g., by altering response gating in the striatum) or by biasing attention towards motivationally important aspects of the environment (e.g., by influencing posterior cortical areas via the thalamus). All told, research addressing vmpfc from the perspective of value has been profitable to date. Many questions remain, but on-going efforts to define formal models based on an interdisciplinary understanding of motivated behavior, and testing these models with converging approaches, provide a high-yield way forward.

5 References 1. Camerer, C. F. Neuroeconomics: Opening the Gray Box. Neuron 60, (2008). 2. O Doherty, J. P. The problem with value. Neurosci. Biobehav. Rev. 43, (2014). 3. Levy, D. J. & Glimcher, P. W. The root of all value: a neural common currency for choice. Current Opinion in Neurobiology (2012). doi: /j.conb Kahnt, T., Park, S. Q., Haynes, J.-D. & Tobler, P. N. Disentangling neural representations of value and salience in the human brain. Proc. Natl. Acad. Sci. U. S. A. 111, (2014). 5. Bartra, O., McGuire, J. T. & Kable, J. W. The valuation system: a coordinate-based meta-analysis of BOLD fmri experiments examining neural correlates of subjective value. Neuroimage 76, (2013). 6. Padoa-Schioppa, C. Neurobiology of Economic Choice: A Good-Based Model. Annu. Rev. Neurosci. 34, (2011). 7. Wallis, J. D. Cross-species studies of orbitofrontal cortex and value-based decisionmaking. Nature Neuroscience 15, (2011). 8. Rudebeck, P. H. & Murray, E. a. The Orbitofrontal Oracle : Cortical Mechanisms for the Prediction and Evaluation of Specific Behavioral Outcomes. Neuron 84, (2014). 9. Stalnaker, T. T. a, Cooch, N. N. K. & Schoenbaum, G. What the orbitofrontal cortex does not do. Nat Neurosci 18, (2015). 10. Winecoff, A. et al. Ventromedial Prefrontal Cortex Encodes Emotional Value. J. Neurosci. 33, (2013). 11. Henri-Bhargava, A., Simioni, A. & Fellows, L. K. Ventromedial frontal lobe damage disrupts the accuracy, but not the speed, of value-based preference judgments. Neuropsychologia 50, (2012). 12. Tsuchida, A., Doll, B. B. & Fellows, L. K. Beyond Reversal: A Critical Role for Human Orbitofrontal Cortex in Flexible Learning from Probabilistic Feedback. J. Neurosci. 30, (2010). 13. Walton, M. E., Behrens, T. E. J., Noonan, M. P. & Rushworth, M. F. S. Giving credit where credit is due: orbitofrontal cortex and valuation in an uncertain world. Ann. N. Y. Acad. Sci. 1239, (2011). 14. Xie, J. & Padoa-Schioppa, C. Neuronal remapping and circuit persistence in economic decisions. Nat. Neurosci. 19, (2016). 15. Xia, C., Stolle, D., Gidengil, E. & Fellows, L. K. Lateral orbitofrontal cortex links social impressions to political choices. J Neurosci 35, (2015). 16. Rich, E. L. & Wallis, J. D. Decoding subjective decisions from orbitofrontal cortex. Nat. Neurosci. 19, (2016). 17. Kolling, N., Behrens, T. E. J., Mars, R. B. & Rushworth, M. F. S. Neural mechanisms of

6 foraging. Science 336, 95 8 (2012). 18. Doll, B. B., Simon, D. A. & Daw, N. D. The ubiquity of model-based reinforcement learning. Curr. Opin. Neurobiol. 22, (2012). 19. Rudebeck, P. H., Mitz, A. R., Chacko, R. V & Murray, E. A. Effects of amygdala lesions on reward-value coding in orbital and medial prefrontal cortex. Neuron 80, (2013). 20. Hebscher, M. & Gilboa, A. A boost of confidence: The role of the ventromedial prefrontal cortex in memory, decision-making, and schemas. Neuropsychologia 90, (2016).