Receptor Theory and Biological Constraints on Value

Transcription

1 Receptor Theory and Biological Constraints on Value GREGORY S. BERNS, a C. MONICA CAPRA, b AND CHARLES NOUSSAIR b a Department of Psychiatry and Behavorial sciences, Emory University School of Medicine, Atlanta, Georgia, USA b Department of Economics, Emory University, Atlanta, Georgia, USA ABSTRACT: Modern economic theories of value derive from expected utility theory. Behavioral evidence points strongly toward departures from linear value weighting, which has given rise to alternative formulations that include prospect theory and rank-dependent utility theory. Many of the nonlinear forms for value assumed by these theories can be derived from the assumption that value is signaled by neurotransmitters in the brain, which obey simple laws of molecular movement. From the laws of mass action and receptor occupancy, we show how behaviorally observed forms of nonlinear value functions can arise. KEYWORDS: expected utility theory; prospect theory; neuroeconomics; reward; receptor theory INTRODUCTION The last 5 years have seen a vast increase in the number of neuroimaging papers that attempt to identify the neural code for value, whether described as a reward, a reinforcer, or by the economic term, utility. The concept of value that is investigated is a score that is relevant to behavior. An extreme view of the relationship between value and behavior is that of classical economics, which assumes that people make optimal decisions. Optimal decisions are those decisions that maximize the individual s value, or utility, they hope to obtain. This model provides a normative description of how perfectly rational agents behave. A behavioral economist might add assumptions of errors in actions, decision biases, or slow and incomplete learning in constructing descriptive models of behavior. A psychologist, on the other hand, might view decision making as the interaction between cognitive and affective forces. It is into this fray that neuroimagers have stepped. The hope is that measurement of brain activity will resolve these long-standing debates about the Address for correspondence: Gregory S. Berns, Department of Psychiatry and Behavorial Sciences, Emory University School of Medicine, 101 Woodruff Circle, Suite 4000, Atlanta, GA Voice: ; fax: gberns@emory.edu Ann. N.Y. Acad. Sci. 1104: (2007). C 2007 New York Academy of Sciences. doi: /annals

2 302 ANNALS OF THE NEW YORK ACADEMY OF SCIENCES varied motivations that people have for the decisions they make. But, like Alice s looking-glass, the answers seem to depend on who is asking the question. In this article, we propose a parsimonious structure to consider the biophysical constraints of how neuronal systems respond to varying levels of scalar quantities interpreted as value or, alternatively, as the determinants of value. As far as we know, neurons act as detectors of differences. That is, they fire when a change of membrane state occurs. They adapt to levels of stimulation, making them poor signalers of absolute levels of anything, be it intensity of visual stimulation, caloric intake, or value. If we consider value as a quantity, like photic stimulation, then it is possible to extrapolate, from the behavior of analogous neural systems, how the brain perceives value. We begin with the assumption that cognitive processing of value is influenced by the biophysical properties that govern neurotransmitter and receptor binding in the brain. Although decidedly reductionist, this assumption requires only that the brain is the physical organ that controls behavior and that the brain operates according to well-known laws of physics and chemistry. While the biophysical properties of the brain may not capture all the aspects of psychological processes, the physical structure of the brain does place specific constraints on the implementation of psychological processes. Our derivation suggests a hypothesis that measures of brain activation are biological transformations of stimuli, which can be interpreted as biological proxies for utility functions. With minimal assumptions, one can derive properties of a value function that capture many aspects of those postulated in theories of decision making, like expected utility theory 1 and prospect theory. 2 For instance, our model implies a diminishing marginal sensitivity to value and probability, which is consistent with the available evidence from economic experiments. RECEPTOR OCCUPANCY THEORY According to receptor occupancy theory, a biological response results from the interaction between a neurotransmitter and a cellular receptor. 3 The theory has its basis in the law of mass action. In chemical reaction notation, we consider a drug (or neurotransmitter), A, which binds to a receptor, R, and forms a drug receptor complex, AR, which acts as a stimulus to the cell. The result is a cellular response: A + R AR Stimulus Response The magnitude of this response can be described by [ Response = ε [R] [A] ] (1) K d + [A]

3 BERNS et al. 303 where [A] is the concentration of the neurotransmitter, [R] is the concentration of the receptor, K d is the dissociation constant, ε is the efficacy, and is an exponent that describes the stimulus-response relationship of the cell. If we assume that [A] is directly, and linearly, related to the magnitude of an exogenous entity, z, such as the amount of money, pain, or consumption that an individual is presented with or receives, and that the effective maximal receptor concentration, R max, equals ε [R], then [ ] Rmax z Response = (2) K d + z Thus, we make a critical distinction between the quantity, z, and the measurable response to z. By the same logic, z can represent any psychological or economic quantity, which is represented by a scalar quantity. There are two ways that utility can be interpreted in the context of Equation 2. The first is to interpret utility as z, the exogenous entity, and treat it as if it were a physical quantity that is transformed into a cellular response. The second is to interpret Equation 2 as describing a transformation of a physical quantity into utility. To the degree that mental constructs of value become physically instantiated in the brain, Equation 2 describes how this transformation might occur (with certain assumptions). Under either interpretation, the utility may be that of a certain event or the value assigned to a lottery, in which case it may take the expected utility or another functional form. The current neuroeconomic literature does not address the issue of which interpretation might be more appropriate. Although several studies have reported biological correlates of utility, it is not yet clear whether these measurements correspond to utility, 4 6 a version of instantaneous pleasure or anticipated pleasure (analagous to z in Eq. 2), or whether these measurements are, in fact, a response to a psychological construct of utility. For purposes of this article, we conjecture only that biological measurements are constrained by Equation 2, but it is worth considering the two possibilities separately. It is possible that the notion of value (or utility) is a mentally constructed quantity without physical instantiation in a single brain location. While still contained in the brain, a distributed representation might limit the practical application of Equation 2, which is fundamentally a description of the movement of molecules at a single synapse. A variety of studies, however, have suggested that the nucleus accumbens is a location that contains a centralized measure of utility. Although the proposed explanations differ from purely hedonic (experiential) utility, to expected utility, to temporal difference prediction (i.e., a derivative of expected utility), it is reasonable to ask how Equation 2 would transform these parameters into a physical response. Once transformed, this signal could be propagated to other brain regions that require access to value information.

4 304 ANNALS OF THE NEW YORK ACADEMY OF SCIENCES FIGURE 1. The neurotransmitter receptor response function. Assuming a linear cellular response ( = 1 in Eq. 2), the binding of a neurotransmitter to its receptor complex follows the law of mass action and leads to an asymptotic response, regardless of the level of input stimulation (Input Value). The dissociation constant, K d, is the level at which 50% of the receptor is occupied. In this plot, R max = 1. Consider the interpretation of Equation 2 under which z represents the objective quantity of some stimulus, but this quantity serves as an input to a specific neural system, for example, the ventral tegmental dopamine system, whose response would be subjectively experienced as utility. This is the strongest possible interpretation of Equation 2. In essence, this says that the response of the dopamine system corresponds to the subjective experience of utility. If utility determines decisions, then the correspondence to dopamine system activity means that dopamine release would govern decision making. If we assume that = 1, namely the stimulus-response relationship of the neuron is linear, then Equation 2 leads to the well-known concave utility function 7 (FIG. 1). Although the classical form for the utility function is strictly monotonically increasing and concave, which has the property of diminishing but strictly positive marginal returns, the hyperbolic form given by the law of mass action has the property of asymptotically approaching R max. This corresponds to the existence of a saturation point where increasing the quantity of stimulus no longer yields greater utility. Most neural systems detect transient changes, as in the visual system, in which neurons respond to changing levels of stimulation (as opposed to tonic, long-term levels of stimulation). A large body of data on the properties of the dopamine system also suggests that these neurons respond to deviations in reward prediction as opposed to absolute levels of reward Thus it seems reasonable to cast Equation 2 as a response to a change in future reward expectations: [ ] dz Rmax dt Response = K d + dz (3) dt

5 BERNS et al. 305 Thus, the greater the rate of change of expected value, the greater the response, up to the limit imposed by R max. The temporal derivative, dz/dt, can be integrated over an arbitrary time interval to yield the corresponding integrated response. Equation 3 can also be applied to negative rewards, or losses. Losses are likely governed by different neural systems, although dopamine activity has been shown to decrease when expected rewards are not received. Allowing R max to take on different magnitudes for gains and losses yields the well-known S-shaped value function postulated by prospect theory 2 (FIG. 2). The exponent,, describes the transformation of receptor activation into a cellular response. A cellular system that is weakly coupled to the receptor is characterized by > 1, and, conversely, a strongly coupled system has < 1. 3 Because the degree of receptor-coupling affects the dose-response relationship, also affects the shape of the value function (FIG. 3). EVIDENCE Our theory makes a strong assumption: that the response of a specific neural system is the biophysical carrier of utility. Although it is possible that utility is a distributed mental construct, without physical instantiation in a single location, it is worthwhile examining the evidence for the stronger prediction. The vast majority of the experimental work on the reward system has focused on the dopamine system and the brain regions to which it projects. To test Equation 3 in an experimental system, one would need to vary either the magnitude of rewards delivered to a test subject, or vary the contingencies such that the subject s expectations are changing. Both have been done. Monkeys, for example, can discriminate between the magnitudes of a liquid FIGURE 2. The neurotransmitter receptor response function when the input is considered as a change in value from an arbitrary baseline. Here, the R max for losses was set at twice the R max for gains. The function captures all the salient characteristics of the value function postulated in the prospect theory.

6 306 ANNALS OF THE NEW YORK ACADEMY OF SCIENCES FIGURE 3. The effect of the cellular response exponent,, on the neurotransmitterresponse function. Decreasing < 1 implies highly efficient coupling of the cellular response to the neurotransmitter receptor complex and results in a more convexoconcave response function. Increasing > 1 implies inefficient coupling and results in a more complex response function with three inflection points. reward based on the volume received. This is also reflected in corresponding levels of activity change in the midbrain dopamine neurons as well as their target sites in the striatum. 11,12 Importantly, the dopamine response displays a concave functional form, and changes in phasic dopamine activity correlate with future decision making. 13 Although there are comparatively fewer neuroimaging data in humans at a sufficient range of reward values, a recent study in which the dopamine system activity was either augmented by treatment with the dopamine precursor, L-DOPA, or blocked with the receptor antagonist, haloperidol, corresponding changes were observed in the fmri BOLD response in the striatum. 14 Numerous fmri studies have shown corresponding changes in striatal activity, that is, at least, monotonic in the magnitude of expected monetary reward These studies, however, have not offered a sufficient range of rewards to assess the curvature of the function. A milder interpretation of Equation 3 is that value and utility are mental constructs that occur through a complex set of psychological operations, and because of the complexity, are distributed across different brain regions. Receptor theory still applies in this scenario because neurons adhere to physical laws governing molecule movement. But the one-to-one correspondence with value breaks down because the input to each disparate brain region might represent only one aspect of the process of computing value. For example, a visually presented stimulus, say in the form of a number, must first be processed by the visual system before being passed on to any putative value system. Low-level

7 BERNS et al. 307 visual processing might reflect some aspects of value although it would be loose correlation at best. Animal studies use conditioning paradigms, so that the visual stimulus, in addition to its geometric properties, comes to contain value information through associative learning mechanisms. Value becomes reflected not only in dopamine neurons but parietal neurons as well It is not yet possible to determine which way value information flows between these two systems, but regardless of the direction, the information becomes transformed by the above equations. The nonlinearity of value functions is generally accepted as an empirical truism. Most explanations for the nonlinearity derive from psychological or cognitive distortions, which include framing effects, loss aversion, status quo bias, etc. 23,24 Many of the same distortions, however, have been demonstrated in other animals. A well-known motivating stimulus, that has behavioral effects akin to value, is electrical stimulation of certain brain regions. 25 Brainstimulation reward (BSR) can be titrated to replace conventional exogenous rewards like food. BSR, however, shows the same relationship to behavior as do other rewards; namely, as both the magnitude of electrical stimulation and its frequency are increased, the behavioral effect approaches an asymptote. 26 The role of dopamine in BSR is complicated by the fact that electrical stimulation of specific brain regions propagates in multiple directions, some of which eventually reach the midbrain dopamine neurons. Although BSR is more complicated than simply stimulating the dopamine system, the pure physicality of BSR eliminates the role of cognitive heuristics in the determination of the value function. And although multiple neurotransmitter systems are involved, they each obey receptor occupancy constraints. CONCLUSIONS To our knowledge, the physical constraints imposed by receptor occupancy theory have never been proposed as the cause of the functional forms associated with economic utility theories. The commonality across many species for concave forms of utility suggests a common biological mechanism that is independent of the cognitive heuristics used by each animal. Of course, the fact that neurotransmitters and receptors display binding kinetics with a functional form similar to that postulated by economic theory may be a coincidence. The main argument against such a causal relationship derives from the multiplicity of systems in the brain that operate in parallel to coordinate decision making. Nonlinearities in one system may be offset by nonlinearities in another system. However, the strength of our proposal is that it is testable. Manipulation of specific receptors in the context of decision making, either through pharmacological means or through genetic approaches, can isolate the contribution of each neurotransmitter receptor complex to the behaviorally derived value function.

8 308 ANNALS OF THE NEW YORK ACADEMY OF SCIENCES ACKNOWLEDGMENTS We are grateful for the contributions of Jonathan Chappelow, Sara Moore, and Giuseppe Pagnoni. This work was supported by grants from the National Institute on Drug Abuse (DA and DA20116). REFERENCES 1. VON NEUMANN, J.& O. MORGENSTERN Theory of Games and Economic Behavior. Princeton University Press. Princeton. 2. KAHNEMAN, D.& A. TVERKSY Prospect theory: an analysis of decision under risk. Econometrica 47: ROSS, E.M. & T.P. KENAKIN Pharmacodynamics. Mechanisms of drug action and the relationship between drug concentration and effect. In Goodman & Gilman s The Pharmacological Basis of Therapeutics, Vol. Tenth. J.G. Hardman & L.E. Limbird, Eds. McGraw-Hill. New York. 4. KNUTSON, B., J. TAYLOR, M. KAUFMAN, et al Distributed neural representation of expected value. J. Neurosci. 25: HSU, M., M. BHATT, R. ADOLPHS, et al Neural systems responding to degrees of uncertainty in human decision-making. Science 310: HUETTEL, S.A., C.J. STOWE, E.M. GORDON, et al Neural signatures of economic preferences for risk and ambiguity. Neuron 49: BERNOULLI, D. 1763/1958. Exposition of a new theory on the measurement of risk. Econometrica 22: SCHULTZ, W., P. DAYAN & P.R. MONTAGUE A neural substrate of prediction and reward. Science 275: LJUNGBERG, T., P. APICELLA & W. SCHULTZ Responses of monkey dopamine neurons during learning of behavioral reactions. J. Neurophys. 67: SCHULTZ, W.& A. DICKINSON Neuronal coding of prediction errors. Ann. Rev. Neurosci. 23: CROMWELL, H.C. & W. SCHULTZ Effects of expectations for different reward magnitudes on neuronal activity in primate striatum. J. Neurophys. 89: TOBLER, P.N., C.D. FIORILLO & W. SCHULTZ Adaptive coding of reward value by dopamine neurons. Science 307: MORRIS, G., A. NEVET, D. ARKADIR, et al Midbrain dopamine neurons encode decisions for future action. Nat. Neurosci. 9: PESSIGLIONE, M., B. SEYMOUR, G. FLANDIN, et al Dopamine-dependent prediction errors underpin reward-seeking behaviour in humans. Nature 442: KNUTSON, B., A. WESTDORP, E. KAISER, et al FMRI visualization of brain activity during a monetary incentive delay task. Neuroimage 12: KNUTSON, B., C.M. ADAMS, G.W. FONG, et al Anticipation of increasing monetary reward selectively recruits nucleus accumbens. J. Neurosci. 21: RC BREITER, H.C., I. AHARON, D. KAHNEMAN, et al Functional imaging of neural responses to expectancy and experience of monetary gains and losses. Neuron 30:

9 BERNS et al DELGADO, M.R., L.E. NYSTROM, C. FISSEL, et al Tracking the hemodynamic responses to reward and punishment in the striatum. J. Neurophys. 84: O DOHERTY, J., M.L. KRINGELBACH, E.T. ROLLS, et al Abstract reward and punishment representations in the human orbitofrontal cortex. Nat. Neurosci. 4: PLATT, M.L. & P.W. GLIMCHER Neural correlates of decision variables in parietal cortex. Nature 400: DORRIS, M.C. & P.W. GLIMCHER Activity in posterior parietal cortex is correlated with the relative subjective desirability of action. Neuron 44: SUGRUE, L.P., G.S. CORRADO & W.T. NEWSOME Matching behavior and the representation of value in the parietal cortex. Science 304: KAHNEMAN, D.& A. TVERKSY Choices, values, and frames. Am. Psychol. 39: TVERSKY, A.& D. KAHNEMAN Loss aversion in riskless choice: a referencedependent model. Quart. J. Econ. 106: SHIZGAL, P Neural basis of utility estimation. Curr. Opin. Neurobiol. 7: SIMMONS, J.M. & C.R. GALLISTEL Saturation of subjective reward magnitude as a function of current and pulse frequency. Behav. Neurosci. 108: