brain valuation & behavior 9
Rangel, A, et al. (2008) Nature Neuroscience Reviews Vol 9
Stages in decision making process Problem is represented in the brain Brain evaluates the options Action is selected Evaluate the results of our actions
Learning updates the representation, the valuation and the actionselection processes. 154
Valuation key to the decision making process WATER WATER BOTTLED WATER BOTTLED WATER FLAVORED WATER FLAVORED WATER Motivation is a reason or a set of reasons for engaging in a particular behavior. 155
Select between different foods, make a decision, you eat it, and based on your experience, you learn, and then modify your valuation associated with this food 156
157
158
159
160
161
162
Subjective Value Is the average firing rate of a population of neurons coding behavioral preferences. An object has subjective value if it is a reward or a punishment.
Reward is something for which an animal will work. Punishment is something an animal will work to escape or avoid. The object (the dog treat) only has value if the dog will work in order to have the treat.
Skinner Box: Operant Conditioning 170
Frequency and speed of response = subjective value. 171
ICSS (intracranial self stimulation) Pleasure Circuit A rat pressing a lever in order to obtain stimulation reward. Olds and Milner (1958) 172
Rats were placed in a large levered Skinner box (Operant Conditioning Chamber) on 2 consecutive days and were subjected to both an acquisition period and an extinction period. During the 3 hour acquisition period, whenever the rat pressed a lever it would connect a circuit and activate current flow into an electrode placed at a particular location in the rat s brain. Then, during the 30 minute extinction period, the lever was deprived of its special stimulatory powers; when the rat pressed the lever, no current would flow. During both the acquisition and extinction periods, the percentage of time that rats spent lever pressing was recorded. (This percentage measurement was calculated by determining whether or not a rat had pressed a lever at least once during a given 30 second interval.) Olds and Milner then determined whether rats pressed the lever at different rates during the acquisition phase compared to the extinction phase, and asked whether this was related to the specific brain area being stimulated. They reasoned that if the rats pressed at higher rates during acquisition, then stimulation of this brain area must reflect some type of positive, rewarding sensation for the rat. http://web.stanford.edu/group/neurostudents/cgi bin/wordpress/?p=3733 Olds, J. & Milner, P., 1954. Journal of Comparative and Physiological Psychology, 47(6), pp.419 427. 173
Animals would attack the lever with singular focus, pressing repeatedly and forcefully at high rates. Animals would self-stimulate to exhaustion and die because they would not sleep, feed or drink water. Peter Shizgal, Brain Stimulation Reward 174
https://www.youtube.com/watch?v=uofqplulv9a 175
reward prediction error 10
The OFC and NA connection: OFC: Orbital frontal cortex NA: Nucleus Accumbens PUT: Putamen CAU: Caudate 178
Salgado and Kaplitt (2015) Stereotact Funct Neurosurg 93:75-93
Afferent and efferent connections to NAc. Salgado and Kaplitt (2015) Stereotact Funct Neurosurg 93:75-93
181 Schultz, W. (2000) NATURE REVIEWS NEUROSCIENCE VOLUME 1
The food is invisible to the monkey but the monkey can touch the food by placing its hand underneath the protective cover. The peri event time histogram of the neuronal impulses is shown above the raster display, in which each dot denotes the time of a neuronal impulse in reference to movement onset (release of resting key). Each horizontal line represents the activity of the same neuron on successive trials, with the first trials presented at the top and the last trials at the bottom of the raster display. a Touching food reward in the absence of stimuli that predict reward produces a brief increase in firing rate within 0.5 s of movement initiation. Schultz, W. (2000) NATURE REVIEWS NEUROSCIENCE VOLUME 1 182
Touching a piece of apple (top) enhances the firing rate but touching the bare wire or an inedible object that the monkey had previously encountered does not. The traces are aligned to a temporal reference point provided by touching the hidden object (vertical line). Schultz, W. (2000) NATURE REVIEWS NEUROSCIENCE VOLUME 1 183
Dopamine neurons encode an error in the temporal prediction of reward. The firing rate is depressed when the reward is delayed beyond the expected time point (1 s after lever touch). The firing rate is enhanced at the new time of reward delivery whether it is delayed (1.5 s) or precocious (0.5 s). The three arrows indicate, from left to right, the time of precocious, habitual and delayed reward delivery. The original trial sequence is from top to bottom. Data are from a two picture discrimination task. Schultz, W. (2000) NATURE REVIEWS NEUROSCIENCE VOLUME 1 184
Schultz, W. (2006) Annu. Rev. Psychol. 57:87 115
Basic assumptions of animal learning theory defining the behavioral functions of rewards. Contiguity refers to the temporal proximity of a conditioned stimulus (CS), or action, and the reward. Specifically, a reward needs to follow a CS or response by an optimal interval of a few seconds, whereas rewards occurring before a stimulus or response do not contribute to learning (backward conditioning). Schultz, W. (2006) Annu. Rev. Psychol. 57:87 115
Reward and prediction of reward in N. Accumbens Schultz, W. (2006) Annu. Rev. Psychol. 57:87 115 Conditioned Stimulus Reward