How rational are your decisions? Neuroeconomics Hecke CNS Seminar WS 2006/07
Motivation
Motivation Ferdinand Porsche "Wir wollen Autos bauen, die keiner braucht aber jeder haben will."
Outline 1 Introduction 2 Neural Coding of Utilities and Value Functions 3 The Role of Emotion in Decision Making 4 Social Decision Making 5 Learning and Decision Making 6 Mentalizing and Theory of Mind
Introduction Introduction homo economicus is widely assumed to be a rational and self-interested decision maker. in reality routines to approximate optimal decision making: emotionally driven experience based taking into account decisions of others estimate outcome of alternative actions Review: D. Lee: Neural basis of quasi-rational decision making. Current Opinion in Neurobiology 2006, 16: 191 198
Neural Coding of Utilities and Value Functions Neural Coding of Utilities and Value Functions utility experienced utility: subjective pleasure from the effect of an action decision utility: estimate of experienced utility for decision making brain regions that code for utility show neural activity that is affected by the amount of reward striatum, dorsolateral PFC, orbitofrontal cortex, anterior and posterior CC and posterior parietal cortex are also affected by anticipated outcomes while decision making
Neural Coding of Utilities and Value Functions Neural Coding of Utilities and Value Functions
Neural Coding of Utilities and Value Functions Matching Behavior in Monkeys Experiment poissonian reward different rates change without warning
Neural Coding of Utilities and Value Functions A local Version of Matching global matching needs perfect integration local matching Local fractional income relates to instantaneous probability of choice. leaky integrator: oblivion
Neural Coding of Utilities and Value Functions Utility coding delayed saccade task matching task Lateral Interparietal representation of fractional income of saccadic eye movements
The Role of Emotion in Decision Making The Role of Emotion in Decision Making expected utility theory maximizing the utilities weighted by their probabilities Allai s paradox choice 0 $ 1000 $ 5000 $ A 0 1 0 B 0.01 0.89 0.1 C 0.89 0.11 0 D 0.9 0 0.1 Too simple, does not take into account emotions. subjective bias towards certainty via nonlinear transformation regret theory minimization of regret regret: cognitive or emotional reaction associated with the realization that an unchosen action would have yielded better outcome arises from too high imagined outcome
The Role of Emotion in Decision Making Regret in Orbitofrontal Cortex 200-50 50-50 Choice 200 Wait 200 Wait Game value -50 50-50 200 Outcome -50 50-50 -50 50-50 Partial feedback Complete feedback Experiment choice of one gamble Spinning arrow holds at the outcome. The outcome of the alternative gamble is once shown, once not. Subjects rate their emotions. 200-50 50-50 Outcome -50-25 0 25 50 Affective rating Extremely sad Neither sad nor happy Extremely happy
The Role of Emotion in Decision Making Regret in Orbitofrontal Cortex A C Emotional rating Normal subjects (N=18) 50 40 30 20 10 0-10 -20-30 -40-50 -50 50 obtained outcome 50 40 30 B D Orbitofrontal patients (n=5) 50-200 unobtained 40 200 unobtained 30 20 10 Partial 0 feedback -10 (disappointme -20-30 -40-50 -50 50 obtained outcome 50 40 30 Both subjective rating and skin conductance reveal: normal subjects show huge regret lesioned people don t Emotional rating 20 10 0-10 -20-30 -40 20 10 0-10 -20-30 -40 Complete feedback (regret) -50-50 -50 50-50 50 obtained outcome obtained outcome E Skin conductance 1.2 1.0 0.8 0.6 0.4 0.2 F 1.2 1.0 0.8 0.6 0.4 0.2 partial complete -16mm -4mm -1mm 0 [-50,200] [50,200] [Obtained, non-obtained] 0 [-50,200] [50,200] [Obtained, non-obtained] Lesion Overlap
The Role of Emotion in Decision Making Emotion in Decision Making elation and disappointment arise if the outcome is irrespective of the chosen action not good for learning (if the goal is rational behavior) prospect theory choice outcomes are measured relative to the status quo this is supported by several monkey studies and weighted by a nonlinear function of the probabilities
Social Decision Making Social Decision Making game theory characterizes the problem of social interaction by a payoff matrix for multiple players and individual actions Classical game theory (rational decisions!) predicts the reaching of a Nash equilibrium. Nash equilibrium a set of strategies for all players from which no player can increase their payoffs by altering their strategies individually.
Social Decision Making Social Decision Making Ultimatum Game Trust Game
Social Decision Making Social Decision Making beyond Nash eq. ethical valuation of other s decisions not related to payoffs enhanced activation of certain brain regions only with human partners no effect with computer pertners
Social Decision Making Social Decision Making beyond Nash eq. Desire to punish uncooperative others activation of the caudate nucleus IC: decision to punish, punishment costly IF: decision to punish, punishment free IS: decision to send a (free) symbolic message NC: random action, punishment costly
Learning and Decision Making Learning and Decision Making reinforcement learning reward prediction error: discrepancy between predicted and actual reward used to update decision utilities or prediction functions dopamine dopamine neurons in the ventral tegmental area and substantia nigra encode reward prediction errors
Learning and Decision Making Learning and Decision Making belief theory enhancement of reinforcement learning belief of the strategy of the other players it is also updated regularly than maximizing payoff given the beliefs approaching reality belief theoretic algorithms need to alter every expected payoff if a belief is changed (stone/paper/scissors) mixed strategies seem to match behavior neural mechanisms poorly understood but: fmri evidence for increased activity in ventral striatum if reasoning about the other s strategy
Learning and Decision Making Self-Referential Thinking ECTED PROOF difference between 1st and 2nd order belief What do you think, he thinks, you wan t to do? in equilibirium, (certainty) no change in neural activity posterior and anterior cingulate regions, frontal insula, dorsolateral PFC for theory of mind 2nd order beliefs activate the anterior insula mixture of forming beliefs and making choices
Mentalizing and Theory of Mind Mentalizing and Theory of Mind Mentalizing repeated playing with the same other players accumulation of information about the behaviors theory of mind is the ability to represent mental states of others: beliefs desires knowledge theory of mind might play a key role in optimizing decision making strategies during social interactions mostly unique to humans
Mentalizing and Theory of Mind Prisonners Dilemma Game Two players choose to Cooperate or to Defect. They learn about the other s choice afterwards. They get reward according to the matrix on the left.
Mentalizing and Theory of Mind Prisonners Dilemma Game Neural correlates for a theory of mind right mid posterior superior temporal sulcus yellow means more activation with human partners compared to computers in CC and CD outcomes 20 consecutive rounds with the same partner
Mentalizing and Theory of Mind Conclusions the assumption of rationality is false but useful to analyze complex decision making problems to generate hypotheses about corresponding cognitive and neural processes studies of emotion based choices and learning in decision making help to link formal theories to reality additionally, neurobiological studies reveal neural bases