How rational are your decisions? Neuroeconomics

Transcription

1 How rational are your decisions? Neuroeconomics Hecke CNS Seminar WS 2006/07

2 Motivation

3 Motivation Ferdinand Porsche "Wir wollen Autos bauen, die keiner braucht aber jeder haben will."

4 Outline 1 Introduction 2 Neural Coding of Utilities and Value Functions 3 The Role of Emotion in Decision Making 4 Social Decision Making 5 Learning and Decision Making 6 Mentalizing and Theory of Mind

5 Introduction Introduction homo economicus is widely assumed to be a rational and self-interested decision maker. in reality routines to approximate optimal decision making: emotionally driven experience based taking into account decisions of others estimate outcome of alternative actions Review: D. Lee: Neural basis of quasi-rational decision making. Current Opinion in Neurobiology 2006, 16:

6 Neural Coding of Utilities and Value Functions Neural Coding of Utilities and Value Functions utility experienced utility: subjective pleasure from the effect of an action decision utility: estimate of experienced utility for decision making brain regions that code for utility show neural activity that is affected by the amount of reward striatum, dorsolateral PFC, orbitofrontal cortex, anterior and posterior CC and posterior parietal cortex are also affected by anticipated outcomes while decision making

7 Neural Coding of Utilities and Value Functions Neural Coding of Utilities and Value Functions

8 Neural Coding of Utilities and Value Functions Matching Behavior in Monkeys Experiment poissonian reward different rates change without warning

9 Neural Coding of Utilities and Value Functions A local Version of Matching global matching needs perfect integration local matching Local fractional income relates to instantaneous probability of choice. leaky integrator: oblivion

10 Neural Coding of Utilities and Value Functions Utility coding delayed saccade task matching task Lateral Interparietal representation of fractional income of saccadic eye movements

11 The Role of Emotion in Decision Making The Role of Emotion in Decision Making expected utility theory maximizing the utilities weighted by their probabilities Allai s paradox choice 0 $ 1000 $ 5000 $ A B C D Too simple, does not take into account emotions. subjective bias towards certainty via nonlinear transformation regret theory minimization of regret regret: cognitive or emotional reaction associated with the realization that an unchosen action would have yielded better outcome arises from too high imagined outcome

12 The Role of Emotion in Decision Making Regret in Orbitofrontal Cortex Choice 200 Wait 200 Wait Game value Outcome Partial feedback Complete feedback Experiment choice of one gamble Spinning arrow holds at the outcome. The outcome of the alternative gamble is once shown, once not. Subjects rate their emotions Outcome Affective rating Extremely sad Neither sad nor happy Extremely happy

13 The Role of Emotion in Decision Making Regret in Orbitofrontal Cortex A C Emotional rating Normal subjects (N=18) obtained outcome B D Orbitofrontal patients (n=5) unobtained unobtained Partial 0 feedback -10 (disappointme obtained outcome Both subjective rating and skin conductance reveal: normal subjects show huge regret lesioned people don t Emotional rating Complete feedback (regret) obtained outcome obtained outcome E Skin conductance F partial complete -16mm -4mm -1mm 0 [-50,200] [50,200] [Obtained, non-obtained] 0 [-50,200] [50,200] [Obtained, non-obtained] Lesion Overlap

14 The Role of Emotion in Decision Making Emotion in Decision Making elation and disappointment arise if the outcome is irrespective of the chosen action not good for learning (if the goal is rational behavior) prospect theory choice outcomes are measured relative to the status quo this is supported by several monkey studies and weighted by a nonlinear function of the probabilities

15 Social Decision Making Social Decision Making game theory characterizes the problem of social interaction by a payoff matrix for multiple players and individual actions Classical game theory (rational decisions!) predicts the reaching of a Nash equilibrium. Nash equilibrium a set of strategies for all players from which no player can increase their payoffs by altering their strategies individually.

16 Social Decision Making Social Decision Making Ultimatum Game Trust Game

17 Social Decision Making Social Decision Making beyond Nash eq. ethical valuation of other s decisions not related to payoffs enhanced activation of certain brain regions only with human partners no effect with computer pertners

18 Social Decision Making Social Decision Making beyond Nash eq. Desire to punish uncooperative others activation of the caudate nucleus IC: decision to punish, punishment costly IF: decision to punish, punishment free IS: decision to send a (free) symbolic message NC: random action, punishment costly

19 Learning and Decision Making Learning and Decision Making reinforcement learning reward prediction error: discrepancy between predicted and actual reward used to update decision utilities or prediction functions dopamine dopamine neurons in the ventral tegmental area and substantia nigra encode reward prediction errors

20 Learning and Decision Making Learning and Decision Making belief theory enhancement of reinforcement learning belief of the strategy of the other players it is also updated regularly than maximizing payoff given the beliefs approaching reality belief theoretic algorithms need to alter every expected payoff if a belief is changed (stone/paper/scissors) mixed strategies seem to match behavior neural mechanisms poorly understood but: fmri evidence for increased activity in ventral striatum if reasoning about the other s strategy

21 Learning and Decision Making Self-Referential Thinking ECTED PROOF difference between 1st and 2nd order belief What do you think, he thinks, you wan t to do? in equilibirium, (certainty) no change in neural activity posterior and anterior cingulate regions, frontal insula, dorsolateral PFC for theory of mind 2nd order beliefs activate the anterior insula mixture of forming beliefs and making choices

22 Mentalizing and Theory of Mind Mentalizing and Theory of Mind Mentalizing repeated playing with the same other players accumulation of information about the behaviors theory of mind is the ability to represent mental states of others: beliefs desires knowledge theory of mind might play a key role in optimizing decision making strategies during social interactions mostly unique to humans

23 Mentalizing and Theory of Mind Prisonners Dilemma Game Two players choose to Cooperate or to Defect. They learn about the other s choice afterwards. They get reward according to the matrix on the left.

24 Mentalizing and Theory of Mind Prisonners Dilemma Game Neural correlates for a theory of mind right mid posterior superior temporal sulcus yellow means more activation with human partners compared to computers in CC and CD outcomes 20 consecutive rounds with the same partner

25 Mentalizing and Theory of Mind Conclusions the assumption of rationality is false but useful to analyze complex decision making problems to generate hypotheses about corresponding cognitive and neural processes studies of emotion based choices and learning in decision making help to link formal theories to reality additionally, neurobiological studies reveal neural bases