A Ra%onal Perspec%ve on Heuris%cs and Biases. Falk Lieder, Tom Griffiths, & Noah Goodman Computa%onal Cogni%ve Science Lab UC Berkeley

Transcription

1 A Ra%onal Perspec%ve on Heuris%cs and Biases Falk Lieder, Tom Griffiths, & Noah Goodman Computa%onal Cogni%ve Science Lab UC Berkeley

2 Outline 1. What is a good heuris%c? How good are the heuris%cs that people use? 2. How do people decide when to use which heuris%c? Where do heuris%cs come from?

3 Understanding the bounded mind 1. Ra%onal Use of Cogni%ve Resources 2. Ra%onal Metareasoning Biases Heuris%cs Strategy Discovery Strategy Selec%on Cogni%ve Control Bounded Ra%onality Reverse- Engineering Theore%cal Framework Norms Bounded Op%mality

4 problem Computa)onal Level (unbounded) op)mality solu)on Ra)onal Process Models resource- constraints idealized computa)onal architecture more realis)c computa)onal architecture problem + cogni)ve resources bounded op)mality bounded op)mality Algorithmic Level bounded op)mality idealized process model more realis)c process model representa)ons + cogni)ve processes

5 1. Resource- Ra%onal Analysis (Griffiths, Lieder, & Goodman, 2015) Computa%onal Architecture Computa%onal Level Theory Ra%onal Process Model 1. Specify Func%on 5b. Stop Experiments & Empirical Data

6 Example: Decision making under uncertainty argmax a E[ u(o) a, e] Computa%onal Architecture Computa%ons: 1. simulate s outcomes of each ac%on 2. average u%lity of simulated outcomes 3. choose ac%on Ra%onal Process Model 1. Specify Func%on 5b. Stop Experiments & Empirical Data

7 Step 1: Specify the func%on Decision making under uncertainty: a = argmax a E[ u(o) a, e]

8 Step 2: Computa%onal Architecture Computa%ons: 1. Simulate outcomes: 2. Average u%li%es: o ( a ),!,o ( a ) 1 s ( a ) ( a û a = f (o 1,!,o s )) 3. Choose ac%on with max. avg. u%lity: Limited computa%onal resources: limited %me à #simula%ons s ~ q(o a) a = argmax a û a

9 Step 3: Bounded Op%mality: Allocate simula%ons to outcomes vs. bias variance

10 Step 3: Bounded Op%mality U%lity- Weighted Sampling minimizes mean squared error of u%lity es%mate: simula%on frequency probability!q(o) p(o) u(o) extremity Lieder, Hsu, Griffiths (2014)

11 Step 3: Bounded Op%mality U%lity es%ma%on by importance sampling: o 1, o 2, o 3 ~ simulated outcomes q EU es%mates decision p q fun death w 1 =p(o 1 )/ q(o 1 ) w 3 =p(o 3 )/ q(o 3 )

12 Step 4: Evaluate Model Predic%ons Events with extremely high or extremely low u%lity 1. come to mind more easily (heightened availability) 2. are judged to occur more frequently (availability bias in frequency es%ma%on) 3. are overweighted in decision- making (availability bias in DM)

13 Experimental Test (Ludvig, et al., 2014) Madan, et al., 2015)

14 Predic%on 1: Extreme outcomes are more available Which outcome comes to mind first? Extreme First - Moderate First (%) UWS Captures Bias in Memory Rec Gains Losses People UWS

15 Predic%on 2: Extreme outcomes are overes%mated (Madan et al. 2014) How often did this door lead to each outcome? +40: % +20: % 0: % UWS Captures Bias in Frequency F Estima 40 Gains Losses Est. Extreme - Est. Moderate (%) -10 People UWS

16 Prediction 3: Extreme outcomes are overweighted in DM Risky Choice in % Ludvig et al. (2014), Experiments /0 vs /5 vs Gain Trials, People Gain Trials, UWS -40/ 0 vs. -20 Loss Trials, People Loss Trials, UWS -45/-5 vs Block Number

17 Model Predic%ons Events with extremely high or extremely low u%lity 1. come to mind more easily (high availability) 2. are judged to occur more frequently 3. are overweighted in decision- making

18 Example: Decision making under uncertainty argmax a E[ u(o) a, e] Computa%onal Architecture Computa%ons: 1. simulate s outcomes of each ac%on 2. average EU of simulated outcomes 3. choose ac%on Ra%onal Process Model 1. Specify Func%on Experiments & Empirical Data

19 1. Resource- Ra%onal Analysis (Griffiths, Lieder, & Goodman, 2015): Computa%onal Architecture Computa%onal Level Theory Ra%onal Process Model 1. Specify Func%on 5b. Stop Experiments & Empirical Data

20 Conclusion: There is hope for human ra%onality. Bounded Op%mality Reverse- Engineering

21 Conclusion 1. Heuris%cs can be derived by resource- ra%onal analysis. 2. The availability bias may be consistent with bounded op%mality. 3. Heuris%cs and biases are a window on ra%onal informa%on processing rather than a sign of irra%onality.

22 Leveraging BO and RM to make sense of human (i)ra%onality 1. Ra%onal Use of Cogni%ve Resources 2. Ra%onal Metareasoning Biases Heuris%cs Strategy Discovery Strategy Selec%on Cogni%ve Control Bounded Ra%onality Bounded Op%mality

23 Outline 1. What is a good heuris%c? How good are people s heuris%cs? 2. How do people decide when to use which heuris)c? Where do heuris)cs come from?

24 2. Strategy Selec%on as Metareasoning Calculate(EV( Habits( Planning( Strategies( Heuris>cs( TTB?( WADD?( UWS?( Pavlovian( impulses( Decision( Mechanisms( Strategies Decision Problem

25 Metacogni%ve Reinforcement Learning state s t+1 reward r t Agent informa%on informa%n state i t+1 i t internal reward r t Cogni%ve Control System Decision System control signal c t (decision strategy) state s t Environment ac%on a t

26 The problem with metareasoning

27 A solu%on: Learn a simple metacogni%ve model that enables efficient predic%ons! chosen heuris%c h * = argmax h Value Of Computa%on VOC(h,i) + - Expected Gain in Reward Cost(h; f) Learning Internal model f features of decision problem

28 features of decision problem A solu%on: Learn a simple model of the VOC VOC(h;i) E R h, f [ ] γ E T h, f [ ] VOC 1 - γ k E[ R h, f ] = w (R) k,s f k W :,c (R) ˆR ˆT k E[ T h, f ] (T = w ) k,s f k W :,c (R) f 1 f 2 f 3 f n

29 features of decision problem Learning an efficient approxima%on VOC(c;i) E R c, f [ ] γ E T c, f [ ] VOC 1 - γ Learning k E[ R c, f ] = w (R) k,s f k Learning W :,c (R) ˆR ˆT k E[ T c, f ] (T = w ) k,s f k W :,c (R) f 1 f 2 f 3 f n

30 Strategy Selec%on in DM Weighted- Addi)ve (WADD) vs. Take- The- Best (TTB) Σ

31 Rieskamp & Oio (2006) Condi%on 1: Compensatory Problems (TTB fails) Condi%on 2: Noncompensatory Problems (TTB succeeds)

32 People Learn to Choose Heuris%cs that match the structure of their environment (Rieskamp & Oio,2006) 0.8 Prob. of choosing TTB Noncompensatory (ModBaSS) Noncompensatory (People) Noncompensatory (SSL) Compensatory (ModBaSS) Compensatory (People) Compensatory (SSL) Block Number

33 Mixed Environment (Lieder & Griffiths, 2015) Optimal Choices in % with 95% CI Feature-based strategy selection SSL RELACS People Compensatory Trials in %

34 Summary 1. What is a good heuris%c? How good are people s heuris%cs? 2. How do people decide when to use which heuris%c? Where do heuris%cs come from?

35 Outlook: Towards a theory of bounded ra%onality. 1. Ra%onal Use of Cogni%ve Resources 2. Ra%onal Metareasoning Biases Heuris%cs Strategy Discovery Strategy Selec%on Cogni%ve Control Bounded Ra%onality Reverse- Engineering Theore%cal Framework Norms Bounded Op%mality

36 Conclusions 1. Bounded op%mality can be used to derive ra%onal heuris%cs from assump%ons about the mind s computa%onal architecture, the agent s goal, and the structure of the environment. 2. Ra%onal metareasoning can be used to model how people decide when to use which heuris%c. 3. Together, ra%onal heuris%cs and ra%onal strategy selec%on, provide a beier norma%ve standard to evaluate human ra%onality. 4. Alleged evidence for human irra%onality should be evaluated against the standard of bounded op%mality.

37 Thank you! Tom Griffiths Noah Goodman Ming Hsu Dillon Plunkei Jessica Hamrick Stuart Russell Thomas Icard Amitai Shenhav Sebas%an Musslick Daniel Reichman Nick Hay