Structuring and Behavioural Issues in : Part II: Biases and Risk Modelling Theodor J Stewart Department of Statistical Sciences University of Cape Town Helsinki Part II 1 / 45
and Helsinki Part II 2 / 45
Decision Psychology and Trail-blazing work of Kahneman & Tversky Largely related to perception and assessment of risks... But many perceptions and judgements required in seemingly of an identical nature (e.g. weights, pairwise preferences)... Even though direct empirical experiments in the context largely lacking. First review some key Kahneman-Tversky results (with acknowledgement to John Maule of Leeds University Business School) Helsinki Part II 3 / 45
Framing Problem 1 and A large car manufacturer has been hit with economic difficulties and it appears as if three plants need to be closed and 6000 employees laid off. The vice president of production has been exploring alternatives to avoid this crisis, and has developed two plans: Plan A: will save one of the three plants and 2000 jobs Plan B: has a 1/3 probability of saving all three plants (and 6000 jobs), but has a 2/3 probability of saving no plants and no jobs Which plan do you prefer? Helsinki Part II 4 / 45
Framing Problem 2 and A large car manufacturer has been hit with economic difficulties and it appears as if three plants need to be closed and 6000 employees laid off. The vice president of production has been exploring alternatives to avoid this crisis, and has developed two plans: Plan A: will result in the loss of two of the three plants and 4000 jobs Plan B: has a 2/3 probability of resulting in the loss of all three plants and all 6000 jobs, but has a 1/3 probability of losing no plants and no jobs Which plan do you prefer? Helsinki Part II 5 / 45
Framing and Prospect Theory and Previous problems reveal that people are risk seeking for losses and risk averse for gains, across a wide range of problems and for professional and naive decision makers. Basis for Prospect Theory which models the decision making process by two phases editing and evaluation Editing: Forming an internal representation of the problem called the decision frame which leads to perceived worth of outcomes Prospect theory argues that this coding is not absolute, but is relative to a reference point formed as part of the framing. Helsinki Part II 6 / 45
Framing and Prospect Theory (cont.) and Problem 1 starts from the position of all jobs lost, so that the choice is between jobs saved (i.e. gains) Problem 2 starts from the position of no closures, so that the choice is between jobs lost (i.e. losses) People prefer to avoid sure losses, and to avoid risks of forgoing sure gains, leading to convex value functions for losses and concave functions for gains (i.e. S -shaped through the reference point) Slope of the value function is steeper over losses Evaluation is based on the preferences implied by the framing Leads to inconsistencies as there is no correct framing, and reference points can drift Helsinki Part II 7 / 45
Implications of Prospect Theory and Framing the value of outcomes of decisions depends upon the perceived reference point; Most decisions have several possible points of reference; The context of the decision biases people to adopt one rather than another reference point; Losses loom more strongly than gains! Reference points may change depending upon whether the current situation is aggregated or segregated from the previous decision outcomes People are generally unaware of these affects and the bias that they introduce into the decision making process, so that the decision may be strongly influenced by factors of which the decision maker is unaware. Helsinki Part II 8 / 45
Decision Heuristics and Decision makers use decision rules, or heuristics to simplify difficult problems... Can be highly effective for familiar problems, but may create biases in new contexts Kahneman & Tversky experimented with assessments of probability or likelihood... But results appear equally relevant to assessing relative importance (between and within criteria) Three common heuristics per Kahneman & Tversky: Representativeness: Judging probability of an event on the basis of how typical it seems to be Availability: Judging likelihood by the ease with which instances of it happening in the past can be recalled Anchoring & Adjustment: changing contexts Inadequate adaptation to Helsinki Part II 9 / 45
Representativeness and One of my classes includes 70% engineers and 30% social scientists. One of the students, Graham, is 25 yrs old, not married but lives with Jane and their two children. He is a man of high ability, is deeply concerned with social issues, and regularly reads the Guardian. What are the chances that Graham is a social scientist? Another student, Peter, is 30 yrs old and married with no children. A man of high ability and high motivation, he promises to be quite successful in his field. He is well liked by his colleagues. What do you think are the chances that Peter is a social scientist? What if the proportions in the class were 30% engineers and 70% social scientists? Many people concentrate on whether Graham or Peter sound like social scientists or engineers, and ignore the class composition Helsinki Part II 10 / 45
Availability and Judging likelihood by the ease with which instances of it happening in the past can be recalled All other things being equal, the more times something has happened in the past, the more memories you will have This is a reasonable assumption, but factors other than frequency of occurrence can affect the ease of remembering something. For example: dramatic/novel/distinctive things are remembered better People asked to judge the probabilities of various lethal events consistently overestimate dramatic causes of death (e.g. flood, homicide), and underestimate mundane ones (e.g. diabetes) I can remember me making a significant contribution more than you, therefore I judge that I am going to make a more significant contribution in the future! Just talking about an event may make it appear more likely Helsinki Part II 11 / 45
Anchoring and Adjustment and Problem 1: Problem 2: Do you think that the percentage of small businesses which fail in the first year is greater or less than 80%? Give your estimate of the actual percentage. Do you think that the percentage of small businesses which fail in the first year is greater or less than 20%? Give your estimate of the actual percentage. Without actually completing the calculations fully estimate what the value of: 1x2x3x4x5x6x7x8 Without actually completing the calculations fully estimate what the value of: 8x7x6x5x4x3x2x1 Helsinki Part II 12 / 45
and Implications for Helsinki Part II 13 / 45
Weight Assessment and Framing issues How are criteria defined? Those expressed as loss avoidance (vs. gains) may receive greater weight Is this genuinely desired, a bias, or manipulation? Anchoring/adjustment issues Anchoring to all criteria equally important? Is this the reason for known effects of attribute splitting? How do we compensate? Helsinki Part II 14 / 45
Weight Assessment (cont.) and Availability issues How often has the criterion played a role in the past? But how relevant is this past to the current context? Has this criterion been the subject of recent discussion Practice of AHP tends to encourage context-free assessment MAVT/Swing weighting tends to focus attention on the current context Helsinki Part II 15 / 45
Partial Value Assessment and Framing/Reference Points may influence whether functions are concave/convex/s-shaped Value functions for quantitative criteria may be anchored on a linear function Simulation studies show this to be a serious source of bias AHP emphasis on pairwise ratio assessment Implies a reference point as the zero on the scale, seldom explicit Each paired comparison risks a changed framing/reference My preference for fixed and explicit reference points on a thermometer scale Helsinki Part II 16 / 45
Illustrative thermometer scale and 100 80 z i6 (best) z i5 60 40 z i4 z i3 20 z i2 0 z i1 (worst) Helsinki Part II 17 / 45
and Implications for Practice: A Simulation Study Helsinki Part II 18 / 45
and Some advantages of a goal programming approach: Preference information in terms of aspiration levels are less demanding than detailed trade-off information needed for MAVT; Used in interactive mode, goal programming is less sensitive to weight assessment that MAVT/outranking; Simulation studies suggest that goal programming is less sensitive to violations of preferential independence than MAVT. Interpretation of heuristics/cognitive biases less self-evident. Helsinki Part II 19 / 45
Interactive (Generalized) GP and The decision alternative x X implies vector of performance measures z 1 (x),...,z m (x) (to be maximized) Initial set of goals (aspiration levels) g 1,...,g m for each criterion Define deviations δ j (x) = max{0; g j z j (x)} Find solution minimizing max m j=1 w jδ j (x) + ǫ m j=1 w jδ j (x) Adjust goals and repeat until a satisficing solution is obtained Helsinki Part II 20 / 45
Potential Heuristics/Biases and Anchoring and adjustment: Initial goals anchored to prior experience Revised goals anchored to previous goals and/or last solution Availability: Initial and subsequent goals affected by easily recalled instances Reference point effect: Avoidance of sure loss could lead to under-adjustment of goals that need to be reduced. But what are the effects of these biases? Helsinki Part II 21 / 45
Simulation Study and Assumed true preferences given by an additive model with (in general) S-shaped partial values Started with ideals as initial goals g 0 = g 0 1,..., g0 m (thus only investigating goal adjustment) Suppose current GP solution is given by z = z 1,...,z m Simulate an error-corrupted but informed unbiased estimate of the best available point, say ẑ = ẑ 1,...,ẑ m ẑ should be the unbiased new goal in a direction d = ẑ z from latest solution Helsinki Part II 22 / 45
Simulation of Biases and Avoidance of sure loss: d adjusted to: { d j if d j 0 ˆd j = ρd j if d j < 0 Anchoring modelled by a new goal vector: where g = γg 0 + (1 γ)(z + φˆd). The parameter γ represents the degree of anchoring to the previous goal; The parameter φ represents anchoring to z (where φ < 1 implies anchoring, but we also allow φ > 1 in some studies) Helsinki Part II 23 / 45
Some Results and By far the largest effect was that of φ (anchoring to previous solution) Value of Mean Relative Value Mean % Rank φ Value 0.5 77.9 17.7 1.0 89.3 7.0 2.0 91.2 5.5 Anchoring to previously seen solutions has by far the largest potential to degrade performance; Anchoring to previous goals is potentially much less influential, but still significant; Resistance to sure loss has a potential influence similar in magnitude to that of anchoring to previous goals. Helsinki Part II 24 / 45
and Helsinki Part II 25 / 45
Uncertainty and Risk and All decision making involves risks and uncertainties Internal uncertainty: Structure/assumptions of the preference model Judgmental inputs from the decision maker External uncertainty: Aleatory and Epistemological Uncertainty about the environment Uncertainty about related decision areas We ll focus on the external uncertainties Helsinki Part II 26 / 45
Response to Uncertainties and Recognition of uncertainty often emerges from structuring Formally through the CAUSE analysis Informally through I m not sure responses But seldom included in formal models... although sensitivity analysis is recognized as critical Some possibilities: Probabilistic models as with expected utility Inclusion of risk measures as criteria Scenario planning Helsinki Part II 27 / 45
Multiattribute Utility Theory and Models tend to be complex... to explain and to fit Simplest non-trivial model is the multiplicative... Base selections on the expectation of U(z) defined by: 1 + ku(z) = m [1 + kk i u i (z i )] i=1 where the multivariate risk aversion k parameter satisfies: 1 + k = m [1 + kk i ] i=1 The u i (z i ) need to be von Neumann-Morgenstern utility functions Helsinki Part II 28 / 45
MAUT (cont.) and Multiplicative model still requires: Elicitation from decision makers of indifferences between difficult multivariate lotteries Specification of complete multivariate distributions Few real applications reported in the literature Tendency to evaluate the expectation of an additive value function: m w i v i (z i ) i=1 Helsinki Part II 29 / 45
Additive model and Can work well in many cases... BUT? Consider two-dimensional lotteries with two possible outcomes on each criterion, zi 0 < z1 i for i = 1, 2. Standardize the value functions such that v 1 (z 0 1 ) = v 2(z 0 2 ) = 0 and v 1 (z 1 1 ) = v 2(z 1 2 ) = 1. Consider a choice between two lotteries: 1. Equal chances on (z1 0 ; z0 2 ) and (z1 1 ; z1 2 ); and 2. Equal chances on (z 0 1 ; z1 2 ) and (z1 1 ; z0 2 ). The additive model gives an expected utility of (w 1 + w 2 )/2 for both lotteries... forcing indifference between the two lotteries irrespective of the actual weights. Helsinki Part II 30 / 45
Risk as a Criterion and Markowitz portfolio theory models a risky single-criterion objective (monetary reward) in terms of expectation and standard deviation Why not extend to any fundamental criterion? Some questions: Is standard deviation an inevitable measure of risk? Are risk measures preferentially independent of the expected performances? Helsinki Part II 31 / 45
Measures of Risk and Bell (1988) defined one switch preferences in which if a DM s preference switches from one lottery to another with increasing wealth, they will never switch back to preference for the first as wealth further increases. One-switch preferences imply a utility of the form z be cz, expectation of which combines (additively): The expected reward; and The expectation of be cz, which can be viewed as a measure of risk. Some empirical work has suggested that perceptions of risk in fishery management relates to probabilities of achieving one or more goals easily understood Helsinki Part II 32 / 45
and and Helsinki Part II 33 / 45
and Originated in Shell (van der Heijden, 1996) as a technique for identifying uncertain and uncontrollable factors which may impact on the consequences of decisions in strategic management Scenarios are constructed to describe the current and plausible, but challenging, future states of the organizational environment. The primary goal of scenario planning is to provide a structured conversation to sensitize decision makers to external and uncontrollable uncertainties Not traditionally linked to formal analysis... but can be! Helsinki Part II 34 / 45
Scenario Construction and Van der Heijden suggests five principles: At least two scenarios are required to reflect uncertainty, but more than four has proved (in his experience) to be impractical; Each scenario must be plausible, meaning that it can be seen to evolve in a logical manner from the past and present; Each scenario must be internally consistent; Scenarios must be relevant to the client s concerns and they must provide a useful, comprehensive and challenging framework against which the client can develop and test strategies and action plans; The scenarios must produce a novel perspective on the issues of concern to the client. Helsinki Part II 35 / 45
Scenario-Based and Set up s scenarios (in some way!) Evaluate alternatives in terms of each criterion, under the conditions pertaining to each scenario... i.e.: z ar = (z1 ar, zar 2,...,zar m ) for alternative a under scenario r Requires some form of preferential independence between criteria and between scenarios Aggregate preferences across criteria and across scenarios in some way... But keep it simple!! Key questions: How to choose or construct the scenarios? How to perform the aggregation? Helsinki Part II 36 / 45
MAUT and Scenarios and An obvious approach: Construct a standard utility function U(z)... Possibly using an additive approximation For each alternative calculate: s r=1 p ru(z ar ) BUT... what do the probabilities p r mean, and is expectation a valid operation? The set of scenarios is not the sample space Probability relative probability density The specific constructed scenario might be highly unlikely of itself, and yet represent extremes which have to be accounted for. Helsinki Part II 37 / 45
Alternative Models and Model A (after Goodwin & Wright): View all n s vectors z ar as possible m-dimensional outcomes (where n = number of alternatives) Use an appropriate deterministic methodology to evaluate these outcomes G & W use value functions, but outranking or reference point methods also applicable! Tabulate evaluations in an n s table, and aggregate across scenarios informally, i.e. by judgement (G & W) Or formally using an model treating scenarios as criteria Helsinki Part II 38 / 45
Alternative Models (cont.) and Model B: Treat the combination of original criterion j and scenario r as a metacriterion Apply to the (non-stochastic) problem with m s criteria... again implying a preferential independence property across all ms metacriteria Weights on the metacriteria may be assessed hierarchically, but in which order? An additive value function approach to Model B and additive value function approaches to both stages of Model A are mathematically equivalent. Helsinki Part II 39 / 45
Some Key Questions and Might preference structures change from scenario to scenario? If so, then this might better be handled by Model B Are there other theoretical, practical or behavioural reasons to prefer one model over the other? Are specific methods more appropriate to one model or the other? How many scenarios are needed? How should scenarios be constructed? Should the primary emphasis be on plausibility or on achieving representivity of ranges of variation? How should weights be assessed? Helsinki Part II 40 / 45
Simulation Studies and Still unpublished! Initially structured to investigate key questions in the context of a value function approach to both stages of Model A or B (essentially equivalent in this case) Generate random problem settings and random preference structures within the multiplicative MAUT class Discretize action and sample spaces Simulate selections using different models, different parameter settings, and judgemental errors Helsinki Part II 41 / 45
Problem Generation and For any one run, fix numbers of alternatives (n), criteria (m) and true scenarios (S) Generate effects from z ar i Central effects zi a0 surfaces = z a0 i + m ai d r i where: non-dominated on convex or concave Criterion-scenario interactions generated as d r i = m k=1 γi k Zr k, where γi k U[ α, +α] and standardized to give m k=1 (γi k )2 = 1, and Zk r N(0, 1) Alternative-criterion interactions generated from m ai U[ ρ, +ρ] Helsinki Part II 42 / 45
Problem Generation (cont.) and A selected number s of modelled scenarios are generated in one of two ways: Either constructed from combinations of extreme fractiles; Or selected from true scenarios, randomly or to maximize dispersion Different sets of weights placed on scenarios True preferences generated from additive sigmoidal utility functions Helsinki Part II 43 / 45
Preliminary Simulation Results and Based on n = 20, m = 6 and 100 repetitions per case For 3 9 selected scenarios, the overwhelmingly largest effect is that of choice of scenarios: Constructed scenarios (combinations of 70th 90th percentiles of distributions) give an average rank of selected option as 2.1 Under the same conditions, sampling of true scenarios gives an average rank of 6.1 This phenomenon conflicts with conventional scenario planning approaches which seek plausible scenarios Helsinki Part II 44 / 45
Preliminary Results (Cont.) and With constructed scenarios: No. of scenarios: 3 5 9 Mean rank: 2.5 2.0 1.9... Five scenarios may be better than three, but little need for much more Largest effect (but still moderate) is that of alternative-criterion interactions (ρ): ρ: 1 2 4 Mean rank: 1.7 2.1 2.6 Results highly robust to other parameters Clearly a substantial potential for further investigation... needs much more research Helsinki Part II 45 / 45