Att vara eller inte vara (en Bayesian)?... Sherlock-conundrum

Transcription

1 Att vara eller inte vara (en Bayesian)?... Sherlock-conundrum (Thanks/blame to Google Translate) Gianluca Baio University College London Department of Statistical Science Åkeshovsslott, Stockholm, 27 November 2018 Gianluca Baio (UCL) The Sherlock conundrum Stockholm, 27 Nov / 21

2 Disclaimer... Before I begin: Gianluca Baio (UCL) The Sherlock conundrum Stockholm, 27 Nov / 21

3 Disclaimer... Before I begin: My personal view of the world: Statisticians should be in charge of everything. Gianluca Baio (UCL) The Sherlock conundrum Stockholm, 27 Nov / 21

4 Disclaimer... Before I begin: My personal view of the world: Statisticians should be in charge of everything. And actually, come to think about it: Bayesian Statisticians should be in charge of all Statisticians. Gianluca Baio (UCL) The Sherlock conundrum Stockholm, 27 Nov / 21

5 Disclaimer... Before I begin: My personal view of the world: Statisticians should be in charge of everything. And actually, come to think about it: Bayesian Statisticians should be in charge of all Statisticians. So I probably will be very annoying in the next half hour But luckily no non-bayesian Statistician has been harmed in the making of this slides Gianluca Baio (UCL) The Sherlock conundrum Stockholm, 27 Nov / 21

6 What is statistics all about? Typically, we observe some data and we want to use them to learn about some unobservable feature of the general population in which we are interested To do this, we use statistical models to describe the probabilistic mechanism by which (we assume!) that the data have arisen Data generating process Normal distribution θ Parameters 2σ µ p(y θ) y Data NB: Roman letters (y or x) typically indicate observable data, while Greek letters (θ, µ, σ,... ) indicate population parameters Gianluca Baio (UCL) The Sherlock conundrum Stockholm, 27 Nov / 21

7 Sampling variability Probability calculus The entire population Some samples of size 5 x 5 x x 2 4 x x 6 8 x 1 x 9 x 4 x 5 x 3 x 1 x 2 x 1 x9 x 10 x 4 x 8 x 10 x 3 x 7 x 3 x 9 x 1 x 7 x 6... Population size N = 10 True population Mean µ True Standard deviation σ Sample size n = 5 Sample Mean x Sample Standard deviation s x Gianluca Baio (UCL) The Sherlock conundrum Stockholm, 27 Nov / 21

8 Sampling variability Statistics The entire population The observed sample x 5 x x 2 4 x x 6 8 x 1 x 9 x 4 x 5 x 3 x 1 x 2 x 1 x9 x 10 x 4 x 8 x 10 x 3 x 7 x 3 x 9 x 1 x 7 x 6... Population size N = 10 True population Mean µ True Standard deviation σ Sample size n = 5 Sample Mean x Sample Standard deviation s x In reality we observe only one such sample (out of the many possible in fact there are 252 different ways of picking at random 5 units out of a population of size 10!) and we want to use the information contained in that sample to infer about the population parameters (e.g. the true mean and standard deviation) Gianluca Baio (UCL) The Sherlock conundrum Stockholm, 27 Nov / 21

9 The Sherlock conundrum... Gianluca Baio (UCL) The Sherlock conundrum Stockholm, 27 Nov / 21

10 Deductive vs inductive inference Hypothesis 1 Hypothesis 2 Hypothesis 3 = 0% = 5% = 10% 5% 0% 5% 10% 15% Gianluca Baio (UCL) The Sherlock conundrum Stockholm, 27 Nov / 21

11 Deductive vs inductive inference Deduction Hypothesis 1 Hypothesis 2 Hypothesis 3 = 0% = 5% = 10% 5% 0% 5% 10% 15% Standard (frequentist) procedures fix the working hypotheses and, by deduction, make inference on the observed data: If my hypothesis is true, what is the probability of randomly selecting the data that I actually observed? If small, then deduce weak support of the evidence to the hypothesis Gianluca Baio (UCL) The Sherlock conundrum Stockholm, 27 Nov / 21

12 Deductive vs inductive inference Deduction Hypothesis 1 Hypothesis 2 Hypothesis 3 = 0% = 5% = 10% 5% 0% 5% 10% 15% Standard (frequentist) procedures fix the working hypotheses and, by deduction, make inference on the observed data: If my hypothesis is true, what is the probability of randomly selecting the data that I actually observed? If small, then deduce weak support of the evidence to the hypothesis Assess Pr(Observed data Hypothesis) Directly relevant for standard frequentist summaries, eg p-values, Confidence Intervals, etc NB: Comparison with data that could have been observed, but haven t! Gianluca Baio (UCL) The Sherlock conundrum Stockholm, 27 Nov / 21

13 Confidence intervals Drug to cure headaches true probability of success: π = 40/ Gianluca Baio (UCL) The Sherlock conundrum Stockholm, 27 Nov / 21

14 Confidence intervals Drug to cure headaches true probability of success: π = 40/ You get to see data for, say, n = 10 individuals, under the true data generating process (DGP): y = (y 1,..., y 10) = (0, 0, 1, 1, 0, 1, 0, 1, 0, 1) Can make estimates to infer from sample to population n Sample mean: ȳ = ˆπ = y i = 5 10 = 0.5 i=1 ˆπ(1 ˆπ) Standard error: se(ˆπ) = = 0.16 n Gianluca Baio (UCL) The Sherlock conundrum Stockholm, 27 Nov / 21

15 Confidence intervals Drug to cure headaches true probability of success: π = 40/ You get to see data for, say, n = 10 individuals, under the true data generating process (DGP): y = (y 1,..., y 10) = (0, 0, 1, 1, 0, 1, 0, 1, 0, 1) Can make estimates to infer from sample to population n Sample mean: ȳ = ˆπ = y i = 5 10 = 0.5 i=1 ˆπ(1 ˆπ) Standard error: se(ˆπ) = = 0.16 n Can compute the interval estimate (using some approximation/theoretical results...) 95% CI [ˆπ 2se(ˆπ); ˆπ + 2se(ˆπ)] = [ ; ] = [0.19; 0.81] Gianluca Baio (UCL) The Sherlock conundrum Stockholm, 27 Nov / 21

16 Confidence intervals Drug to cure headaches true probability of success: π = 40/ You get to see data for, say, n = 10 individuals, under the true data generating process (DGP): y = (y 1,..., y 10) = (0, 0, 1, 1, 0, 1, 0, 1, 0, 1) Can make estimates to infer from sample to population n Sample mean: ȳ = ˆπ = y i = 5 10 = 0.5 i=1 ˆπ(1 ˆπ) Standard error: se(ˆπ) = = 0.16 n Can compute the interval estimate (using some approximation/theoretical results...) 95% CI [ˆπ 2se(ˆπ); ˆπ + 2se(ˆπ)] = [ ; ] = [0.19; 0.81] Assuming the observed sample is representative of the DGP and using the sample estimates, if we were able to replicate the experiment over and over again under the same conditions, 95% of the times, the estimate for the true probability of success will be included in the interval [0.19; 0.81] That s how you interpret a 95% Confidence Interval! Gianluca Baio (UCL) The Sherlock conundrum Stockholm, 27 Nov / 21

17 Confidence intervals Simulate n sim(e.g. = 20) studies sampling data from a DGP assuming true π = 0.5 (although in fact π = 0.55!) and n = 10 For each, compute the estimated probability ˆπ Observed values Simulations π = 0.55 Successes Failures ˆπ 20 ˆπ 12 Gianluca Baio (UCL) The Sherlock conundrum Stockholm, 27 Nov / 21

18 Confidence intervals 60% of the time, it works every time! Gianluca Baio (UCL) The Sherlock conundrum Stockholm, 27 Nov / 21

19 Sample size calculations Can I use 3 patients for my study?... (Please go away!) Gianluca Baio (UCL) The Sherlock conundrum Stockholm, 27 Nov / 21

20 Sample size calculations Designing a study Designing a study is just as important as analysing it If we don t have enough information in the data, we won t be able to detect an underlying signal Related to hypothesis testing Decision on Null Reject Fail to reject Null hypothesis H 0 True False Type I error α Correct inference (False positive) (True positive) Correct inference Type II error β (True negative) (False negative) 1 Set the Type I error to some low level (typically: α = 0.05) 2 Set the Type II error to some set level (typically: β = 0.10 or β = 0.20) 3 Define the clinically relevant outcome (eg difference in treatment effects), δ 4 Set an estimate of variability in the underlying population 5 Use assumptions about sampling variability and determine minimum number of observations to be able to detect δ Originally devised to guide quality control of processes Gianluca Baio (UCL) The Sherlock conundrum Stockholm, 27 Nov / 21

21 I need a pee(-value)... Analysing the study Interpretation: Under the null hypothesis (ie IF it is true), what is the probability of observing something as extreme or even more extreme as the observed data? p(y H 0) (Model under null hypothesis) If p < 0.01 strong evidence against H 0 If 0.01 < p < 0.05 fairly strong evidence against H 0 If p > 0.05 little or no evidence against H 0 y obs (Observed data) Gianluca Baio (UCL) The Sherlock conundrum Stockholm, 27 Nov / 21

22 Two sides of the same coin? Often, hypothesis testing and p-values are seen as the same thing. They are not! Gianluca Baio (UCL) The Sherlock conundrum Stockholm, 27 Nov / 21

23 Two sides of the same coin? Often, hypothesis testing and p-values are seen as the same thing. They are not! Hypothesis testing (HT) Considers formally two competing hypotheses a null and an alternative (NB: that determines the treatment effect) Sets the probabilities of error α and β Aims at rejecting the null so it has a binary outcome (yes/no) Gianluca Baio (UCL) The Sherlock conundrum Stockholm, 27 Nov / 21

24 Two sides of the same coin? Often, hypothesis testing and p-values are seen as the same thing. They are not! Hypothesis testing (HT) Considers formally two competing hypotheses a null and an alternative (NB: that determines the treatment effect) Sets the probabilities of error α and β Aims at rejecting the null so it has a binary outcome (yes/no) Significance testing (ST, p-values) Concerned with the sampling distribution of the data under the null hypothesis Measures the strength of the evidence for/against the null, but has no formal involvement of alternative explanations for the observed data Gianluca Baio (UCL) The Sherlock conundrum Stockholm, 27 Nov / 21

25 Two sides of the same coin? Often, hypothesis testing and p-values are seen as the same thing. They are not! Hypothesis testing (HT) Considers formally two competing hypotheses a null and an alternative (NB: that determines the treatment effect) Sets the probabilities of error α and β Aims at rejecting the null so it has a binary outcome (yes/no) Significance testing (ST, p-values) Concerned with the sampling distribution of the data under the null hypothesis Measures the strength of the evidence for/against the null, but has no formal involvement of alternative explanations for the observed data p α even if often the threshold is set at 0.05 for both! α is set by the researcher p is computed from the data (as extreme or more extreme than those observed) Gianluca Baio (UCL) The Sherlock conundrum Stockholm, 27 Nov / 21

26 Two sides of the same coin? Often, hypothesis testing and p-values are seen as the same thing. They are not! Hypothesis testing (HT) Considers formally two competing hypotheses a null and an alternative (NB: that determines the treatment effect) Sets the probabilities of error α and β Aims at rejecting the null so it has a binary outcome (yes/no) Significance testing (ST, p-values) Concerned with the sampling distribution of the data under the null hypothesis Measures the strength of the evidence for/against the null, but has no formal involvement of alternative explanations for the observed data p α even if often the threshold is set at 0.05 for both! α is set by the researcher p is computed from the data (as extreme or more extreme than those observed) NB: Confusingly, experimental studies are designed under a HT setting, but analysed under a ST setting! Increasing recognition of pitfalls in science toward-evidence-based-medical-statistics-1-p-value-fallacy Gianluca Baio (UCL) The Sherlock conundrum Stockholm, 27 Nov / 21

27 Is there another way?... Gianluca Baio (UCL) The Sherlock conundrum Stockholm, 27 Nov / 21

28 Deductive vs inductive inference Hypothesis 1 Hypothesis 2 Hypothesis 3 = 0% = 5% = 10% 5% 0% 5% 10% 15% Induction The Bayesian philosophy proceeds fixing the value of the observed data and, by induction, makes inference on unobservable hypotheses What is the probability of my hypothesis, given the data I observed? If less than the probability of other competing hypotheses, then weak support of the evidence to the hypothesis Gianluca Baio (UCL) The Sherlock conundrum Stockholm, 27 Nov / 21

29 Deductive vs inductive inference Hypothesis 1 Hypothesis 2 Hypothesis 3 = 0% = 5% = 10% 5% 0% 5% 10% 15% Induction The Bayesian philosophy proceeds fixing the value of the observed data and, by induction, makes inference on unobservable hypotheses What is the probability of my hypothesis, given the data I observed? If less than the probability of other competing hypotheses, then weak support of the evidence to the hypothesis Assess Pr(Hypothesis Observed data) Can express in terms of an interval estimate: Pr(a parameter b Data) NB: Unobserved data have no role in the inference! Gianluca Baio (UCL) The Sherlock conundrum Stockholm, 27 Nov / 21

30 Bayesian inference Basic ideas Disease No disease π = π = ve ve Suppose a patient is tested for HIV. The test comes up negative ( ve) Given the assumptions/model, this indicates fairly strong evidence against the hypothesis that the true status is Disease basically, p = 0.04 Gianluca Baio (UCL) The Sherlock conundrum Stockholm, 27 Nov / 21

31 Bayesian inference Basic ideas Disease No disease π = π = ve ve Suppose a patient is tested for HIV. The test comes up negative ( ve) Given the assumptions/model, this indicates fairly strong evidence against the hypothesis that the true status is Disease basically, p = 0.04 But: how prevalent is the disease in the population? We can model our prior knowledge about this and combine this information with the evidence from the data (using Bayes theorem) Pr(Disease ve) = Pr(Disease) Pr( ve Disease) Pr( ve) Update uncertainty given the evidence provided by the data Gianluca Baio (UCL) The Sherlock conundrum Stockholm, 27 Nov / 21

32 Bayesian inference prior vs posterior The evidence from the data alone, tells us that the observed result is extremely unlikely under the hypothesis of Disease This is strongly dependent on the context, as provided by the prior knowledge/epistemic uncertainty, though! Posterior given ve Prior probability of disease Gianluca Baio (UCL) The Sherlock conundrum Stockholm, 27 Nov / 21

33 Bayesian modelling (Super) silly example: drug Existing knowledge Population registries Observational studies Small/pilot RCTs Expert options p(θ) p(y θ) θ Encode the assumption that a drug has a response rate between 20 and 60% Gianluca Baio (UCL) The Sherlock conundrum Stockholm, 27 Nov / 21

34 Bayesian modelling (Super) silly example: drug Existing knowledge Population registries Observational studies Small/pilot RCTs Expert options p(θ) p(y θ) Current data Large(r) scale RCT Observational study Relevant summaries θ Observe a study with 150 responders out of 200 patients given the drug Gianluca Baio (UCL) The Sherlock conundrum Stockholm, 27 Nov / 21

35 Bayesian modelling (Super) silly example: drug Existing knowledge Population registries Observational studies Small/pilot RCTs Expert options p(θ) Updated knowledge p(y θ) Current data Large(r) scale RCT Observational study Relevant summaries p(θ y) θ Update knowledge to describe revised state of science Gianluca Baio (UCL) The Sherlock conundrum Stockholm, 27 Nov / 21

36 Prior information vs prior distribution The main objective of Bayesian inference is to quantify the level of uncertainty For unobservable quantities (eg parameters) For unobserved quantities (eg future data) Only one language probability distribution to describe any type of uncertainty Gianluca Baio (UCL) The Sherlock conundrum Stockholm, 27 Nov / 21

37 Prior information vs prior distribution The main objective of Bayesian inference is to quantify the level of uncertainty For unobservable quantities (eg parameters) For unobserved quantities (eg future data) Only one language probability distribution to describe any type of uncertainty There are very many ways of encoding the qualitative knowledge into a probability distribution Normal (on logit scale) Beta(9.2,13.8) Probability of success Gianluca Baio (UCL) The Sherlock conundrum Stockholm, 27 Nov / 21

38 But how can I form a prior? I don t know anything about this parameter!... Gianluca Baio (UCL) The Sherlock conundrum Stockholm, 27 Nov / 21

39 But how can I form a prior? I don t know anything about this parameter!... Gianluca Baio (UCL) The Sherlock conundrum Stockholm, 27 Nov / 21

40 But how can I form a prior? I don t know anything about this parameter!... Gianluca Baio (UCL) The Sherlock conundrum Stockholm, 27 Nov / 21

41 But how can I form a prior? I don t know anything about this parameter!... Predicting the output of the 2017 UK General Election using poll data Data: number of people out of the N i respondents in poll i intending to vote for party p (multinomial counts) Objective of estimation: (π 1,..., π P ) = population vote share for each party Can model π p = (φ p/ φ p) and log(φ p) = α p + β px p Gianluca Baio (UCL) The Sherlock conundrum Stockholm, 27 Nov / 21

42 But how can I form a prior? I don t know anything about this parameter!... Predicting the output of the 2017 UK General Election using poll data Data: number of people out of the N i respondents in poll i intending to vote for party p (multinomial counts) Objective of estimation: (π 1,..., π P ) = population vote share for each party Can model π p = (φ p/ φ p) and log(φ p) = α p + β px p Prior vote share in Eng/Sco/Wal Other 'Uninformative' Non informative informative prior prior Informative prior prior Plaid Cymru Green SNP Liberal Democrat UKIP Labour Conservative Gianluca Baio (UCL) The Sherlock conundrum Stockholm, 27 Nov / 21

43 Bayesian computation In artificially simplified modelling structures, Bayesian computations are just as easy as standard statistical models Thomas Bayes (1763) Set up (what we now call) a Binomial model for number of successes out of a set number of trials Applied to billiard balls Pierre-Simon Laplace (1786) Analysed data on christening in Paris from 1745 to 1770 using (what we now consider) a Bayesian model Concludes that he was morally certain that Pr(new born is boy data) 0.5 divine providence to account for the fact that males died at higher rates... Gianluca Baio (UCL) The Sherlock conundrum Stockholm, 27 Nov / 21

44 Bayesian computation In artificially simplified modelling structures, Bayesian computations are just as easy as standard statistical models Thomas Bayes (1763) Set up (what we now call) a Binomial model for number of successes out of a set number of trials Applied to billiard balls Pierre-Simon Laplace (1786) Analysed data on christening in Paris from 1745 to 1770 using (what we now consider) a Bayesian model Concludes that he was morally certain that Pr(new born is boy data) 0.5 divine providence to account for the fact that males died at higher rates... But they can become very complicated in realistic models Alan Turing (1940s) breaking the Enigma code Using prior information and guess a stretch of letters in an Enigma message, measure their belief in the validity of these guesses and add more clues as they arrived Gianluca Baio (UCL) The Sherlock conundrum Stockholm, 27 Nov / 21

45 Bayesian computation In artificially simplified modelling structures, Bayesian computations are just as easy as standard statistical models Thomas Bayes (1763) Set up (what we now call) a Binomial model for number of successes out of a set number of trials Applied to billiard balls Pierre-Simon Laplace (1786) Analysed data on christening in Paris from 1745 to 1770 using (what we now consider) a Bayesian model Concludes that he was morally certain that Pr(new born is boy data) 0.5 divine providence to account for the fact that males died at higher rates... But they can become very complicated in realistic models Alan Turing (1940s) breaking the Enigma code Using prior information and guess a stretch of letters in an Enigma message, measure their belief in the validity of these guesses and add more clues as they arrived Since the 1990s, rely on computer simulations and a suite of algorithms called Markov Chain Monte Carlo (MCMC) Highly generalisable can throw at it virtually any complexity Can still be computationally intensive but variants of vanilla implementations can be made very efficient Gianluca Baio (UCL) The Sherlock conundrum Stockholm, 27 Nov / 21

46 What a Bayesian can do... Posterior number of seats: polls from 01 May to 22 May 400 Con Lab UKIP Lib Dem SNP Green PCY Other Gianluca Baio (UCL) The Sherlock conundrum Stockholm, 27 Nov / 21

47 What I do (most of the times)... Health technology assessment (HTA) Objective: Combine costs & benefits of a given intervention into a rational scheme for allocating resources Gianluca Baio (UCL) The Sherlock conundrum Stockholm, 27 Nov / 21

48 What I do (most of the times)... Health technology assessment (HTA) Objective: Combine costs & benefits of a given intervention into a rational scheme for allocating resources Statistical model Estimates relevant population parameters θ Varies with the type of available data (& statistical approach!) Gianluca Baio (UCL) The Sherlock conundrum Stockholm, 27 Nov / 21

49 What I do (most of the times)... Health technology assessment (HTA) Objective: Combine costs & benefits of a given intervention into a rational scheme for allocating resources e = fe(θ) c = fc(θ)... Statistical model Economic model Estimates relevant population parameters θ Varies with the type of available data (& statistical approach!) Combines the parameters to obtain a population average measure for costs and clinical benefits Varies with the type of available data & statistical model used Gianluca Baio (UCL) The Sherlock conundrum Stockholm, 27 Nov / 21

50 What I do (most of the times)... Health technology assessment (HTA) Objective: Combine costs & benefits of a given intervention into a rational scheme for allocating resources Statistical model e = fe(θ) c = fc(θ)... Economic model ICER = g( e, c) EIB = h( e, c; k)... Decision analysis Estimates relevant population parameters θ Varies with the type of available data (& statistical approach!) Combines the parameters to obtain a population average measure for costs and clinical benefits Varies with the type of available data & statistical model used Summarises the economic model by computing suitable measures of cost-effectiveness Dictates the best course of actions, given current evidence Standardised process Gianluca Baio (UCL) The Sherlock conundrum Stockholm, 27 Nov / 21

51 What I do (most of the times)... Health technology assessment (HTA) Objective: Combine costs & benefits of a given intervention into a rational scheme for allocating resources Uncertainty analysis Assesses the impact of uncertainty (eg in parameters or model structure) on the economic results Mandatory in many jurisdictions (including NICE, in the UK) Fundamentally Bayesian! Statistical model e = fe(θ) c = fc(θ)... Economic model ICER = g( e, c) EIB = h( e, c; k)... Decision analysis Estimates relevant population parameters θ Varies with the type of available data (& statistical approach!) Combines the parameters to obtain a population average measure for costs and clinical benefits Varies with the type of available data & statistical model used Summarises the economic model by computing suitable measures of cost-effectiveness Dictates the best course of actions, given current evidence Standardised process Gianluca Baio (UCL) The Sherlock conundrum Stockholm, 27 Nov / 21

52 Bayesian approach to HTA Uncertainty analysis Assesses the impact of uncertainty (eg in parameters or model structure) on the economic results Mandatory in many jurisdictions (including NICE, in the UK) Fundamentally Bayesian! Estimation & PSA (one stage) p(θ) θ p(θ y) p(y θ) y Statistical model Economic model Decision analysis Estimates relevant population parameters θ Varies with the type of available data (& statistical approach!) Integrated approach Spiegelhalter, Abrams & Myles (2004) Baio, Berardi & Heath (2017) Combines the parameters to obtain a population average measure for costs and clinical benefits Varies with the type of available data & statistical model used Summarises the economic model by computing suitable measures of cost-effectiveness Dictates the best course of actions, given current evidence Standardised process Gianluca Baio (UCL) The Sherlock conundrum Stockholm, 27 Nov / 21

53 Bayesians do it better... Potential correlation between costs & clinical benefits Strong positive correlation effective treatments are innovative and result from intensive and lengthy research are associated with higher unit costs Negative correlation more effective treatments may reduce total care pathway costs e.g. by reducing hospitalisations, side effects, etc. Because of the way in which standard models are set up, bootstrapping generally only approximates the underlying level of correlation MCMC does a better job! Gianluca Baio (UCL) The Sherlock conundrum Stockholm, 27 Nov / 21

54 Bayesians do it better... Potential correlation between costs & clinical benefits Strong positive correlation effective treatments are innovative and result from intensive and lengthy research are associated with higher unit costs Negative correlation more effective treatments may reduce total care pathway costs e.g. by reducing hospitalisations, side effects, etc. Because of the way in which standard models are set up, bootstrapping generally only approximates the underlying level of correlation MCMC does a better job! Joint/marginal normality not realistic Costs usually skewed and benefits may be bounded in [0; 1] Can use transformation (e.g. logs) but care is needed when back transforming to the natural scale Should use more suitable models (e.g. Beta, Gamma or log-normal) generally easier under a Bayesian framework Gianluca Baio (UCL) The Sherlock conundrum Stockholm, 27 Nov / 21

55 Bayesians do it better... Potential correlation between costs & clinical benefits Strong positive correlation effective treatments are innovative and result from intensive and lengthy research are associated with higher unit costs Negative correlation more effective treatments may reduce total care pathway costs e.g. by reducing hospitalisations, side effects, etc. Because of the way in which standard models are set up, bootstrapping generally only approximates the underlying level of correlation MCMC does a better job! Joint/marginal normality not realistic Costs usually skewed and benefits may be bounded in [0; 1] Can use transformation (e.g. logs) but care is needed when back transforming to the natural scale Should use more suitable models (e.g. Beta, Gamma or log-normal) generally easier under a Bayesian framework Particularly as the focus is on decision-making (rather than just inference), we need to use all available evidence to fully characterise current uncertainty on the model parameters and outcomes A Bayesian approach is helpful in combining different sources of information Propagating uncertainty is a fundamentally Bayesian operation! Gianluca Baio (UCL) The Sherlock conundrum Stockholm, 27 Nov / 21

56 Bayesians do it better... Just imagine what an Italian Bayesian Statistican can do... Gianluca Baio (UCL) The Sherlock conundrum Stockholm, 27 Nov / 21

57 Tack! Gianluca Baio (UCL) The Sherlock conundrum Stockholm, 27 Nov / 21