Stated Preference Methods Research in Health Care Decision Making A Critical Review of Its Use in the European Regulatory Environment.

Stated Preference Methods Research in Health Care Decision Making A Critical Review of Its Use in the European Regulatory Environment. Kevin Marsh, Evidera Axel Mühlbacher, Hochschule Neubrandenburg Janine van Til, University of Twente 7 November 2017

Objectives To map which stated preference methods are being used in the European regulatory environment A more systematic consideration of preferences as part of regulatory decisions Identification, weighting and aggregation of decision criteria Outline, conclusion: Identification of gaps in the use of preference methods Implications of existing experience for the use of stated preference methods A research agenda for the development of stated preference methods Limitations with the approach adopted in this research

Current task Done already Overview of the Review Problem: Literature and website reviews revealed little information on stated preference methods used Purpose of survey: Identification of stated preference methods used by the institutions involved in HTA

INTRODUCTION

TBD SIG WG objectives Definition of preference methods Process / findings to date Next steps Introduce the workshop

CASE STUDY 1 CEA VS MCDA

Case study 1: Comparison NICE GYMESZI Location England and Wales Hungary Decision type Reimbursement Reimbursement Technology Medicines New hospital medical technologies (devices) Stakeholder preference General population Decision makers Method TTO Point allocation Status In use In use

Case study 1: NICE ICER = C 1 C 0 E 1 E 0 QALY Years of life Utility Patient experience General population preference Abbreviations: ICER = incremental cost-effectiveness ratio; QALY = quality-adjusted life year 8

Case study 1: NICE ICER = C 1 C 0 E 1 E 0 QALY Years of life Utility Patient experience General population preference Abbreviations: ICER = incremental cost-effectiveness ratio; QALY = quality-adjusted life year NICE (2014) Consultation Paper. Value Based Assessment of Health Technologies Abbreviations: HRQoL = health-related quality of life; ICER = incremental cost-effectiveness ratio 9

Case study 1: GYMESZI Implemented in 2010 Evaluation of 14 applications for medical technology Criteria and weights established by a committee comprising of: The Health Financing Agency The Ministry of Health Clinical experts Health economists 10

Case study 1: Assessment Criteria Description NICE GYMESZI Completeness Are all criteria considered Consistency/ transparency Are criteria treated consistently across evaluations? Opportunity cost Are costs appropriated considered Trade-offs Are criteria weights valid Criteria properties Does the analytical framework appropriately reflect the nature of the criteria /? 11

Case study 1: Poll Which of the two examples do you think is the appropriate way to incorporate preferences into reimbursement decisions NICE GYMESZI NEITHER 12

CASE STUDY 2 IQWIG S PILOTS: DCE VS AHP

German pilot DCE Hepatitis C

Discrete Choice Experiment Mühlbacher, A. & Johnson, F.R. Appl Health Econ Health Policy (2016) 14: 253. doi:10.1007/s40258-016-0232-7

German pilot DCE Hepatitis C Objectives: The pilot s aim was to investigate to what extent a DCE can be used as a method for the identification, weighting and prioritization in the case of multiple endpoints Method: DCE including 7 attributes of antiviral treatment of hepatitis C: Treatment efficacy (Probability of sustained viral response 6 months after end of therapy) Adverse events (Duration of flu-like symptoms after injection, Probability of gastro-intestinal symptoms, Probability of psychiatric symptoms, Probability of skin symptoms and/or alopecia) Treatment burden (Duration of antiviral therapy, Frequency of interferon injections) 2 Subsamples: Patient and expert respondents

German pilot DCE Hepatitis C Conclusion: For patients it was shown that it was feasible to weight patientrelevant outcomes via a DCE In the comparison of patient preferences and opinions of healthcare professionals, the sequence was the same for 4 of the 7 attributes; however, the magnitude of the weighting deviated for further attributes. DCE method can be applied to other questions

German pilot AHP Depression

German pilot AHP Depression Objectives: This pilot project s goal was to examine to what extent the AHP method can be applied in health economic evaluations in Germany in the identification, weighting, and prioritization of multiple patientrelevant outcomes. Method: AHP evaluating 11 endpoints of antidepressant treatment Response Remission Cognitive function Reduction of anxiety Social function Avoidance of relapse Reduction of pain Other serious adverse events (Attempted) Suicide Other adverse events Sexual dysfunction 2 Subsamples: Patient and expert respondents

German pilot AHP Depression Results: Conclusion: AHP method can be applied both in patients and healthcare professionals AHP enables elicitation of preferences of individuals for certain treatment goals and outcomes in a step-bystep approach and calculation of the weights for each of these outcomes by means of a matrix algebra Danner, Marion, et al. "Integrating patients' views into health technology assessment: Analytic hierarchy process (AHP) as a method to elicit patient preferences." International journal of technology assessment in health care 27.4 (2011): 369-375.

German pilot DCE PERIODONTAL DISEASE [Available in German only] VALUE IN HEALTH 19 ( 2 0 1 6 ) A347 A766

German pilot DCE PERIODONTAL DISEASE Objective: The pilot s aim was to explore whether a preference elicitation using a DCE can be conducted within 3 months Method: DCE including 4 attributes of treatment and disease characteristics of periodontal treatment alternative Tooth loss Symptoms & complaints Frequency of periodontist visits Cost Patient respondents

German pilot DCE PERIODONTAL DISEASE Results: Relative importance of attributes Tooth loss (0.73 relative weight) Symptoms & complaints (0.22) Frequency of periodontist visits (0.03) Costs (0.02) Conclusion: DCE is feasible within 3 months

Mühlbacher/Kaczynski (2015), Mühlbacher/Johnson (2016), Neidhardt et al. (2012), Mulye (1998), Helm et al. (2003) DCE versus AHP Criterion DCE AHP Methodological approach Decompositional Compositional Basis assumption Evaluation process Reality level Characteristics independently of one another, but interactions can be tested; any combinations of attributes are possible Holistic evaluation of stimuli High, but sometimes assessment task is complex Characteristics independently of one another Pair-wise comparisons of alternatives and decision criteria Less realistic, but easy assessment task Benefit/ value model Additive part worth model Weighted additive model Flexibility according benefit/ value function Target object/ respondents High. Different utility functions possible Market segment on the basis of individual customer Scale level of the input Ordinal or interval scaled Interval scaled Scale level of the output Interval scaled Ratio scaled Estimation technique e.g. OLS, RPL, MXL, LC ( ) e.g. Eigenvalue Interpretation of importance weights Feedback during evaluation process Part worth value of attribute (utility scale) Validity testing (live feedback not easy) Cognitive stress for respondents High. Grows with the increasing of attributes Minor Survey range Less, but complex assessment of complete stimuli Minor. Only additive value function possible Single decision-maker or group Relative importance of one criteria for target achievement (no utility scale) Consistency test and sensitivity analysis possible Various, but easy pair-wise comparisons Restrictions on use Up to six attributes with 2-4 levels Various attributes possible

Backup DCE PERIODONTAL DISEASE VALUE IN HEALTH 20 ( 2 0 1 7 ) A 3 9 9 A 8 1 1

Backup DCE PERIODONTAL DISEASE Objective: The study s aim is to test validity with changing attribute and multidimensionality Method: Two DCE decision-making models (model 1 and model 2) Model 1 Model 2 Tooth loosening/tooth loss Tooth loosening/tooth loss (compound attribute with varying (compound attribute with varying severity and the number of teeth severity and the number of teeth concerned) concerned) Gum bleeding, Gum bleeding, Pain in everyday life (considered both Pain in everyday life (considered the severity and the duration of pain) both the severity and the duration of Pain during therapy pain) Therapy administration Pain during therapy Application of antibiotics Side effect: infection Side effect: antibiotic resistance

Backup DCE PERIODONTAL DISEASE Results Model 1 Random parameter logit model (95% confidence interval) (model 1, N=300)

Backup DCE PERIODONTAL DISEASE Results Model 2 Random parameter logit model (95% confidence interval) (model 2, N=310)

CASE STUDY 3 EMA S PILOTS: SWING WEIGHTING WORKSHOPS VS PATIENT SURVEYS

Articles: Incorporating Patient Preferences Into Drug Development and Regulatory Decision Making: Results From a Quantitative Pilot Study With Cancer Patients, Carers, and Regulators Postmus et al., 2017 Is quantitative benefit risk modelling of drugs desirable or possible? Phillips et al., 2011

Is quantitative benefit risk modelling of drugs desirable or possible? - Phillips et al., 2011 Objective: To determine whether quantitative benefit-risk modelling is possible Method: MCDA with the expected utility rule Case study: Weight Loss Drug Value Tree: one favourable effect and five unfavourable effects Value Functions: concave value function, provided by an expert roleplaying an assessor Weight Elicitation: Swing Weighting

Quantitative Pilot Study With Cancer Patients, Carers, and Regulators - Postmus et al., 2017 Objectives: To explore the feasibility of a lean method for eliciting individual patient preferences Method: MCDA Case study: Treatment of Melanoma Value Tree: one favourable effect and two unfavourable effects Value Functions: linear value function, assumption by team Weight Elicitation: Swing Weighting Ordinal Judgements

MCDA - Swing Weighting Weighting equates the units of preference value across all scales. Philips et al., 2011 Best Worst Weight Loss 15% 0% Postmus et al., 2016 Best Worst Overall Survival 65% 45% Anxiety No Yes Sleep Disorders No Yes Mood Alterations No Yes Depressive Disorders No Yes Irritability No Yes Moderate Toxicity 5% 20% Severe Toxicity 15% 35%

MCDA - Swing Weighting Example Postmus Full Ranking

MCDA - Swing Weighting Bisection Method Worst Survival (1st) X < or > 55%? 45% Severe Toxicity (2nd) 15% 35%? 45% ----------------------55%---------------------> 65% 15% ----------------> 35%

MCDA - Swing Weighting Bisection Method Worst Survival (1st) X < or > 50%? 45% Severe Toxicity (2nd) 15% 35% 50%? 45% ----------------------55%---------------------> 65% 15% -> 35%

MCDA - Swing Weighting Bisection Method Worst Survival (1st) X < or > 60%? 45% Severe Toxicity (2nd) 15% 35% 60%? 45% ----------------------55%---------------------> 65% 15% -------------------------------> 35% Weight of least importnat criterion is determined relative to 2nd most important criterion

Postmus et al., 2016

Is quantitative benefit risk modelling of drugs desirable or possible? - Phillips et al., 2011 Method: MCDA with the expected utility rule Case study: Weight Loss Drug Value Tree: one favourable effect and five unfavourable effects Value Functions: concave value function, provided by an expert roleplaying an assessor Weight Elicitation: Swing Weighting Relative Judgements

SWING Weighting Philips et al., 2011 Data collection: Role-playing by experts -> Verbal elicitation process Unfavourable Effects: Compare all unfavourable effects. Determine most important effect (W = 100) Determine relative weight all other effects compared to most important (½; ¼). Favourable vs. Unfavourable Effects: 0 -> 15% Weightloss OR All Unfavourable Effects -> No Unfavourable Effects

Benefits of SWING weighting - Philips Swing weighting results in explicit assessment of assessors judgements, which could increase transparancy and consistency Swing weighing can accommodate any kind of data Using pre-defined scales (and global estimates of weight ranges) allows weights to be assigned to all scales even before data about the options are considered

Benefits of SWING weighting - Postmus Bisection swing weighting was considered intuitive by patients Regulators felt that outcomes where useful to identify subgroups of patients Recommendation to combine with face-to-face meetings with patients to understand preference constructions and context Regulators felt that outcomes where easy to interpret, time frame of elicitation process was feasible 49

Case study 3: Assessment Philips et al., 2011 Postmus et al., 2016 Elicitation Process Ratio Statements Ordinal Statements Value tree Lower level of Aggregration High level of Aggregration Outcomes (weights) Potentially high(er) level of discrimination Lower level of discrimination Outcomes (value tree) Elicited from experts Assumed linear by decision analysts Feasibility Questions based on studies Can patients/regulators participate in verbal elicitation process? What is the feasibility with a higher number of criteria?

CONCLUSION

TBD Pull together the themes emerging from the case studies