Contact. Download handouts here: Hand outs. Fundamentals of evaluaeon. Benefit- risk analysis 23/05/2011

Transcription

1 Contact PATIENT- CENTERED BENEFIT- RISK ANALYSIS: the case for AnalyEc Hierarchy Process ISPOR Workshop W3 Monday May, Maarten J. IJzerman, PhD John F.P. Bridges, PhD Maarten J. IJzerman, PhD University of Twente, the Netherlands m.j.ijzerman@utwente.nl John F.P. Bridges, PhD Johns Hopkins Bloomberg School of Public Health jbridges@jhsph.edu Hand outs Slide deck Journal and conference papers: Dolan JG: MulE- criteria clinical decision support. A primer on the use of muleple- criteria decision making methods to promote evidence- based paeent- centered healthcare. The PaEent 010 3(4): 9-48 (with permission) IJzerman MJ, Bridges JFP and Hummel JM: Benefit- risk analysis in healthcare: the case for analyec hierarchy process. ISPOR BalEmore, May For addieonal copies: m.j.ijzerman@utwente.nl Download handouts here: hbp://dl.dropbox.com/u/114039/dolan.pdf hbp://dl.dropbox.com/u/114039/slides.pdf hbp://dl.dropbox.com/u/114039/handout.pdf Benefit- risk analysis Benefit- risk analysis is a process by which individual or compeeng interveneons are assessed Benefit- risk important in regulatory decisions (e.g. FDA) and in public health guidelines (e.g. USPSTF). Benefit- risk analysis has an implicit value systems Benefits, efficacy, treatment effects are good Risks, adverse events, side effects are bad An order may be applied to these (e.g. primary endpoint) Such value systems are not universally beneficial as there may be a need to make tradeoffs. Fundamentals of evaluaeon IdenEficaEon: What outcomes do we care about? e.g. Benefits, Risks, Costs?? Measurement How do these outcomes vary? e.g. RCT, systemaec review, CER ValuaEon Are all outcomes valued equally? DeliberaEon How do decision makers interpret this data? How is a decision made? 1

2 Thinking about benefits and risks More risks MulEple benefits and risks Less benefits Unambiguously bad Safer Efficacious Unambiguously good More benefits Benefits Harms Less risks Favors treatment Favors control Levitan B (011) A Concise Display of MulEple End Points for Benefit Risk Assessment, Clinical Pharmacology & TherapeuEcs (011) 89 1, doi: /clpt MulEple criteria Such approaches are beneficial for the comparison of evidence on muleple criteria: NICE CER, end- of- life, rare diseases, innovaeon etc IQWiG MulEple paeent relevant endpoints FDA/CER/USPSTF benefits and risks Several key queseons remain: How does one priorieze the criteria? How does one weight the criteria? Can a cardinal index be formed as a single tool to assess net- benefits? This workshop Introduce multi-criteria decision making methods (MCDA) and AHP in particular Basics of MCDA and AHP Demonstration An example: Introduce the use of AHP to weigh patient-relevant endpoints and to measure patient preferences in HTA Discussion: Comparative effectiveness research and benefit-risk assessment: is there a role for multi-criteria decision analysis using AHP? Join the discussion #ISPOR #MCDA #ahpinhealth 1

3 This workshop Introduce multi-criteria decision making methods (MCDA) and AHP in particular Basics of MCDA and AHP Demonstration An example: Introduce the use of AHP to weigh patient-relevant endpoints and to measure patient preferences in HTA Discussion: Comparative effectiveness research and benefit-risk assessment: is there a role for multi-criteria decision analysis using AHP? An overview of MCDA Compensatory vs. non- compensatory Composed vs. decomposed methods Direct vs. indirect preference esemaeon MCDA: They all employ a decision tree or value structure 14 An example using rank- weights An example using rank- weights Clinical Outcome Adverse Events Out- of- Pocket Costs 1 3 Clinical Outcome Adverse Events Out- of- Pocket Costs Drug A symptom relief high blood pressure 5 US$ Drug A 1 symptom relief high blood pressure 5 US$ Drug B pareal relief high blood pressure 3 US$ Drug B pareal relief high blood pressure 1 3 US$ Drug C pareal relief No adverse events 5 US$ Drug C pareal relief 1 No adverse events 5 US$ An example using rank- weights An example using rank- weights 0,50 0,33 0,17 Clinical Outcome Adverse Events Out- of- Pocket Costs 0,50 0,33 0,17 Clinical Outcome Adverse Events Out- of- Pocket Costs Drug A 0,50 symptom relief 0,5 high blood pressure 0,55 US$ Drug A 0,50 symptom relief 0,5 high blood pressure 0,55 US$ Drug B 0,5 pareal relief 0,5 high blood pressure 0,503 US$ Drug B 0,5 pareal relief 0,5 high blood pressure 0,503 US$ Drug C 0,5 pareal relief 0,50 No adverse events 0,55 US$ Drug C 0,5 pareal relief 0,50 No adverse events 0,55 US$ drug A: 0,375 drug B: 0,93 and drug C: 0,33 3

4 MCDA and AHP A decision problem MCDA methods utilize a decision matrix to provide a systematic analytical approach for integrating risk levels, uncertainty, and valuation, which enables evaluation and ranking of many alternatives. (Belton & Steward, 00) MAUT, MAVT and AHP are the more complex MCDA techniques available The AHP structures a decision into a hierarchy of criteria, sub criteria and alternatives. By means of pairwise comparisons of two (sub) criteria or alternatives, it generates inconsistency ratios and weighting factors to prioritise the criteria and alternatives. Sensitivity analysis can be applied to test the robustness of the priorities. (Saaty, 1989) MCDA decision structure AHP space: Matrix with pairs of criteria or alternatives Choose your favorite restaurant 0,5 0,3 0, Style Price Travel 0,5 0,3 0, 0,7 0, 0,1 0,3 0,1 0,6 A criteria criterion 1 criterion criterion n criterion 1 a 11 a 1 a 1n criterion a 1 a a n Thai Italian Greek 15$ 0$ 5$ Car Bicycle Foot Restaurant 1 Restaurant Restaurant 3 criterion n a n1 a n a nn Matrix Eigenvalue: prioriees A criteria criterion 1 criterion criterion n criterion criterion 1/ criterion n 1/ 1/3 1 4

5 Data colleceon opeons In AHP, data is collected using pair- wise comparisons Pairs are compared on an 18- point scale Data can be collected using Computer supported interview Group decision support mode Survey queseonnaire Key- pads Use bubons 1 to 9 for scoring Switch between blue and red Clinical benefit Adverse events Out- of- pocket cost 5

6 Uncertainty in AHP SensiEvity analysis Changing the importance of criteria Fuzzy AHP SimulaEng decision makers uncertainty Monte Carlo simulaeon ProbabilisEc SA using cumulaeve distribueon funceon of inputs Why AHP? and why not? AHP is developed to support decision making AHP allows group decision making (up to 0 participants) Explicit and transparent judgment, i.e. clear decision criteria AHP weights can be obtained in small samples (n=1) Immediate response to the decision maker about preference No complicated post-hoc modeling required Individual preferences can be easily obtained Useful if decision involves multiple ( sub-)criteria Decision structure is flexible and allows post-hoc adaptations AHP allows (probabilistic) sensitivity analyses 6

7 Potential pitfalls of AHP u u u Attributes and (patient-reported) outcomes should be independent due to the additive value function; u No interaction between levels and attributes (An. Network Process); Rank reversal of alternatives is possible; u E.g. if A>B and A>C, then A>C (transitivity requirement) u In particular in surveys without immediate feedback u Less of a problem in computer assisted interviewing The range of performance of the alternatives need to be clear, before weighing the main attributes Which criterion is most important in the treatment of low- back pain? Clinical Outcome Adverse Events Out- of- Pocket Costs And now Clinical Outcome Adverse Events Out- of- Pocket Costs Full pain relief risk of stroke 4000 US$ ParEal pain relief immediate death 3000 US$ This workshop AHP in healthcare decision making Introduce multi-criteria decision making methods (MCDA) and AHP in particular Basics of MCDA and AHP Demonstration An example: Introduce the use of AHP to weigh patient-relevant endpoints and to measure patient preferences in HTA Discussion: Comparative effectiveness research and benefit-risk assessment: is there a role for multi-criteria decision analysis using AHP? 1. Patient. Clinicians 3. Biomedical innovation 4. Health care management 5. Health policy To understand the preferences of patients and clinicians, and to guide shared decision- making. To support the development of clinical guidelines regarding treatment decisions and the development of practice guidelines. To support the definition of design features and to analyse trade- offs related to implementation and coverage of new technologies. To support strategic planning of e.g. health facilities and operational management decisions within hospital facilities. To support governmental policy makers in handling multiple trade- offs and stakes. Dolan&Bordley,1993 Dolan, 011 Van Til et al, 008 Hummel et al, 000 Hilgerink et al, 011 Wu, Lin et al, 011 Cho & Kim, 011 Hummel & IJzerman, ISAHP

8 AHP in healthcare decision making Use of AHP in patient-centered healthcare health policy product development health care management clinical practice guidelines shared decision making 1. Ranking (patient-relevant) outcome measures Determination of most important outcome measure (for which to construct an efficiency frontier (IQWIG)). Weighing (patient-relevant) outcome measures MCDA methods can be used to obtain relative weights for attributes and to estimate preference for treatment 3. Value based pricing, e.g. calculation of a ceiling price based on patient-weighted outcomes estimation of the marginal value of an incremental improvement Use of AHP to rank and weigh patientrelevant endpoints Response weighing patient-relevant endpoints for use of antidepressants Based on benefits assessment based on IQWIG Reports: A05-0A (SNRIs duloxetine, venlafaxine) and A05-0C (Bupropion, Mirtazapin, Reboxitin) Both commissioned by the G-BA Approach Definition of decision tree with IQWIG team Selection of representatives Panel session with experts and patients Prioritize endpoints Efficacy Adverse events Disease specific QoL Remission No relaps Social function Anxiety Pain Serious adverse events Adverse events Suicide and attempted suicide Other serious adverse events Sexual dysfunction Other adverse events Hummel, Danner, van Manen, Volz, Ijzerman, Gerber: Use of AHP in prioriezing PRO s for IQWiGs benefits assessment. 011, Submibed Cognitive function How patients and experts value patient relevant endpoints Treatment preference: integration of clinical evidence Rank 1 Outcome measure for experts Rank 1 Outcome measure for patients 0,5 0,45 0,4 0,35 0,3 0,5 0, 0,15 paeents experts Pooled effect / clinical outcome (OR) 0,1 0,05 0 effeceveness Adverse events Quality of life Response Remission Relaps Weighted clinical outcome on linear scale 8

9 Patient weighted performance of three antidepressants This workshop Introduce multi-criteria decision making methods (MCDA) and AHP in particular Basics of MCDA and AHP Demonstration An example: Introduce the use of AHP to weigh patient-relevant endpoints and to measure patient preferences in HTA Discussion: Comparative effectiveness research and benefit-risk assessment: is there a role for multi-criteria decision analysis using AHP? 6 9