Multicriteria Decision Analysis of Drug Harms in the UK and Europe

Transcription

1 Multicriteria Decision Analysis of Drug Harms in the UK and Europe Professor Lawrence Phillips London School of Economics & Political Science and Facilitations Ltd 12 th JRC Annual Training on Composite Indicators & Multicriteria Decision Analysis Ispra, September 2014

2 Nutt, D. J., King, L. A., Phillips, L. D., & on behalf of the Independent Scientific Committee on Drugs. (2010). Drug harms in the UK: a multicriteria decision analysis. The Lancet, 376( ). 2

3 3 Method Study design 16 harm criteria developed by the UK s ACMD 20 drugs ISCD members plus 2 external experts Meeting facilitated as a decision conference (facilitated workshop) Value Tree

4 4 Decision Conference A one-to-three-day workshop To resolve important issues of concern Attended by key players who represent the diversity of perspectives on the issues Facilitated by an impartial specialist in group processes & decision analysis Using a requisite (just-good-enough) model created on-the-spot to help provide structure to thinking

5 5 Decision conference + MCDA A methodology for appraising options on multiple criteria, and combining them into one overall appraisal MCDA converts all input evaluations of decision outcomes into common units of value added

6 6 A system not based on MCDA For the harm of psychoactive drugs, the team evaluated the extent of harm as defined by the criteria.

7 7 The 20 drugs Heroin Crack Cocaine Alcohol Tobacco Amphetamine Mephedrone Buprenorphine Benzodiazepines Cannabis Anabolic Steroids Ketamine LSD Mushrooms Ecstasy Khat Butane Methadone GHB Methylamphetamine

8 8 Scoring the drugs The most harmful drug on each criterion was scored at 100. All other drugs were scored relative to that drug. E.g., a drug considered half as harmful was given a score of 50. This creates a unique ratio scale for each criterion. 100 Most harm No harm

9 9

10 10 Weighting the criteria Some criteria represent more harm than others. Swing-weights equate the units of harm on all the criteria: the swing in harm from no harm on a criterion to the most harmful drug. The group considered this question to compare the levels of most harm on the criteria: How big is the difference in harm and how much do you care about that difference? 100 Most harm No harm

11 11

12 12 The resulting criteria weights These show the relative swings in harms; NOT just the importance of the criteria.

13 13 Results Ratios of the total numbers represent ratios of harms, e.g., alcohol is three times as harmful as cocaine.

14 14 Alcohol Harm to Others Harm to Users

15 15 Why is alcohol so harmful? Half the harm from these four

16 16 Why is tobacco so harmful? 78% of the harm from these four

17 17 Compare alcohol with tobacco harm Alcohol more harmful Tobacco more harmful

18 UK Drugs Act classification 18 Correlation of UK Drugs Act classification with ISCD results at November A 2B 3C 4U linear r = 0.04 without the unclassified drugs: linear r = ISCD results

19 UK Drugs Act classification 19 Correlation of UK Drugs Act classification (with khat at Class C) with ISCD results 1A 2B 3C 4U mushrooms, LSD, Ecstasy mephedrone, cannabis, amphetamine buprenorphine, khat linear r = 0.01 without the unclassified drugs: linear r = ISCD results

20 European replication May

21 ISCD results

22 22 UK 2010 vs. Europe UK r = Supports the reliability of judgements Europe

23 23 MCDA for 12 nicotine products Cigarettes Cigars Small cigars Pipes Water pipe Smokeless refined Smokeless unrefined Snus ENDS Oral products Patch Nasal sprays Manufactured and hand-rolled cigarettes in which the tobacco is wrapped in paper. Smoked cigars: rolled tobacco leaf. Used like a cigarette but wrapped in tobacco leaf (a product largely limited to the USA but widely used there). A tube with a small bowl at one end for smoking tobacco. Where tobacco smoke is bubbled through water. Non-snus (and other) smokeless refined tobacco products used orally, including leaf chewing tobacco and snuff. Non-snus (and other) smokeless unrefined tobacco products used orally, including leaf chewing tobacco and snuff. A low nitrosamine and non-fermented smokeless tobacco product. Electronic Nicotine Delivery System products, e.g. e- cigarettes. Orally-available nicotine products. Dermal nicotine delivery products. Nasal nicotine delivery products.

24 24 ISCD results July TO USERS 67 TO OTHERS Nutt, D. J., Phillips, L. D., Balfour, D., Curran, H. V., Dockrell, M., Foulds, J.,... Sweanor, D. (2014). Estimating the harms of nicotine-containing products using the MCDA approach. European Addiction Research. doi: /

25 25

26 26 Research to improve benefit-risk assessment EMA Benefit-Risk: WP1: Describes current B-R regulatory practice in EU WP2: Reviews of 18 quantitative approaches to B-R WP3: Modelling of five field tests of new drugs being considered by the CHMP WP4: Recommends tools and processes Report on Risk Perception Study Module Source: Click on Special topics, Supporting research, Benefit-risk methodology IMI Protect, WP5: Reviews 47 approaches & recommends 13 for further investigation Reviews visualisation methods; two parts, current & proposed Investigates approaches for six previously-approved drugs (in 8 reports, totalling 29MB) Publishes Recommendations on website (3.85MB) Source:

27 27

28 28

29 29 Efficacy & Safety Benefits & Risks Efficacy & Safety Data Favourable & Unfavourable Effects Clinical Relevance of the Effects Benefits & Risks Regulators & medical experts Judgement required Physicians & patients

30 30 What do we mean by importance? Imagine you suffer from frequent migraines. Which is more important to you for possible treatment with a triptan? Reduction of functional disability (mild or gone 2 hours after dosing) Myocardial infarction (within 48 hours of dosing) CIRS, June 2011: A syndicate group assigned twice as much weight to the reduction of functional disability as to MI. How can we account for this?

31 31 Aspects of importance Context Migraine sufferer or not Impact on work and family Your risk attitude??? Real-world difference Extent of migraine-associated functional disability (as compared to mild or gone within 2 hours) Current heart condition (as it might affect probability of MI within 48 hours of dosing) How much those differences matter to you It s your judgement call!

32 32 A major error in weighting Assessing value trade-offs independent of the range of comparisons. Reference: Keeney, R. (2002). Common mistakes in making value trade-offs. Operations Research, 50(6), Keeney s statement applies to every approach that combines benefits and risks, qualitative, semi-quantitative or quantitative. (And that is only one of 12 mistakes.)

33 33 How important is this effect? How big is this effect difference, and how much do you care about it?

34 34 United Kingdom s CoRWM Purpose: to recommend long-term solution for managing high- and low-level radioactive waste Review of options to be open, transparent and inclusive To inspire public confidence Largest public consultation exercise ever conducted in the UK, including stakeholders Key issues, criteria Criterion weights

35 35 Criteria 26 criteria under the objectives. Panels of experts define the criteria and develop scoring scales. The panels score the options on each of the criteria.

36 36 Weighting: define criteria Criterion 1: Radiation The extent to which an option is expected to protect individual members of the public from exposure to radiation during the first 300 years. 9 inherently resilient to possible adverse events : : 1 very poor resilience readily fails challenge associated with any one of several foreseeable events by substantial margin System continues to operate within its design envelope for all reasonably foreseeable nonmalicious hazardous events Option fails to maintain protection by a substantial margin beyond the defined reference level under any one of a range of reasonably foreseeable hazardous events, requiring significant and time-consuming corrective action in order to recover.

37 37 Weighting: define criteria Criterion 25: Flexibility Extent to which the option is expected to allow for future choice and respond to unforeseen or changed circumstances over the 300-year time span. 9 System is fully monitored and adaptable, and the waste is easily retrievable using the existing system : 5 Some system elements can be monitored, adaptability is limited, and waste retrieval is moderately difficult : Point 5 defined so 1-5 and 5-9 are equal value increments 1 Monitoring options is severely restricted, the system is not adaptable, and waste retrieval is very difficult

38 38 Weight the criteria Compare swings in added value from 1 to 9. Criterion 1: Radiation 9 inherently resilient to possible adverse events : : 1 very poor resilience readily fails challenge associated with any one of several foreseeable events by substantial margin Criterion 25: Flexibility 9 System is fully monitored and adaptable, and the waste is easily retrievable using the existing system : : 1 Monitoring options is severely restricted, the system is not adaptable, and waste retrieval is very difficult How big is the 1 to 9 difference in its impact, and how much do you care about that difference?

39 Favourable Effects Unfavourable Effects 39 Triptan: efficacy & safety data Decreased Pain Decreased Sensitivity Other Individual Risks Rapid Onset Outcome Units Scale Headache Relief Pain-free Response Sustained Response Reduced Sensitivity to Sound and Light Reduction in Functional Disability Reduction in Nausea and Vomiting Transient Triptans Sensations CNS Adverse Events Chest-related Adverse Events Myocardial Infarction % 0-70 % 0-70 % 0 70 % 0 70 % 0 70 % 0 70 % 0 70 % 0-7 % 0 7 % 0-7 No. / 1000 pat-yrs 8-16 Triptan 15 mg Rate (95% CI) 23.3 (20.7,25.9) 57.0 (53.9,60.1) 25.0 (22.3,27.7) 26.0 (23.3,28.7) 46.0 (42.9,49.1) 44.8 (41.7,47.9) 51.0 (47.9,54.1) 2.2 (1.3,3.1) 4.0 (2.8,5.2) 1.5 (0.7,2.3) 13.1 (6,20) Triptan 30 mg Rate (95% CI) 31.0 (28.1,33.9) 63.6 (60.6,66.6) 41.4 (38.3,44.5) 31.0 (28.1,33.9) 53.0 (49.9,56.1) 56.0 (52.9,59.1) 64.0 (61,67) 4.0 (2.8,5.2) 6.4 (4.9,7.9) 4.0 (2.8,5.2) 16 (8.2, 23.8) NSAID Rate (95% CI) 22.7 (20.1,25.3) 44.2 (41.1,47.3) 25.9 (23.2,28.6) 11.1 (9.2,13) 31.0 (28.1,33.9) 39.8 (36.8,42.8) 48.8 (45.7,51.9) 0.1 (0.0,0.3) 2.0 (1.1,2.9) 0.1 (0.0,0.3) 10.0 (3.8,16.2)

40 Favourable Effects Unfavourable Effects 40 Triptan: efficacy & safety data Decreased Pain Decreased Sensitivity Other Individual Risks Rapid Onset Outcome Units Scale Headache Relief Pain-free Response Sustained Response Reduced Sensitivity to Sound and Light Reduction in Functional Disability Reduction in Nausea and Vomiting Transient Triptans Sensations CNS Adverse Events Chest-related Adverse Events Myocardial Infarction % 0-70 % 0-70 % 0 70 % 0 70 % 0 70 % 0 70 % 0 70 % 0-7 % 0 7 % 0-7 No. / 1000 pat-yrs 8-16 Triptan 15 mg Rate (95% CI) 23.3 (20.7,25.9) 57.0 (53.9,60.1) 25.0 (22.3,27.7) 26.0 (23.3,28.7) 46.0 (42.9,49.1) 44.8 (41.7,47.9) 51.0 (47.9,54.1) 2.2 (1.3,3.1) 4.0 (2.8,5.2) 1.5 (0.7,2.3) 13.1 (6,20) Triptan 30 mg Rate (95% CI) 31.0 (28.1,33.9) 63.6 (60.6,66.6) 41.4 (38.3,44.5) 31.0 (28.1,33.9) 53.0 (49.9,56.1) 56.0 (52.9,59.1) 64.0 (61,67) 4.0 (2.8,5.2) 6.4 (4.9,7.9) 4.0 (2.8,5.2) 16 (8.2, 23.8) NSAID Rate (95% CI) 22.7 (20.1,25.3) 44.2 (41.1,47.3) 25.9 (23.2,28.6) 11.1 (9.2,13) 31.0 (28.1,33.9) 39.8 (36.8,42.8) 48.8 (45.7,51.9) 0.1 (0.0,0.3) 2.0 (1.1,2.9) 0.1 (0.0,0.3) 10.0 (3.8,16.2)

41 Favourable Effects Unfavourable Effects 41 Triptan: efficacy & safety data Decreased Pain Decreased Sensitivity Other Individual Risks Rapid Onset Outcome Units Scale Headache Relief Pain-free Response Sustained Response Reduced Sensitivity to Sound and Light Reduction in Functional Disability Reduction in Nausea and Vomiting Transient Triptans Sensations CNS Adverse Events Chest-related Adverse Events Myocardial Infarction % 0-70 % 0-70 % 0 70 % 0 70 % 0 70 % 0 70 % 0 70 % 0-7 % 0 7 % 0-7 No. / 1000 pat-yrs 8-16 Triptan 15 mg Rate (95% CI) 23.3 (20.7,25.9) 57.0 (53.9,60.1) 25.0 (22.3,27.7) 26.0 (23.3,28.7) 46.0 (42.9,49.1) 44.8 (41.7,47.9) 51.0 (47.9,54.1) 2.2 (1.3,3.1) 4.0 (2.8,5.2) 1.5 (0.7,2.3) 13.1 (6,20) Triptan 30 mg Rate (95% CI) 31.0 (28.1,33.9) 63.6 (60.6,66.6) 41.4 (38.3,44.5) 31.0 (28.1,33.9) 53.0 (49.9,56.1) 56.0 (52.9,59.1) 64.0 (61,67) 4.0 (2.8,5.2) 6.4 (4.9,7.9) 4.0 (2.8,5.2) 16 (8.2, 23.8) NSAID Rate (95% CI) 22.7 (20.1,25.3) 44.2 (41.1,47.3) 25.9 (23.2,28.6) 11.1 (9.2,13) 31.0 (28.1,33.9) 39.8 (36.8,42.8) 48.8 (45.7,51.9) 0.1 (0.0,0.3) 2.0 (1.1,2.9) 0.1 (0.0,0.3) 10.0 (3.8,16.2)

42 Favourable Effects Unfavourable Effects 42 Triptan: efficacy & safety data Decreased Pain Decreased Sensitivity Other Individual Risks Rapid Onset Outcome Units Scale Headache Relief Pain-free Response Sustained Response Reduced Sensitivity to Sound and Light Reduction in Functional Disability Reduction in Nausea and Vomiting Transient Triptans Sensations CNS Adverse Events Chest-related Adverse Events Myocardial Infarction % 0-70 % 0-70 % 0 70 % 0 70 % 0 70 % 0 70 % 0 70 % 0-7 % 0 7 % 0-7 No. / 1000 pat-yrs 8-16 Triptan 15 mg Rate (95% CI) 23.3 (20.7,25.9) 57.0 (53.9,60.1) 25.0 (22.3,27.7) 26.0 (23.3,28.7) 46.0 (42.9,49.1) 44.8 (41.7,47.9) 51.0 (47.9,54.1) 2.2 (1.3,3.1) 4.0 (2.8,5.2) 1.5 (0.7,2.3) 13.1 (6,20) Triptan 30 mg Rate (95% CI) 31.0 (28.1,33.9) 63.6 (60.6,66.6) 41.4 (38.3,44.5) 31.0 (28.1,33.9) 53.0 (49.9,56.1) 56.0 (52.9,59.1) 64.0 (61,67) 4.0 (2.8,5.2) 6.4 (4.9,7.9) 4.0 (2.8,5.2) 16 (8.2, 23.8) NSAID Rate (95% CI) 22.7 (20.1,25.3) 44.2 (41.1,47.3) 25.9 (23.2,28.6) 11.1 (9.2,13) 31.0 (28.1,33.9) 39.8 (36.8,42.8) 48.8 (45.7,51.9) 0.1 (0.0,0.3) 2.0 (1.1,2.9) 0.1 (0.0,0.3) 10.0 (3.8,16.2)

43 43 Define the effects Functional disability Proportion of patients who experience moderate or severe baseline disability with mild or no disability measured at 2 hours after dosing. MI Number of patients per 1000 patient-years with myocardial infarction within 48 hours of dosing. Modify these definitions so they are comprehensible to non-medical people. Operationalise as much as possible (e.g., moderate or severe, measured, MI)

44 44 Create scales with plausible ranges Functional disability Proportion of patients who experience moderate or severe baseline disability with mild or no disability measured at 2 hours after dosing. MI Number of patients per 1000 patient-years with myocardial infarction within 48 hours of dosing. Nonresponder Placebo mean

45 45 Rank swings for added therapeutic value Functional disability Proportion of patients who experience moderate or severe baseline disability with mild or no disability measured at 2 hours after dosing. MI Number of patients per 1000 patient-years with myocardial infarction within 48 hours of dosing. The red dots define your least preferred positions Starting at the red dots, would you prefer to move to 70 or to 8?

46 46 Weight swings for added value Functional disability Proportion of patients who experience moderate or severe baseline disability with mild or no disability measured at 2 hours after dosing. MI Number of patients per 1000 patient-years with myocardial infarction within 48 hours of dosing. Participants at the 2011 CIRS workshop judged 0 to 70 was more preferred. This swing was judged To be twice as valued as this swing,

47 47 Check consistency of weights Adjust weights to achieve consistency. Balance-beam comparisons Equal added therapeutic value Ratios of numbers represent ratios of added therapeutic value. Sums of numbers represent total added therapeutic value. Headache relief + Pain-free response give 30% more added therapeutic value than Sustained response

48 48 Weighting, in summary 1. Clarify the context 2. Identify efficacy and safety data 3. Define favourable and unfavourable effects 4. Create scales with plausible ranges 5. Rank criteria for their swings in therapeutic value from least to most preferable positions 6. Assign 100 to the largest swing; judge other swings as ratios to 100 of added clinical value 7. Do balance-beam comparisons; adjust weights to achieve consistency

49 49 How to apply it for patients weights Step 1: Assess swings for favourable effects Use paired comparison technique throughout Step 2: Assess swings for unfavourable effects Step 3: Compare 100 on FE with 100 on UFE

50 50 To sum up MCDA does not give the right answer. Nothing can because there is no right answer. MCDA does provide a useful tool for thinking, a structure for rational debate and a serious guide to decision making. It is a model that illuminates ; it provides clarity. MCDA enables rapid exploration of different perspectives on the issues and of uncertainty in the data. MCDA can be expanded with related model types However, MCDA requires careful design of social processes: engaging the right people in the right way at the right time.

51 51 A guide to further reading UIT Cambridge Ltd, Gives the sciencebased evidence about the harm of drugs. Dodgson, J., Spackman, M., Pearman, A., & Phillips, L. (2000). Multi-Criteria Analysis: A Manual. London: Department of the Environment, Transport and the Regions, republished 2009 by the Department for Communities and Local Government. Download free at MCDA is described in Chapter 6. CRC Press, Overview of B-R assessment across the R&D spectrum. Chapter 5 describes methods & applications, including MCDA.

52 52