Indirect Comparisons: heroism or heresy. Neil Hawkins With acknowledgements to Sarah DeWilde

Transcription

1 Indirect Comparisons: heroism or heresy Neil Hawkins With acknowledgements to Sarah DeWilde 1

2 Outline Brief introduction to indirect comparisons Two practical examples Heroism or Heresy? 2

3 A Taxonomy of Comparisons A B Direct Comparison A B Naive or Unadjusted Indirect Comparison: Absolute effect estimates from individual trial arms. A B C C Adjusted indirect comparison: Difference between relative treatment A A B B C C Network Meta-Analysis: Adjusted Indirect comparison extended to more complex networks of trial evidence 3

4 Adjusted Indirect Comparisons Based on an assumption of transitivity HR AB = HR AC /HR BC Log Transformation: LN HR AB = LN HR AC LN HR BC 4

5 Assumptions Transitivity on the chosen scale:a-b = (A-C)-(B-C) Requires exchangeability of relative treatment effects: Between Subjects Within trials (randomisation) Between trials including the same comparators (pairwise meta-analysis) Between trials comparing different comparators Ultimately, between different treatment comparisons 5

6 Indirect comparisons: heroism or heresy? 6

7 Stuart Pocock: Heresy their statistical methods are so complex, an extension of multivariable Bayesian hierarchical random effects models for mixed multiple treatment comparisons that many are mystified by whether the conclusions make sense. from Safety of drug-eluting stents: demystifying network meta-analysis Vol 370 December 22/29,

8 Lu & Ades: Heroism... to ignore indirect evidence either makes the unwarranted claim that it is irrelevant, or breaks the established precept of systematic review that synthesis should embrace all available evidence. Stat Med 2004;23:

9 Cochrane: Not Sure Indirect comparisons are not randomized comparisons, and cannot be interpreted as such. They are essentially observational findings across trials, and may suffer the biases of observational studies, for example due to confounding. Reference: Cochrane Handbook

10 Ref: NEJM 359;12 10

11 Could the (-ve) results of PRoFESS have been predicted? ESPS2 ESPRIT CAPRIE ASP ASP + ER DP CLOP 11

12 Network Graph (Stroke Endpoint) CLOP ASP + ER DP 0.92(0.8:1.07) 0.79(0.67:0.92) ASP 12

13 ASP + ERDP vs. CLOP Indirect Comparison: ESPS2, ESPRIT & CAPRIE Trials Odds Ratio 0.85 ( 0.66:1.06 ) Direct Comparison: PRoFESS Trial Odds Ratio 1.02 (0.93 to 1.11) What happened? Indirect Comparison ASP + ERDP vs CLOP Odds Ratio 0.85 (0.69:1.06) 13

14 Kent & Thaler (2008) the results of the PRoFESS trial show us once again that the compelling logic of the transitive property, so reliable in mathematics, has little authority in the often illogical world of clinical trials Although the inconsistency among trial results should make us examine the trials for differences in design or populations that might support explanatory hand-waving, it is also reasonable to conclude from these comparisons that efficacy should not be the sole, or perhaps even the major, determinant of treatment decisions for antiplatelet therapy after stroke. 14

15 Extended Trial Network MATCH ESPS2 ESPRIT CHARISMA ASP + CLOP CAPRIE ASP + ER DP ATC2002 ASP high dose ASP low dose ASP med dose CLOP 15

16 Extended Network Graph ASP low dose 1.5(1.34:1.68) 1.31(1.08:1.6) ASP high dose ASP + ER DP 1.08(0.94:1.25) 1.59(1.39:1.83) 0.84(0.64:1.08) 1.06(0.95:1.18) CLOP 1.08(0.93:1.25) ASP + CLOP 0.83(0.66:1.05) ASP med dose 16

17 ASP + ERDP vs CLOP Extended Network Meta-Analysis: Odds Ratio 1.11 ( 0.87 to1.4) Direct Comparison: PRoFESS Trial Odds Ratio 1.02 (0.93 to 1.11) Extended Network Meta-Analysis + PRoFESS Odds Ratio 1.03 ( 0.94 to 1.13 ) 17

18 Kent and Thaler (2008) In the era of comparative effectiveness, when multiple agents are pitted against one another, randomized trials often cannot be understood in isolation. Rather, they need to be interpreted in the context of a sometimes complex network of other similar or relevant evidence. The reduction of such complex networks to treatment recommendations is not always straightforward, since different paths within the network may give inconsistent results, and the network may be incoherent 18

19 Ref: Statist. Med. 2007; 26:

20 NMA of Anti-TNFs in RA MTX Adalimunab Infliximab Etanercept Anakinra Log Odds Ratio of ACR50 20

21 Ref: Statist. Med. 2007; 26:

22 Adjusted for Disease Duration MTX Adalimunab Infliximab Etanercept Anakinra Log Odds Ratio of ACR50 22

23 Final thought: exchangeability is implicit in clinical decision-making RCT: A vs Placebo: Exchangeability RCT: B vs Placebo: Future Patient: 23

24 Heroism, Heresy, or Pragmatism Formal methods of indirect comparisons are invaluable for analysing more complex networks Assumptions required for indirect comparisons are related to the assumptions of generalisability Identifying and seeking to explain the heterogeneity and incoherence identified in networks of trial evidence should improve our understanding of comparative effectiveness 24