Aussagekraft von Metaanalysen. Prof. Stefan Leucht Klinik für Psychiatrie und Psychotherapie der TU-München

Aussagekraft von Metaanalysen Prof. Stefan Leucht Klinik für Psychiatrie und Psychotherapie der TU-München

Disclosures In the past 3 years: Consulting/advisory board honoraria from Alkermes, Bristol- Myers Squibb, Eli Lilly, Janssen, Johnson & Johnson, Lundbeck, MedAvante, Roche Lecture honoraria from AstraZeneca, Bristol-Myers Squibb, Eli Lilly, Essex Pharma, Janssen, Johnson & Johnson, Lundbeck Institute, Pfizer, Sanofi-Aventis Eli Lilly has provided medication for a trial with Stefan Leucht as the primary investigator

Definitions Systematic Review: Means the systematic approach in terms of literature search, selection, presentation and analysis of the data Meta-analysis: Means the mathematical combination of the results of different studies on one question

Why do we need systematic reviews? In 10000 medical journals 2 million articles are published every year A general practicioner would have to read 19 articles everyday, 365 days per year to cover relevant reports More than 300 randomised controlled studies about the atypical antipsychotics are available

The principal steps in the development of a systematic review

I. BEFORE: Writing a protocol Which patients Which interventions Which outcomes Literature search (databases, search strings) Statistical method The protocol is reviewed by two editors of the Cochrane Schizophrenia Group and it is published in the Cochrane Library

II. Literature search Not only MEDLINE Not only English/Dutch Electronic databases, conference abstract books, book chapters, contacting pharmaceutical companies, contacting study authors, FDA webpage

Calculation of Effect Sizes for Continuous Variables Effect size = (mean A mean B)/pooled standard deviation Example: (90 80)/20 = 0.50

Ilustration of the meaning of effect size source : http://rpsychologist.com/d3/cohend/

Cohen s rule Standardised mean difference of 0.20 = small 0.50 = medium 0.80 = large

Principle of meta-analysis, example: Olanzapine versus quetiapine for schizophrenia (Komossa et al. Cochrane review 2009)

BPRS: Amisulpride vs. typical antipsychotics Rüther (1989) vs. perazine 1 Pichot (1988) Ziegler (1989) Klein (1985) Costa e Silva (1989) Delcker (1990) Möller (1997) Wetzel (1999) vs. flupentixol Puech (1998) Carrière (2000) Colonna (2000) Amisulpride pooled: r = 0.11, p<0,0001, 10 studies, n = 1654-0.4-0.3-0.2-0.1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 1 endpoint analysis, not used for mean effect size r (95% CI) Leucht et al. Am J Psychiatry 2002

BPRS: Amisulpride vs. typical antipsychotics Leucht 2008 Rüther (1989) vs. perazine 1 Pichot (1988) Ziegler (1989) Klein (1985) Costa e Silva (1989) Delcker (1990) Möller (1997) Wetzel (1999) vs. flupentixol Puech (1998) Carrière (2000) Colonna (2000) Amisulpride pooled: r = 0.11, p<0,0001, 10 studies, n = 1654-0.4-0.3-0.2-0.1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 1 endpoint analysis, not used for mean effect size r (95% CI) Leucht et al. Am J Psychiatry 2002

III. Data extraction 2. Dichotomous Variables (yes no Outcomes, e.g. stroke, relapse) For both groups: Number of events (e.g. relapse), total N Effect size measures: Absolute risk difference, relative risk difference, Odds Ratio

Calculation of Effect Sizes for Dichotomous Variables Risk: 1 out of 10 patients relapsed, i.e. 1/10 = 0.1 (or 10%) Absolute risk (response) difference Risk of the intervention group Risk of the control group, e.g. 2%-4% = (-) 2% Relative risk reduction (or response ratio) Risk of the intervention group divided by the risk of the control group, e.g. 2% / 4% = 0.5 = 50% Number needed to treat How many patients must be treated to have one relapse less? Inverse of the absolute risk difference here 1/2% = 1/0.02 = 50

Reduction in the Risk of Dying From Breast Cancer 100,000 Women without mammography 100,000 Women with mammography Breast cancer mortality in 10 years 0.36% (360/100,000) 0.29% (290/100,000) Relative mortality reduction = 20% (1 [0.29%/0.36%]) Absolute mortality reduction = 0.07% (0.36%-0.29%) Data from Kürzl Deutsches Ärzteblatt 9/2004

Meta-analyses are often the only way to objectively summarise the evidence if there are many studies

The overall outcome reported in the abstract of head to head comparisons of atypical antipsychotics strongly depends on the sponsor In a blinded analysis of the abstracts of 33 head to head comparisons of atypical antipsychotics in about 90% the overall outcome was in favour of the sponsor Heres et al. Am J Psych 2006

Leucht et al. Lancet 2009 220 RCTs 150 db RCTs 20000 participants 10 SGAs 10 outcomes Multiple moderator analyses

First episode versus multiple episode patients (relapse 7 12 months) Rationale: 20% of first episode patients will not have a 2nd episode within 5 years (Robinson et al. Arch Gen Psych 1999, Shepherd et al. BrJPsych Suppl. 1994) They are thought to have a better prognosis Do they need maintenance treatment? Problem: 20% can not be identified in advance Study or Subgroup 2.1.1 first episode Boonstra 2010 Chen 2010 Crow 1986 Hogarty 1973 Kane 1982 McCreadie 1989 Pietzcker 1993 Rifkin 1979 Subtotal (95% CI) Total events Favours experimental Control Risk Ratio Risk Ratio Events Total Events Total Weight M-H, Random, 95% CI M-H, Random, 95% CI 2 27 20 10 0 0 7 1 9 89 54 36 11 8 36 12 255 67 167 Heterogeneity: Tau² = 0.00; Chi² = 5.85, df = 7 (P = 0.56); I² = 0% Test for overall effect: Z = 6.72 (P < 0.00001) 2.1.2 not first episode Andrews 1976 Arato 2002 Cheung 1981 Doddi 1979 Eklund 1991 Hirsch 1973 Hogarty 1973 Hough 2010 Kramer 2007 Leff 1971 Marjerrison 1964 Nishikawa 1982 Nishikawa 1984 Odejide 1982 Pfizer 2000 Pietzcker 1993 Rifkin 1979 Sampath 1992 Troshinsky 1962 Subtotal (95% CI) Total events 1 73 2 1 2 3 52 45 33 7 4 16 35 5 24 13 4 4 1 325 15 207 15 10 20 41 156 206 105 20 54 20 74 35 71 86 39 12 24 1210 6 50 8 3 16 25 107 130 82 12 2 10 13 15 43 49 14 9 12 606 17 71 15 10 23 40 143 204 102 15 34 10 13 35 75 75 18 12 19 931 1.2% 9.2% 2.3% 1.1% 2.4% 3.1% 9.2% 8.9% 8.8% 5.7% 1.7% 9.1% 9.1% 4.1% 8.1% 6.7% 3.8% 4.3% 1.3% 100.0% Heterogeneity: Tau² = 0.14; Chi² = 69.45, df = 18 (P < 0.00001); I² = 74% Test for overall effect: Z = 7.93 (P < 0.00001) 10 56 42 24 7 4 23 1 11 89 66 39 17 7 40 4 273 Test for subgroup differences: Chi² = 1.36, df = 1 (P = 0.24), I² = 26.5% 3.2% 39.2% 31.7% 14.4% 0.6% 0.6% 9.5% 0.8% 100.0% 0.24 [0.07, 0.84] 0.48 [0.34, 0.69] 0.58 [0.39, 0.86] 0.45 [0.25, 0.81] 0.10 [0.01, 1.59] 0.10 [0.01, 1.56] 0.34 [0.17, 0.69] 0.33 [0.03, 4.19] 0.47 [0.38, 0.58] 0.19 [0.03, 1.40] 0.50 [0.39, 0.64] 0.25 [0.06, 0.99] 0.33 [0.04, 2.69] 0.14 [0.04, 0.55] 0.12 [0.04, 0.36] 0.45 [0.35, 0.57] 0.34 [0.26, 0.45] 0.39 [0.29, 0.53] 0.44 [0.23, 0.84] 1.26 [0.24, 6.51] 0.82 [0.64, 1.07] 0.49 [0.38, 0.64] 0.33 [0.14, 0.82] 0.59 [0.40, 0.86] 0.23 [0.14, 0.39] 0.13 [0.05, 0.34] 0.44 [0.19, 1.05] 0.07 [0.01, 0.46] 0.39 [0.31, 0.49] 0.01 0.1 1 10 100 Favours experimental Favours control Leucht et al. Lancet 2012 Cochrane Database Syst Rev 2012

Meta-analyses are often the only way to objectively summarise the evidence if there are many studies They help to clarify hidden issues

Haloperidol reduces mania more than some second generation antipsychotics Scherk et al. Arch Gen Psych 2007

Meta-analyses are often the only way to objectively summarise the evidence if there are many studies They help to clarify hidden issues They sometimes reject dogmas

Time course of antipsychotic effect Psychotic symptoms after subtraction of placebo effect Percentage improvement per week 16 14 12 10 8 6 4 2 Meta-analysis of 53 studies with 8,177 patients 0 0 1 2 Week 3 4 5 Slide obtained with generous permission from Ofer Agid Agid et al. Arch Gen Psychiatry 2003; 60: 1228 1235

Meta-analysis of the cognitive effects of conventional antipsychotics (Mishara and Goldberg Biol Psych 2004)

Kirsch I, Deacon BJ, Huedo Medina TB, Scoboria A, Moore TJ, Johnson BT. Initial severity and antidepressant benefits: a meta analysis of data submitted to the Food and Drug Administration. PLoS Med 2008; 5: e45. Menand L. Head case: can psychiatry be a science? The New Yorker 2010; 1 March: 68 74.

Vergleich von 94 Metaanalysen somatischer Krankheitsbilder mit 33 Metaanalysen psychiatrischer Krankheitsbilder Nicht- Psychopharmaka Psychopharmaka SMD (median) 0.37 0.41 SMD (mean) 0.45 0.49 95% Konfidenzintervall 0.37-0.53 0.41-0.57 Leucht S, Hierl S, Kissling W, Dold M, Davis JM. Putting the efficacy of psychiatric and general medicine medication into perspective: review of meta analyses. Br J Psychiatry 2012, 200: 97 106.

Meta-analyses are often the only way to objectively summarise the evidence if there are many studies They help to clarify hidden issues They sometimes reject dogmas They sometimes show that there is no evidence

Meta-analyses of old antipsychotic drugs with regionally restricted use Perazine only 5 small randomised studies,n = 286! Benperidol only one small RCT (unpublished manuscript) Leucht and Hartung Cochrane Database of systematic reviews 2006

Example for metaregression: decreasing antipsychotic drug versus placebo difference over time (Leucht et al. Molecular Psychiatry 2009)

Latest methodological development: Multiple Treatments Meta-analysis ( network meta-analysis ) 1. Cipriani et al. Lancet 2009;373:746 758; 2. Cipriani et al. Lancet 2011;378:1306 1315

Multiple treatments meta analysis. Overall probability of antimanic treatments to be the best terms of both efficacy and acceptability, showing the separate contributions to the overall scores of efficacy (blue) and acceptability (red). 100 80 87 79 75 68 60 40 62 59 50 43 41 39 Acceptability 20 23 21 7 3 Efficacy 0 Cipriani et al. Lancet 2011

Advantages of Multiple-Treatments Meta-analysis 1. Uses all the data (direct and indirect) 2. Allows to calculate a hierarchy of drugs for an outcome: this is what guidelines need!

Principle of Multiple Treatments Metaanalysis ( network meta-analysis ) B A Examples: 1. Cipriani et al. Lancet 2009;373:746 758; 2. Cipriani et al. Lancet 2011;378:1306 1315 C There are trials for A vs B and A vs C but none for B vs C MTM, multiple treatments meta-analysis

Principle of MTM B B A C There are trials to compare A vs B and A vs C, but none to compare B vs C A C Trial results to compare B vs C can be estimated from those of A vs B and A vs C MTM, multiple treatments meta-analysis S Leucht, personal communication

Main problem of Multiple-Treatments Meta-analysis Can direct and indirect evidence be combined? The coherence of direct and indirect evidence is examined by statistical tests

Limitations of Meta-analyses The apples and oranges problem - heterogeneity, different study quality etc. In meta-analysis there are many judgement calls The original studies are frequently so poorly reported that meta-analytic procedures are not possible Publication bias

Funnel-plot without publication bias Funnel-plot showing possible publication bias (n) 700 600 500 400 300 200 100 0-0,3-0,2-0,1 0 0,1 (n) 700 600 500 400 300 200 100 0-0,3-0,2-0,1 0 0,1

The results of meta-analyses are consistent The effect size of haloperidol versus placebo derived from 11 double-blind trials with 1540 participants, which is used as a benchmark. The effect sizes of the SGAs compared to FGAs have been added to haloperidol versus placebo for illustration. HAL, haloperidol; AMI, amisulpride; ARI, aripiprazole; CLO, clozapine; OLA, olanzapine; QUE, quetiapine; RIS, risperidone; SER, sertindole; ZIP, ziprasidone; ZOT, zotepine; SGA, second-generation antipsychotics; FGA, first-generation antipsychotics Reproduced with permission Leucht et al. Psychological Med 2009;39:1591 1602

Thank you for your attention