How to Conduct a Meta-Analysis

Transcription

1 How to Conduct a Meta-Analysis Faculty Development and Diversity Seminar ludovic@bu.edu Dec 11th, 2017

2 Periodontal disease treatment and preterm birth We conducted a metaanalysis of randomized controlled trials to determine whether periodontal disease treatment with scaling and/or root planing during pregnancy may reduce preterm birth incidence Treatment resulted in significantly lower preterm birth incidence (odds ratio, 0.55, 95% confidence interval, 0.35 to 0.86; P=.008) Polyzos et al. Am J Obstet Gynecol /23

3 Periodontal disease treatment and preterm birth 11 trials (6558 women) were included Treatment had no significant effect on the overall rate of preterm birth (odds ratio 1.15, 95% confidence interval 0.95 to 1.40; P=0.15) Polyzos et al. BMJ /23

4 3/23 What is a meta-analysis? Estimation of a common effect size across studies Allows exploring heterogeneity Optional part of a systematic review SRs MAs

5 4/23 Meta-analysis is a two-stage process 1. an effect size and its variance is calculated for each study 2. a summary (pooled) effect estimate is calculated as a weighted mean of the effect sizes estimated in the individual studies To compute the weighted mean, we assign more weight to the more precise studies

6 5/23 Periodontal disease treatment and preterm birth Treatment No treatment Study x n x n Weight OR [95% CI] Sadatmansuri % 0.12 [ 0.01, 2.45 ] Jeffcoat % 0.43 [ 0.15, 1.28 ] Offenbacher % 0.45 [ 0.16, 1.25 ] Lopez % 0.55 [ 0.22, 1.39 ] Lopez % 0.51 [ 0.26, 1.02 ] Tarranum % 0.36 [ 0.19, 0.67 ] Oliveira % 0.84 [ 0.46, 1.53 ] Michalowicz % 1.14 [ 0.72, 1.81 ] Macones % 1.29 [ 0.85, 1.95 ] Newnham % 1.04 [ 0.69, 1.56 ] Offenbacher % 1.27 [ 0.92, 1.76 ] IV, fixed effect model Q= 25.9, P value= ; I²= 61.4 % Favours treatment % 0.94 [ 0.80, 1.11 ] Favours no treatment Odds Ratio (log scale)

7 6/23 Quantifying heterogeneity Differences across studies in a meta-analysis (statistical) heterogeneity = variation in the true effects underlying different studies which may manifest itself in more observed variation than expected by chance alone

8 Common way to quantify heterogeneity I 2 coefficient between 0% (no observed heterogeneity) and 100% Proportion of total variability explained by heterogeneity 0% to 40%: might not be important 30% to 60%: may represent moderate heterogeneity 50% to 90%: may represent substantial heterogeneity 75% to 100%: considerable heterogeneity Higgins and Thompson. Stat Med /23

9 8/23 Fixed-effect meta-analysis Treatment No treatment Study x n x n Weight OR [95% CI] Sadatmansuri % 0.12 [ 0.01, 2.45 ] Jeffcoat % 0.43 [ 0.15, 1.28 ] Offenbacher % 0.45 [ 0.16, 1.25 ] Lopez % 0.55 [ 0.22, 1.39 ] Lopez % 0.51 [ 0.26, 1.02 ] Tarranum % 0.36 [ 0.19, 0.67 ] Oliveira % 0.84 [ 0.46, 1.53 ] Michalowicz % 1.14 [ 0.72, 1.81 ] Macones % 1.29 [ 0.85, 1.95 ] Newnham % 1.04 [ 0.69, 1.56 ] Offenbacher % 1.27 [ 0.92, 1.76 ] IV, fixed effect model Q= 25.9, P value= ; I²= 61.4 % Favours treatment % 0.94 [ 0.80, 1.11 ] Favours no treatment Odds Ratio (log scale)

10 9/23 Fixed-effect meta-analysis Treatment No treatment Study x n x n Weight OR [95% CI] Sadatmansuri % 0.12 [ 0.01, 2.45 ] Jeffcoat % 0.43 [ 0.15, 1.28 ] Offenbacher % 0.45 [ 0.16, 1.25 ] Lopez % 0.55 [ 0.22, 1.39 ] Lopez 2005 Tarranum 2007 Oliveira Random error 5.91% 0.51 [ 0.26, 1.02 ] 6.89% 0.36 [ 0.19, 0.67 ] 7.70% 0.84 [ 0.46, 1.53 ] Michalowicz % 1.14 [ 0.72, 1.81 ] Macones % 1.29 [ 0.85, 1.95 ] Newnham % 1.04 [ 0.69, 1.56 ] Offenbacher % 1.27 [ 0.92, 1.76 ] IV, fixed effect model True common effect size % 0.94 [ 0.80, 1.11 ] Odds Ratio (log scale)

11 10/23 Random-effects meta-analysis Treatment No treatment Study x n x n Weight OR [95% CI] Sadatmansuri % 0.12 [ 0.01, 2.45 ] Jeffcoat % 0.43 [ 0.15, 1.28 ] Offenbacher % 0.45 [ 0.16, 1.25 ] Lopez % 0.55 [ 0.22, 1.39 ] Lopez % 0.51 [ 0.26, 1.02 ] Tarranum % 0.36 [ 0.19, 0.67 ] Oliveira % 0.84 [ 0.46, 1.53 ] Michalowicz % 1.14 [ 0.72, 1.81 ] Macones % 1.29 [ 0.85, 1.95 ] Newnham % 1.04 [ 0.69, 1.56 ] Offenbacher % 1.27 [ 0.92, 1.76 ] IV, random effects model Q= 25.9, P value= ; I²= 61.4 % Favours treatment % 0.79 [ 0.58, 1.06 ] Favours no treatment Odds Ratio (log scale)

12 11/23 Random-effects meta-analysis Treatment No treatment Study x n x n Weight OR [95% CI] Sadatmansuri 2006 Jeffcoat 2003 Offenbacher True effect underlying each study 0.90% 0.12 [ 0.01, 2.45 ] 5.23% 0.43 [ 0.15, 1.28 ] 5.63% 0.45 [ 0.16, 1.25 ] Lopez % 0.55 [ 0.22, 1.39 ] Lopez 2005 Tarranum 2007 Oliveira Random error 9.12% 0.51 [ 0.26, 1.02 ] 9.77% 0.36 [ 0.19, 0.67 ] 10.24% 0.84 [ 0.46, 1.53 ] Michalowicz % 1.14 [ 0.72, 1.81 ] Macones % 1.29 [ 0.85, 1.95 ] Newnham % 1.04 [ 0.69, 1.56 ] Offenbacher % 1.27 [ 0.92, 1.76 ] Random effects model True mean effect size Distribution of effects θ i ~ N(θ, τ 2 ) % 0.79 [ 0.58, 1.06 ] Odds Ratio (log scale)

13 12/23 Assessing the quality of a body of evidence Biases resulting from flaws in RCT design and conduct Small-study effect and reporting biases Publication bias Selective reporting of outcomes

14 Flaws in the design and conduct of trials 13/23

15 14/23 Effects of flaws in the design and conduct of trials Including biased trials will cause meta-analyses to be biased Change in average intervention effect (bias) Increase in between-trial heterogeneity Effect of bias will vary across meta-analyses

16 15/23 Incorporating risk of bias assessments into analysis Including all studies in a MA may produce a result with high precision but biased because of flaws in some of the studies Including only the studies at low risk of bias may produce an unbiased result but imprecise It is recommended to restrict the primary analysis to studies at low (or low and unclear) risk of bias

17 Subgroup analysis according to Risk of Bias Polyzos et al. BMJ /23

18 17/23 Funnel plot to assess small-study effect 0 Standard Error Effect 10

19 18/23 Asymmetrical funnel plot 0 Standard Error Effect 10

20 19/23 Asymmetrical funnel plot 0 Standard Error 1 2 Unpublished studies Effect 10

21 20/23 Asymmetrical funnel plot 0 Standard Error 1 2 Small studies all finding positive effects Effect 10

22 Small-study effect Polyzos et al. BMJ /23

23 22/23 Take-home messages Meta-analysis is the statistical combination of results from two or more separate studies Improve the estimation of treatment effects by pooling results of similar studies and exploring sources of heterogeneity Potential to mislead seriously if variation across studies, within-study biases and reporting biases are not carefully considered We must consider the degree of statistical heterogeneity, risk of bias and small-study effects when selecting the most appropriate statistical model to combine the evidence The best decision may be to focus on the best available evidence (ie, larger and methodologically robust trials)

24 23/23 Resources Cochrane Handbook Cochrane Review Manager OpenMeta PRISMA reporting guideline