Meta-analysis of few small studies in small populations and rare diseases

Similar documents
Meta-analysis of two studies in the presence of heterogeneity with applications in rare diseases

Bayesian random-effects meta-analysis made simple

HIERARCHICAL MODELS A framework for evidence synthesis

Applications of Bayesian methods in health technology assessment

CLINICAL REGISTRIES Use and Emerging Best Practices

Methods for evidence synthesis in the case of very few studies. Guido Skipka IQWiG, Cologne, Germany

A Re-Analysis of the Cochrane Library Data: The Dangers of Unobserved Heterogeneity in Meta-Analyses

Kernel Density Estimation for Random-effects Meta-analysis

A re-evaluation of random-effects meta-analysis

ST440/550: Applied Bayesian Statistics. (10) Frequentist Properties of Bayesian Methods

Changing EDSS progression in placebo cohorts in relapsing MS:

Treatment effect estimates adjusted for small-study effects via a limit meta-analysis

University of Bristol - Explore Bristol Research. Peer reviewed version. Link to published version (if available): /jrsm.

A Brief Introduction to Bayesian Statistics

Introduction to Bayesian Analysis 1

A Case Study: Two-sample categorical data

Evidence synthesis with limited studies

Modelling heterogeneity variances in multiple treatment comparison meta-analysis Are informative priors the better solution?

Bayesian performance

Dynamic borrowing of historical data: Performance and comparison of existing methods based on a case study

A note on the graphical presentation of prediction intervals in random-effects meta-analyses

Historical controls in clinical trials: the meta-analytic predictive approach applied to over-dispersed count data

Advanced IPD meta-analysis methods for observational studies

A FRAMEWORK FOR SIMULATIONS IN CLINICAL RESEARCH

Models for potentially biased evidence in meta-analysis using empirically based priors

heterogeneity in meta-analysis stats methodologists meeting 3 September 2015

Meta-analysis using individual participant data: one-stage and two-stage approaches, and why they may differ

Meta-analysis of external validation studies

Performance of the Trim and Fill Method in Adjusting for the Publication Bias in Meta-Analysis of Continuous Data

Estimation of the predictive power of the model in mixed-effects meta-regression: A simulation study

Bayesian Mediation Analysis

Using Statistical Principles to Implement FDA Guidance on Cardiovascular Risk Assessment for Diabetes Drugs

Statistical Tolerance Regions: Theory, Applications and Computation

How to use prior knowledge and still give new data a chance?

Bayesian Tolerance Intervals for Sparse Data Margin Assessment

Meta Analysis. David R Urbach MD MSc Outcomes Research Course December 4, 2014

University of Bristol - Explore Bristol Research. Peer reviewed version. Link to published version (if available): /bmsp.

Bayesian Analysis of Between-Group Differences in Variance Components in Hierarchical Generalized Linear Models

An Empirical Assessment of Bivariate Methods for Meta-analysis of Test Accuracy

Bayesian Dose Escalation Study Design with Consideration of Late Onset Toxicity. Li Liu, Glen Laird, Lei Gao Biostatistics Sanofi

An Introduction to Bayesian Statistics

Methods Research Report. An Empirical Assessment of Bivariate Methods for Meta-Analysis of Test Accuracy

Comparing treatments evaluated in studies forming disconnected networks of evidence: A review of methods

Bayesian growth mixture models to distinguish hemoglobin value trajectories in blood donors

Index. Springer International Publishing Switzerland 2017 T.J. Cleophas, A.H. Zwinderman, Modern Meta-Analysis, DOI /

No Large Differences Among Centers in a Multi-Center Neurosurgical Clinical Trial

Meta-analysis: Basic concepts and analysis

Phase I dose-escalation trials with more than one dosing regimen

University of Bristol - Explore Bristol Research

Empirical assessment of univariate and bivariate meta-analyses for comparing the accuracy of diagnostic tests

Thresholds for statistical and clinical significance in systematic reviews with meta-analytic methods

Bayesian Models for Combining Data Across Domains and Domain Types in Predictive fmri Data Analysis (Thesis Proposal)

Using mixture priors for robust inference: application in Bayesian dose escalation trials

APPLYING THE LAW OF ITERATED LOGARITHM TO CUMULATIVE META-ANALYSIS OF A CONTINUOUS ENDPOINT

TRIPODS Workshop: Models & Machine Learning for Causal I. & Decision Making

Estimating the Power of Indirect Comparisons: A Simulation Study

Estimation of effect sizes in the presence of publication bias: a comparison of meta-analysis methods

Bayes-Verfahren in klinischen Studien

Meta-analysis to compare combination therapies: A case study in kidney transplantation

Bayesian and Frequentist Approaches

Bayesian Confidence Intervals for Means and Variances of Lognormal and Bivariate Lognormal Distributions

Practical Bayesian Design and Analysis for Drug and Device Clinical Trials

Project Funded under FP7 - HEALTH Grant Agreement no Funded under FP7 - HEALTH Grant Agreement no

C2 Training: August 2010

Combining Risks from Several Tumors Using Markov Chain Monte Carlo

Selection and estimation in exploratory subgroup analyses a proposal

Bayesian hierarchical modelling

Missing data. Patrick Breheny. April 23. Introduction Missing response data Missing covariate data

Bayesian methods in health economics

Bayesian Models for Combining Data Across Subjects and Studies in Predictive fmri Data Analysis

Bayesian Estimation of a Meta-analysis model using Gibbs sampler

Hierarchical Meta-Analysis: A Simulation Study Comparing Classical Random Effects and Fully Bayesian Methods

Package SAMURAI. February 19, 2015

2014 Modern Modeling Methods (M 3 ) Conference, UCONN

Fundamental Clinical Trial Design

Bayesians methods in system identification: equivalences, differences, and misunderstandings

The basic premise of meta-analysis is that the average of

A Comparison of Methods for Determining HIV Viral Set Point

1 Introduction. st0020. The Stata Journal (2002) 2, Number 3, pp

Att vara eller inte vara (en Bayesian)?... Sherlock-conundrum

Trial Sequential Analysis (TSA)

Bayesian Meta-Analysis in Drug Safety Evaluation

Bayesian Logistic Regression Modelling via Markov Chain Monte Carlo Algorithm

Comparison of Meta-Analytic Results of Indirect, Direct, and Combined Comparisons of Drugs for Chronic Insomnia in Adults: A Case Study

Sampling Weights, Model Misspecification and Informative Sampling: A Simulation Study

Day Hospital versus Ordinary Hospitalization: factors in treatment discrimination

An Exercise in Bayesian Econometric Analysis Probit and Linear Probability Models

Appendix 1. Sensitivity analysis for ACQ: missing value analysis by multiple imputation

Effective Implementation of Bayesian Adaptive Randomization in Early Phase Clinical Development. Pantelis Vlachos.

Bayesian Methods LABORATORY. Lesson 1: Jan Software: R. Bayesian Methods p.1/20

For any unreported outcomes, umeta sets the outcome and its variance at 0 and 1E12, respectively.

Bayesian Model Averaging for Propensity Score Analysis

Bayes Factors for t tests and one way Analysis of Variance; in R

AALBORG UNIVERSITY. Prediction of the Insulin Sensitivity Index using Bayesian Network. Susanne G. Bøttcher and Claus Dethlefsen

Institutional Ranking. VHA Study

Methods for meta-analysis of individual participant data from Mendelian randomization studies with binary outcomes

Bayesian meta-analysis of Papanicolaou smear accuracy

Bayesian Nonparametric Methods for Precision Medicine

Transcription:

Meta-analysis of few small studies in small populations and rare diseases Christian Röver 1, Beat Neuenschwander 2, Simon Wandel 2, Tim Friede 1 1 Department of Medical Statistics, University Medical Center Göttingen, Göttingen, Germany 2 Novartis Pharma AG, Basel, Switzerland September 7, 2015 This project has received funding from the European Union s Seventh Framework Programme for research, technological development and demonstration under grant agreement number FP HEALTH 2013-602144. C. Röver et al. Meta-analysis of few small studies September 7, 2015 1 / 13

Overview Meta analysis the random-effects model frequentist approaches the Bayesian approach example Simulation study heterogeneity estimation effect estimation Conclusions C. Röver et al. Meta-analysis of few small studies September 7, 2015 2 / 13

Meta analysis The random effects model assume 1,2 : y i Normal(θ i, s 2 i ), θ i Normal(Θ, τ 2 ) y i Normal(Θ, s i 2 +τ 2 ) model components: Data: estimates y i standard errors s i Parameters: true parameter value Θ heterogeneity τ 1 L. V. Hedges, I. Olkin. Statistical methods for meta-analysis. Academic Press, 1985. 2 J. Hartung, G. Knapp, B. K. Sinha. Statistical meta-analysis with applications. Wiley, 2008. C. Röver et al. Meta-analysis of few small studies September 7, 2015 3 / 13

Meta analysis The random effects model assume 1,2 : y i Normal(θ i, s 2 i ), θ i Normal(Θ, τ 2 ) y i Normal(Θ, s i 2 +τ 2 ) model components: Data: estimates y i standard errors s i Parameters: true parameter value Θ heterogeneity τ 1 L. V. Hedges, I. Olkin. Statistical methods for meta-analysis. Academic Press, 1985. 2 J. Hartung, G. Knapp, B. K. Sinha. Statistical meta-analysis with applications. Wiley, 2008. C. Röver et al. Meta-analysis of few small studies September 7, 2015 3 / 13

Meta analysis The random effects model assume 1,2 : y i Normal(θ i, s 2 i ), θ i Normal(Θ, τ 2 ) y i Normal(Θ, s i 2 +τ 2 ) model components: Data: estimates y i standard errors s i Parameters: true parameter value Θ heterogeneity τ 1 L. V. Hedges, I. Olkin. Statistical methods for meta-analysis. Academic Press, 1985. 2 J. Hartung, G. Knapp, B. K. Sinha. Statistical meta-analysis with applications. Wiley, 2008. C. Röver et al. Meta-analysis of few small studies September 7, 2015 3 / 13

Meta analysis The random effects model assume 1,2 : y i Normal(θ i, s 2 i ), θ i Normal(Θ, τ 2 ) y i Normal(Θ, s i 2 +τ 2 ) model components: Data: estimates y i standard errors s i Parameters: true parameter value Θ heterogeneity τ 1 L. V. Hedges, I. Olkin. Statistical methods for meta-analysis. Academic Press, 1985. 2 J. Hartung, G. Knapp, B. K. Sinha. Statistical meta-analysis with applications. Wiley, 2008. C. Röver et al. Meta-analysis of few small studies September 7, 2015 3 / 13

Meta analysis The random effects model assume 1,2 : y i Normal(θ i, s 2 i ), θ i Normal(Θ, τ 2 ) y i Normal(Θ, s i 2 +τ 2 ) model components: Data: estimates y i standard errors s i Parameters: true parameter value Θ heterogeneity τ 1 L. V. Hedges, I. Olkin. Statistical methods for meta-analysis. Academic Press, 1985. 2 J. Hartung, G. Knapp, B. K. Sinha. Statistical meta-analysis with applications. Wiley, 2008. C. Röver et al. Meta-analysis of few small studies September 7, 2015 3 / 13

Meta analysis The random effects model assume 1,2 : y i Normal(θ i, s 2 i ), θ i Normal(Θ, τ 2 ) y i Normal(Θ, s i 2 +τ 2 ) model components: Data: estimates y i standard errors s i Parameters: true parameter value Θ heterogeneity τ Θ Rof primary interest ( effect ) τ R + nuisance parameter ( between-trial heterogeneity ) 1 L. V. Hedges, I. Olkin. Statistical methods for meta-analysis. Academic Press, 1985. 2 J. Hartung, G. Knapp, B. K. Sinha. Statistical meta-analysis with applications. Wiley, 2008. C. Röver et al. Meta-analysis of few small studies September 7, 2015 3 / 13

Meta analysis Frequentist approaches usual frequentist procedure: (1) derive heterogeneity estimate ˆτ (2) conditional on τ = ˆτ, derive - estimate ˆΘ - standard error ˆσ Θ 3 G. Knapp, J. Hartung. Improved tests for a random effects meta-regression with a single covariate. Statistics in Medicine 22(17):2693 2710, 2003. C. Röver et al. Meta-analysis of few small studies September 7, 2015 4 / 13

Meta analysis Frequentist approaches usual frequentist procedure: (1) derive heterogeneity estimate ˆτ (2) conditional on τ = ˆτ, derive - estimate ˆΘ - standard error ˆσ Θ confidence interval via Normal approximation: ˆΘ ± ˆσ Θ z (1 α/2) 3 G. Knapp, J. Hartung. Improved tests for a random effects meta-regression with a single covariate. Statistics in Medicine 22(17):2693 2710, 2003. C. Röver et al. Meta-analysis of few small studies September 7, 2015 4 / 13

Meta analysis Frequentist approaches usual frequentist procedure: (1) derive heterogeneity estimate ˆτ (2) conditional on τ = ˆτ, derive - estimate ˆΘ - standard error ˆσ Θ confidence interval via Normal approximation: ˆΘ ± ˆσ Θ z (1 α/2) (uncertainty in τ not accounted for) 3 G. Knapp, J. Hartung. Improved tests for a random effects meta-regression with a single covariate. Statistics in Medicine 22(17):2693 2710, 2003. C. Röver et al. Meta-analysis of few small studies September 7, 2015 4 / 13

Meta analysis Frequentist approaches usual frequentist procedure: (1) derive heterogeneity estimate ˆτ (2) conditional on τ = ˆτ, derive - estimate ˆΘ - standard error ˆσ Θ confidence interval via Normal approximation: ˆΘ ± ˆσ Θ z (1 α/2) (uncertainty in τ not accounted for) Knapp-Hartung approach 3 : compute 1 (y i ˆΘ) 2 q := k 1 s 2 i i + ˆτ 2 confidence interval via Student-t approximation: ˆΘ ± max{ q, 1} ˆσ Θ t (k 1);(1 α/2) 3 G. Knapp, J. Hartung. Improved tests for a random effects meta-regression with a single covariate. Statistics in Medicine 22(17):2693 2710, 2003. C. Röver et al. Meta-analysis of few small studies September 7, 2015 4 / 13

Meta analysis Bayesian approach Bayesian approach 4 set up model likelihood specify prior information about unknowns (Θ, τ) posterior results as prior likelihood marginal posterior p(θ y,σ) = p(θ,τ y,σ) dτ... 4 A. J. Sutton, K. R. Abrams. Bayesian methods in meta-analysis and evidence synthesis. Statistical Methods in Medical Research, 10(4):277, 2001. 5 T. C. Smith, D. J. Spiegelhalter, A. Thomas. Bayesian approaches to random-effects meta-analysis: A comparative study. Statistics in Medicine, 14(24):2685, 1995. C. Röver et al. Meta-analysis of few small studies September 7, 2015 5 / 13

Meta analysis Bayesian approach Bayesian approach 4 set up model likelihood specify prior information about unknowns (Θ, τ) posterior results as prior likelihood marginal posterior p(θ y,σ) = p(θ,τ y,σ) dτ... Comments: consideration of prior information propagation of uncertainty straightforward interpretation computationally more expensive, usually done via simulation (MCMC, BUGS) 5 special case of simple random-effects MA may be solved semi-analytically (using bmeta R package) 4 A. J. Sutton, K. R. Abrams. Bayesian methods in meta-analysis and evidence synthesis. Statistical Methods in Medical Research, 10(4):277, 2001. 5 T. C. Smith, D. J. Spiegelhalter, A. Thomas. Bayesian approaches to random-effects meta-analysis: A comparative study. Statistics in Medicine, 14(24):2685, 1995. C. Röver et al. Meta-analysis of few small studies September 7, 2015 5 / 13

Meta analysis Frequentist and Bayesian approaches estimators for τ considered in the following: DerSimonian-Laird estimator (DL) restricted ML estimator (REML) 6 Mandel-Paule estimator (MP) 7 Bayes modal estimator (BM) 8 6 K. Sidik, J.N. Jonkman. A comparison of heterogeneity variance estimators in combining results of studies. Statistics in Medicine 26(9):1964 1981, 2007. 7 A.L. Rukhin, B.J. Biggerstaff, M.G. Vangel. Restricted maximum-likelihood estimation of a common mean and the Mandel-Paule algorithm. Journal of Statistical Planning and Inference 83(2):319 330, 2000. 8 Y. Chung, S. Rabe-Hesketh, I.-H. Choi. Avoiding zero between-study variance estimates in random-effects meta-analysis. Statistics in Medicine 32(23):4071 4089, 2013. C. Röver et al. Meta-analysis of few small studies September 7, 2015 6 / 13

Meta analysis Frequentist and Bayesian approaches estimators for τ considered in the following: DerSimonian-Laird estimator (DL) restricted ML estimator (REML) 6 Mandel-Paule estimator (MP) 7 Bayes modal estimator (BM) 8 priors for τ considered in the following (where Θ = log(or)): half-normal (σ = 0.5) half-normal (σ = 1.0) Uniform (0.0, 4.0) 6 K. Sidik, J.N. Jonkman. A comparison of heterogeneity variance estimators in combining results of studies. Statistics in Medicine 26(9):1964 1981, 2007. 7 A.L. Rukhin, B.J. Biggerstaff, M.G. Vangel. Restricted maximum-likelihood estimation of a common mean and the Mandel-Paule algorithm. Journal of Statistical Planning and Inference 83(2):319 330, 2000. 8 Y. Chung, S. Rabe-Hesketh, I.-H. Choi. Avoiding zero between-study variance estimates in random-effects meta-analysis. Statistics in Medicine 32(23):4071 4089, 2013. C. Röver et al. Meta-analysis of few small studies September 7, 2015 6 / 13

Example Crins et al. (2014) data 9 Liver transplant example: steroid resistant rejection (SRR) Heffron (2003) Ganschow (2005) Gras (2008) 2.00 [ 3.78, 0.21 ] 0.45 [ 1.77, 0.88 ] 1.88 [ 4.11, 0.36 ] data: 3 estimates data: (log ORs) and standard errors 5.00 3.00 1.00 1.00 log odds ratio 9 N.D. Crins et al. Interleukin-2 receptor antagonists for pediatric liver transplant recipients: A systematic review and meta-analysis of controlled studies. Pediatric Transplantation 18(8):839 850, 2014. C. Röver et al. Meta-analysis of few small studies September 7, 2015 7 / 13

Example Crins et al. (2014) data 9 Liver transplant example: steroid resistant rejection (SRR) data: 3 estimates data: (log ORs) and standard errors Heffron (2003) Ganschow (2005) Gras (2008) Unif [0,4] (tau = 1.11) HNorm 1.0 (tau = 0.59) HNorm 0.5 (tau = 0.34) 2.00 [ 3.78, 0.21 ] 0.45 [ 1.77, 0.88 ] 1.88 [ 4.11, 0.36 ] 1.29 [ 3.70, 0.92 ] 1.24 [ 2.70, 0.13 ] 1.21 [ 2.35, 0.09 ] 5.00 3.00 1.00 1.00 log odds ratio 9 N.D. Crins et al. Interleukin-2 receptor antagonists for pediatric liver transplant recipients: A systematic review and meta-analysis of controlled studies. Pediatric Transplantation 18(8):839 850, 2014. C. Röver et al. Meta-analysis of few small studies September 7, 2015 7 / 13

Example Crins et al. (2014) data 9 Liver transplant example: steroid resistant rejection (SRR) data: 3 estimates data: (log ORs) and standard errors Heffron (2003) Ganschow (2005) Gras (2008) Unif [0,4] (tau = 1.11) HNorm 1.0 (tau = 0.59) HNorm 0.5 (tau = 0.34) DL (tau = 0.38) DL KnHa (tau = 0.38) 2.00 [ 3.78, 0.21 ] 0.45 [ 1.77, 0.88 ] 1.88 [ 4.11, 0.36 ] 1.29 [ 3.70, 0.92 ] 1.24 [ 2.70, 0.13 ] 1.21 [ 2.35, 0.09 ] 1.21 [ 2.27, 0.15 ] 1.21 [ 3.54, 1.13 ] 5.00 3.00 1.00 1.00 log odds ratio 9 N.D. Crins et al. Interleukin-2 receptor antagonists for pediatric liver transplant recipients: A systematic review and meta-analysis of controlled studies. Pediatric Transplantation 18(8):839 850, 2014. C. Röver et al. Meta-analysis of few small studies September 7, 2015 7 / 13

Example Crins et al. (2014) data 9 Liver transplant example: steroid resistant rejection (SRR) data: 3 estimates data: (log ORs) and standard errors Heffron (2003) Ganschow (2005) Gras (2008) Unif [0,4] (tau = 1.11) HNorm 1.0 (tau = 0.59) HNorm 0.5 (tau = 0.34) DL (tau = 0.38) DL KnHa (tau = 0.38) REML (tau = 0.51) REML KnHa (tau = 0.51) MP (tau = 0.33) MP KnHa (tau = 0.33) BM (tau = 0.93) BM KnHa (tau = 0.93) 2.00 [ 3.78, 0.21 ] 0.45 [ 1.77, 0.88 ] 1.88 [ 4.11, 0.36 ] 1.29 [ 3.70, 0.92 ] 1.24 [ 2.70, 0.13 ] 1.21 [ 2.35, 0.09 ] 1.21 [ 2.27, 0.15 ] 1.21 [ 3.54, 1.13 ] 1.24 [ 2.38, 0.10 ] 1.24 [ 3.75, 1.27 ] 1.20 [ 2.24, 0.16 ] 1.20 [ 3.49, 1.09 ] 1.32 [ 2.78, 0.14 ] 1.32 [ 4.53, 1.89 ] 5.00 3.00 1.00 1.00 log odds ratio 9 N.D. Crins et al. Interleukin-2 receptor antagonists for pediatric liver transplant recipients: A systematic review and meta-analysis of controlled studies. Pediatric Transplantation 18(8):839 850, 2014. C. Röver et al. Meta-analysis of few small studies September 7, 2015 7 / 13

Example Crins et al. (2014) data different analyses yield different answers k = 2 to 3 studies is a common scenario (majority of meta analyses in Cochrane Database 10 ) 10 R.M. Turner et al. Predicting the extent of heterogeneity in meta-analysis, using empirical data from the Cochrane Database of Systematic Reviews. International Journal of Epidemiology 41(3):818 827, 2012. E. Kontopantelis et al. A re-analysis of the Cochrane Library data: The dangers of unobserved heterogeneity in meta-analyses. PLoS ONE 8(7):e69930, 2013. 11 S.E. Brockwell, I.R. Gordon. A comparison of statistical methods for meta-analysis. Statistics in Medicine 20(6):825 840, 2001. Y. Chung, S. Rabe-Hesketh, I.-H. Choi. Avoiding zero between-study variance estimates in random-effects meta-analysis. Statistics in Medicine 32(23):4071 4089, 2013. C. Röver et al. Meta-analysis of few small studies September 7, 2015 8 / 13

Example Crins et al. (2014) data different analyses yield different answers k = 2 to 3 studies is a common scenario (majority of meta analyses in Cochrane Database 10 ) how does performance compare in general, especially for few studies? 10 R.M. Turner et al. Predicting the extent of heterogeneity in meta-analysis, using empirical data from the Cochrane Database of Systematic Reviews. International Journal of Epidemiology 41(3):818 827, 2012. E. Kontopantelis et al. A re-analysis of the Cochrane Library data: The dangers of unobserved heterogeneity in meta-analyses. PLoS ONE 8(7):e69930, 2013. 11 S.E. Brockwell, I.R. Gordon. A comparison of statistical methods for meta-analysis. Statistics in Medicine 20(6):825 840, 2001. Y. Chung, S. Rabe-Hesketh, I.-H. Choi. Avoiding zero between-study variance estimates in random-effects meta-analysis. Statistics in Medicine 32(23):4071 4089, 2013. C. Röver et al. Meta-analysis of few small studies September 7, 2015 8 / 13

Example Crins et al. (2014) data different analyses yield different answers k = 2 to 3 studies is a common scenario (majority of meta analyses in Cochrane Database 10 ) how does performance compare in general, especially for few studies? Simulation study 11, varying amount of heterogeneity 10 R.M. Turner et al. Predicting the extent of heterogeneity in meta-analysis, using empirical data from the Cochrane Database of Systematic Reviews. International Journal of Epidemiology 41(3):818 827, 2012. E. Kontopantelis et al. A re-analysis of the Cochrane Library data: The dangers of unobserved heterogeneity in meta-analyses. PLoS ONE 8(7):e69930, 2013. 11 S.E. Brockwell, I.R. Gordon. A comparison of statistical methods for meta-analysis. Statistics in Medicine 20(6):825 840, 2001. Y. Chung, S. Rabe-Hesketh, I.-H. Choi. Avoiding zero between-study variance estimates in random-effects meta-analysis. Statistics in Medicine 32(23):4071 4089, 2013. C. Röver et al. Meta-analysis of few small studies September 7, 2015 8 / 13

Simulation study heterogeneity estimation: bias τ 2 0 0.01 0.02 0.05 0.1 0.2 0.5 1 2 bias (heterogeneity τ) 0.2 0.0 0.2 0.4 0.6 k = 3 Unif [0,4] HNorm (1.0) HNorm (0.5) DL REML MP BM 0 0.1 0.2 0.5 1 τ C. Röver et al. Meta-analysis of few small studies September 7, 2015 9 / 13

Simulation study heterogeneity estimation: bias τ 2 0 0.01 0.02 0.05 0.1 0.2 0.5 1 2 bias (heterogeneity τ) 0.2 0.0 0.2 0.4 0.6 k = 5 Unif [0,4] HNorm (1.0) HNorm (0.5) DL REML MP BM 0 0.1 0.2 0.5 1 τ C. Röver et al. Meta-analysis of few small studies September 7, 2015 9 / 13

Simulation study heterogeneity estimation: bias τ 2 0 0.01 0.02 0.05 0.1 0.2 0.5 1 2 bias (heterogeneity τ) 0.2 0.0 0.2 0.4 0.6 k = 10 Unif [0,4] HNorm (1.0) HNorm (0.5) DL REML MP BM 0 0.1 0.2 0.5 1 τ C. Röver et al. Meta-analysis of few small studies September 7, 2015 9 / 13

Simulation study heterogeneity estimation: zero estimates τ 2 0 0.01 0.02 0.05 0.1 0.2 0.5 1 2 fraction of zero τ estimates 0.0 0.1 0.2 0.3 0.4 0.5 0.6 k = 3 DL REML MP 0 0.1 0.2 0.5 1 τ C. Röver et al. Meta-analysis of few small studies September 7, 2015 10 / 13

Simulation study heterogeneity estimation: zero estimates τ 2 0 0.01 0.02 0.05 0.1 0.2 0.5 1 2 fraction of zero τ estimates 0.0 0.1 0.2 0.3 0.4 0.5 0.6 k = 5 DL REML MP 0 0.1 0.2 0.5 1 τ C. Röver et al. Meta-analysis of few small studies September 7, 2015 10 / 13

Simulation study heterogeneity estimation: zero estimates τ 2 0 0.01 0.02 0.05 0.1 0.2 0.5 1 2 fraction of zero τ estimates 0.0 0.1 0.2 0.3 0.4 0.5 0.6 k = 10 DL REML MP 0 0.1 0.2 0.5 1 τ C. Röver et al. Meta-analysis of few small studies September 7, 2015 10 / 13

Simulation study effect estimation: 95% CI coverage τ 2 0 0.01 0.02 0.05 0.1 0.2 0.5 1 2 coverage (effect Θ, 95% interval) 0.80 0.85 0.90 0.95 1.00 Unif [0,4] HNorm (1.0) HNorm (0.5) DL normal DL KnHa REML normal REML KnHa MP normal MP KnHa BM normal BM KnHa k = 3 0 0.1 0.2 0.5 1 τ C. Röver et al. Meta-analysis of few small studies September 7, 2015 11 / 13

Simulation study effect estimation: 95% CI coverage τ 2 0 0.01 0.02 0.05 0.1 0.2 0.5 1 2 coverage (effect Θ, 95% interval) 0.80 0.85 0.90 0.95 1.00 Unif [0,4] HNorm (1.0) HNorm (0.5) DL normal DL KnHa REML normal REML KnHa MP normal MP KnHa BM normal BM KnHa k = 5 0 0.1 0.2 0.5 1 τ C. Röver et al. Meta-analysis of few small studies September 7, 2015 11 / 13

Simulation study effect estimation: 95% CI coverage τ 2 0 0.01 0.02 0.05 0.1 0.2 0.5 1 2 coverage (effect Θ, 95% interval) 0.80 0.85 0.90 0.95 1.00 Unif [0,4] HNorm (1.0) HNorm (0.5) DL normal DL KnHa REML normal REML KnHa MP normal MP KnHa BM normal BM KnHa k = 10 0 0.1 0.2 0.5 1 τ C. Röver et al. Meta-analysis of few small studies September 7, 2015 11 / 13

Simulation study effect estimation: 95% CI length τ 2 0 0.01 0.02 0.05 0.1 0.2 0.5 1 2 mean 95% interval length (effect Θ) 0 1 2 3 4 5 6 Unif [0,4] HNorm (1.0) HNorm (0.5) DL normal DL KnHa REML normal REML KnHa MP normal MP KnHa BM normal BM KnHa k = 3 0 0.1 0.2 0.5 1 τ C. Röver et al. Meta-analysis of few small studies September 7, 2015 12 / 13

Simulation study effect estimation: 95% CI length τ 2 0 0.01 0.02 0.05 0.1 0.2 0.5 1 2 mean 95% interval length (effect Θ) 0 1 2 3 4 5 6 Unif [0,4] HNorm (1.0) HNorm (0.5) DL normal DL KnHa REML normal REML KnHa MP normal MP KnHa BM normal BM KnHa k = 5 0 0.1 0.2 0.5 1 τ C. Röver et al. Meta-analysis of few small studies September 7, 2015 12 / 13

Simulation study effect estimation: 95% CI length τ 2 0 0.01 0.02 0.05 0.1 0.2 0.5 1 2 mean 95% interval length (effect Θ) 0 1 2 3 4 5 6 Unif [0,4] HNorm (1.0) HNorm (0.5) DL normal DL KnHa REML normal REML KnHa MP normal MP KnHa BM normal BM KnHa k = 10 0 0.1 0.2 0.5 1 τ C. Röver et al. Meta-analysis of few small studies September 7, 2015 12 / 13

Conclusions small differences between frequentist methods (on average) differences most pronounced in (common!) case of few studies consideration of estimation uncertainty: application of Knapp-Hartung adjustment crucial for nominal level surprisingly many zero τ estimates Bayesian methods behave as expected: conservative / anticonservative for small / large τ ( Mean coverage (calibration) accurate by construction) Bayesian methods allow to utilize external information (effect and heterogeneity, e.g. 12 ) investigating properties w.r.t. predictive distributions (θ k+1 ) bmeta R package to appear on CRAN soon 12 R.M. Turner et al. Predictive distributions for between-study heterogeneity and simple methods for their application in Bayesian meta-analysis. Statistics in Medicine 34(6):984 998, 2015. C. Röver et al. Meta-analysis of few small studies September 7, 2015 13 / 13

+++ additional slides +++ C. Röver et al. Meta-analysis of few small studies September 7, 2015 13 / 13

Simulation study Setup number of studies: k {3, 5, 10} heterogeneity: τ 2 {0.00, 0.01, 0.02, 0.05, 0.1, 0.2, 0.5, 1.0, 2.0} (I 2 {0.00, 0.06, 0.11, 0.23, 0.37, 0.54, 0.75, 0.85, 0.92}) standard errors s i : truncated χ 2 -distribution 13 10 000 repetitions for each combination (k, τ 2 ) compute Bayesian MAs (3 different priors) compute frequentist MAs (different τ estimators, Normal and Knapp-Hartung approximation) 13 S.E. Brockwell, I.R. Gordon. A comparison of statistical methods for meta-analysis. Statistics in Medicine 20(6):825 840, 2001. Y. Chung, S. Rabe-Hesketh, I.-H. Choi. Avoiding zero between-study variance estimates in random-effects meta-analysis. Statistics in Medicine 32(23):4071 4089, 2013. C. Röver et al. Meta-analysis of few small studies September 7, 2015 13 / 13

Implementation bmeta R package under development > cochran01 <- bmeta(cochran1954[,"mean"], sqrt(cochran1954[,"se2"])) > cochran02 <- bmeta(cochran1954[,"mean"], sqrt(cochran1954[,"se2"]), + mu.prior.mean=150, mu.prior.sd=100, + tau.prior=function(x){return(dexp(x, rate=0.05))}) > > cochran01$summary tau mu mu.pred mode 10.303255 156.504954 154.16345 median 12.888735 157.896520 157.33321 mean 14.844457 158.547999 158.54800 sd 9.950631 8.358115 19.70028 95% lower 0.000000 143.180913 119.77459 95% upper 32.665117 176.106158 200.12309 > > # compute posterior quantiles: > cochran01$qposterior(mu.p=c(0.005, 0.995)) [1] 135.0429 187.3122 > > # plot posterior density: > x <- seq(from=130, to=190, length=100) > plot(x, cochran02$dposterior(mu=x), type="l") > lines(x, cochran01$dposterior(mu=x)) C. Röver et al. Meta-analysis of few small studies September 7, 2015 13 / 13

2024 © healthdocbox.com Privacy Policy | Terms of Service | Feedback