State of the Art in Clinical & Anatomic Pathology. Meta-analysis of Clinical Studies of Diagnostic Tests

Size: px
Start display at page:

Download "State of the Art in Clinical & Anatomic Pathology. Meta-analysis of Clinical Studies of Diagnostic Tests"

Transcription

1 State of the Art in Clinical & Anatomic Pathology Meta-analysis of Clinical Studies of Diagnostic Tests Developments in How the Receiver Operating Characteristic Works N Meta-analytic summaries are needed on clinical studies of diagnostic tests. Meta-analyses on clinical studies of diagnostic tests commonly use the receiver operating characteristic method, which differs conceptually and computationally from the more widely known metaanalytic methods applicable in other contexts, such as in studies of randomized controlled trials. Important conceptual differences for clinical studies of diagnostic tests versus randomized controlled trials are that the study subpopulations are not defined by random allocation and the test threshold typically varies across studies to accommodate rule in versus rule out testing strategies. The receiver operating characteristic method has evolved substantially in the past decade, and the most recent approaches use multilevel regression methods that require iterative computational solutions to estimate the influence of the study-level variables. Using current methodology, a meta-analysis on clinical studies of diagnostic tests can address questions relevant to the clinical application of a diagnostic test that cannot be answered at the level of the individual study. (Arch Pathol Lab Med. 2011;135: ; doi: /arpa SO) Meta-analysis is a statistical technique used to derive an aggregate result from a series of studies that address a particular research question. 1 5 The most common type of meta-analysis in the medical literature is the kind applied to randomized controlled trials (RCTs). However, when the research question is the evaluation and derivation of an aggregate result from a series of clinical studies on a diagnostic test (CSDT), the method of choice is the receiver operating characteristic (ROC) curve. 1 5 Understanding how the ROC method differs from other methods prevalent in the literature and commonly applied to the meta-analysis of RCTs is not intuitive. Several aspects of the meta-analysis methodology applied to CSDTs contrast with those applied to RCTs. Accepted for publication April 13, From the Rural Health Academic Centre, Melbourne Medical School, University of Melbourne, Ballarat, Victoria, Australia. The author has no relevant financial interest in the products or companies described in this article. Reprints: James Hurley, MBBS, MEpidemiol, PhD, FRACP, Rural Health Academic Centre, Melbourne Medical School, University of Melbourne, Drummond St N, Ballarat, Victoria, Australia 3350 ( jamesh@bhs.org.au). James Hurley, MBBS, MEpidemiol, PhD, FRACP Commonly, an RCT will estimate the effect of active versus placebo treatments on an endpoint, such as patient survival. Study endpoints such as survival are dichotomized into survival versus nonsurvival and expressed as a binary outcome. As a binary outcome, the effect size of the RCT can in turn be summarized as an odds ratio (OR), that is, the ratio of odds of survival for patients randomized to receive versus not receive the active treatment. An OR has several useful mathematical and statistical properties but, most particularly, in a meta-analysis as the OR is considered the optimal combinable statistic for deriving measures of both the overall summary effect together with its associated heterogeneity across the studies. A meta-analysis of a series of CSDTs is superficially similar to the meta-analysis of a series of RCTs in that the combinable measure of the study effect across a series of CSDTs is the diagnostic odds ratio (DOR), which is an homologous measure to the OR of the RCT. In this case, the DOR is the ratio of the odds for a positive test in patients with disease (D + ) versus patients without the disease of interest (D 2 ). 1 4 However, there are several key conceptual differences in the design of CSDTs versus that of RCTs (Table 1). These differences affect the respective meta-analytic methods applicable in these contexts 1 4 and versus other contexts For an RCT, the membership of the 2 subpopulations (those receiving active versus placebo treatments) is determined by random allocation. As a result of the random allocation, the 2 subpopulations of an RCT can be presumed to be similar in all respects, except for receipt of the active versus placebo treatments. This study design feature of the RCT optimizes the estimation of the counterfactual effect of exposure to the study therapy. 6 By contrast, for CSDT, the 2 populations (D 2 versus D + ) are constituent within the study population and membership is usually defined by reference to a gold standard test. The baseline parameters of the D 2 and D + subpopulations can be expected to differ from each other in many additional respects, and the CSDT is not an optimal study design for estimating the counterfactual effect of the presence versus the absence of disease. The most obvious difference between the D 2 and D + subpopulations is usually their unequal relative size, with the D + subpopulation usually much smaller than the D 2 subpopulation (as indicated in Figure 1). 2. The result of a CSDT is often presented as the test s sensitivity and specificity, which are more widely Arch Pathol Lab Med Vol 135, December 2011 How the ROC Works Hurley 1585

2 Table 1. Conceptual Counterparts: Randomized Controlled Trials (RCTs) Versus Clinical Studies on a Diagnostic Test (CSDTs) Characteristic RCTs CSDTs Study subpopulations Names of subpopulations C and I D + and D 2 Relative sizes of subpopulations C 5 I Usually, D +, D 2 Formation of subpopulations By random assignment By reference to a gold standard Presumption of equivalence of baseline Tenable if assignment is truly random Usually untenable parameters Study endpoint Usually predefined Test threshold variably defined for rule-in versus rule-out testing strategies Descriptive statistics Event rates: I group Sensitivity, test-positive proportion among D + subpopulation Event rates: C group Specificity, test-positive proportion among D 2 subpopulation Summary effect measures a OR DOR Two-way plot of event rates in the 2 subpopulations L Abbé plot ROC plot Abbreviations: C, control; D +, subpopulation with disease present; D 2, subpopulation with disease absent; DOR, diagnostic odds ratio; I, intervention; OR, odds ratio; ROC, receiver operating characteristic. a For an RCT, the risk ratio and risk difference are additional, combinable, summary effect measures. For a CSDT, usually only the DOR is used in deriving summary effect measure. understood terms than is the DOR. Among the limitations of the DOR is that its interpretation is not intuitive to prospective users of a diagnostic test, in contrast to the common use of the OR to describe the result of an RCT. 7 Other terms used to describe the effect size of an RCT, such as the risk ratio and risk difference, may serve as combinable summary measures in the meta-analysis of RCTs. Another useful descriptive statistic for end users of an RCT meta-analysis is the number needed to treat, derived from the reciprocal of the risk difference. These descriptive statistics for an RCT have no counterpart in common use as descriptors for a CSDT. 3. For a diagnostic test, the dichotomy into testpositive and test-negative results is based on the test threshold (as indicated in Figure 1), which may vary substantially from study to study to accommodate rule in versus rule out testing strategies. The test sensitivity and specificity will be negatively correlated as the test threshold varies across studies (as indicated by the horizontal left to right arrow in Figure 1). This negative correlation between test sensitivity and specificity, resulting from various test thresholds applied across different studies, makes their use in deriving summary measures across series of CSDTs problematic. For these reasons, heterogeneity across the results of a series of CSDTs would be expected to be greater than would typically be the case for a series of RCTs. Hence, the aggregation of CSDT study results to achieve greater precision is usually neither a realistic nor a desirable Figure 1. Subpopulations with disease absent or disease present within a clinical study of a diagnostic test. The test threshold is set higher (as indicated by the arrow) for a rule-in versus a rule-out testing strategy. Abbreviations: FN, false-negative result; FP, falsepositive result; TN, true-negative result; TP, true-positive result Arch Pathol Lab Med Vol 135, December 2011 How the ROC Works Hurley

3 objective, whereas these are often realistic and desirable objectives for the meta-analysis of RCTs. Moreover, the meta-analytic methods used to derive a summary measure from several RCTs are not appropriate for deriving a summary measure from several CSDTs, except in the unusual circumstance in which there is minimal heterogeneity in the study results. ORIGIN OF THE ROC The ROC method has several nonmedical applications, 8 and its origin predates the development of metaanalyses for RCTs by 2 decades. The history of the ROC curve has been eloquently described previously in the ARCHIVES. 3,4 It originated during World War II to evaluate the ability of radar operators (receiver operators) to correctly identify incoming aircraft. Its subsequent application to the evaluation of diagnostic radiologic tests was a natural development for which an extensive literature exists. 9 Of note, in the radiologic context, no gold standard reference exists for the presence or absence of an abnormality on an x-ray image, and the ROC method is used to evaluate degrees of agreement along an ordinal scale among a panel of radiologists on whether an abnormality is present on an x-ray image. 9 This is the simplest type of ROC curve, which, because the scale is ordinal, results in a curve that is stepped, not smooth. The area under the curve (AUC) is a summary measure of the summary receiver operating characteristic (SROC) curve in long-standing use. AUCs of 1.0 and 0.5 indicate perfect proficiency versus proficiency no better than random, respectively. The derivation of the area under a stepped ROC curve requires no sophisticated software and can be done with a pocket calculator. As in the radiologic context, there are applications of the ROC methodology within clinical and anatomic pathology for which the gold standard is clinical opinion measured on an ordinal scale. For example, a panel of renal pathologists were evaluated on their interpretations of renal biopsies and the sensitivity and specificity of their predictions for clinical outcomes in patients with lupus with diffuse proliferative glomerulonephritis. 10 Other examples of a single study of a test evaluated with ROC methodologies at several test thresholds have been previously described. 4 Usually, in clinical pathology applications, and in contrast to the case described above for a radiologic application, the scale of a clinical diagnostic assay is continuous, and there may be a gold standard test available as the comparator; therefore, the derivation of the AUC is not simple, and more complex methods, such as a trapezoidal integration method, are required. An alternative summary measure is the index Q*, which is the point on the SROC curve at which sensitivity equals specificity. The main limitation with the use of the AUC and the index Q* as summary measures for the ROC method is that AUC and the index Q* do not indicate symmetry in the ROC curve, the importance of which is described below. Also, the Q* point may not lie within the range of the sensitivity and specificity of the studies included in the analysis. ADAPTATION OF THE ROC TO CLINICAL PATHOLOGY A meta-analysis of a panel of CSDTs evaluates the same diagnostic test used in several studies with different break points and different study populations. A graphic summary is the true-positive rate (TPR; ie, sensitivity) plotted against the false-positive rate (FPR; Figure 2. An example of 10 studies on the diagnostic test in Table 3 (the first five studies are numbered), together with the fitted summary receiver operating characteristic (SROC) curve, the summary diagnostic odds ratio (solid square), the associated 95% confidence ellipse (inner ellipse), and the prediction ellipse (outer ellipse) derived by the hierarchical SROC method. Study size is proportional to symbol size. The diagonal line indicates an SROC curve for sensitivity specificity, corresponding to noninformative tests. Increases in test threshold, corresponding to studies adopting more a rule-in versus ruleout testing strategy, would correspond to movement along the SROC curve as indicated by the arrow. ie, 1 2 specificity), with each marker in the plot corresponding to a single study. Usually the relative size of each study is represented. Methods described below and downloadable software 11 are now available to interpolate by regression an SROC curve that is now smooth, rather than stepped (Figure 2). Again, there is some similarity between meta-analytic techniques applied to RCTs and CSDTs for event rate plots. A plot of event rates in the 2 subpopulations of a series of RCTs is called a L Abbé plot, which has the control and intervention groups represented along the x-axis and y- axis, respectively. This is the counterpart to the ROC plot generated from a series of CSDTs. For the ROC plot, the rates of positive test results for the D 2 and D + subpopulations are represented along the x-axis and y-axis, respectively. For the L Abbé plot of event rates for the subpopulations of RCTs, the interpolation within the plot will differ in appearance depending on whether the risk difference, risk ratio, or OR has been used in deriving the summary effect measure for the RCTs, whereas for the ROC plot for CSDTs, the DOR is always used, and hence, the interpolated SROC within an ROC plot is always curved. There are other graphic summaries besides the SROC method that can be used to illustrate other summary aspects of a series of CSDTs. 12 A limitation of the SROC plot is that the study-specific 95% confidence intervals are not represented for reasons of clarity in the SROC figure. As a consequence, and unlike the forest plots of an RCT meta-analysis, in an SROC plot, it is not possible to discern the amount of dispersion in the study due to random error versus the amount due to heterogeneity. Arch Pathol Lab Med Vol 135, December 2011 How the ROC Works Hurley 1587

4 Table 2. Steps in the Derivation of a Summary Receiver Operating Characteristic (SROC) Curve Steps Interpretation TPR 5 sensitivity 5 TP/(TP + FN) FPR 5 (1 2 specificity) 5 FP/(FP + TN) Generate ROC plot X-axis FPR 5 (1 2 specificity) Y-axis Sensitivity Derivation of SROC curve 1. Reexpress proportions as odds Logit TPR 5 ln(tp/fn) 2. Transform odds to logits Logit FPR 5 ln(fp/tn) (Steps 1 and 2 are equivalent to a logistic transformation) 3. Calculate D between logits D 5 logit(tpr) 2 logit(fpr); D 5 ln(dor) 4. Calculate S of logits S 5 logit(tpr) + logit(fpr) 5. Linear regression of D on S across all studies D 5 a + bs 6. Back-transform to proportions and plot curve in ROC space Abbreviations: a, estimate of the intercept; b, estimate of the slope; D, difference of the logits; DOR, diagnostic odds ratio; FN, false-negative result; FP, false-positive result; FPR, false-positive rate; ROC, receiver operating characteristic; S, sum of the logits; TN, true-negative result; TP, truepositive result; TPR, true-positive rate. Regression techniques have emerged in the past 20 years to facilitate the derivation of an SROC plot by regression. 2,13 19 There are 6 steps (Table 2) to the derivation of an SROC curve by regression. The first step is to reexpress the TPR proportion or sensitivity, which is TP/(TP + FN), as the odds of a positive test result in the D + population (ie, TP/FN) and the FPR (FPR specificity) proportion, which is TN/(FP + TN) as the odds of a positive test result in the D 2 population (ie, FP/TN), where TP is true-positive, FN is false-negative, TN is true-negative, and FP is false-positive. The second step is to derive logarithms of each of those odds (adding 0.5 to zero cells to avoid indeterminate results in the transformation). These first 2 steps are equivalent to a logit transformation of the TPR and FPR, which is a standard transformation for working with proportions. 20 The third step is to calculate the difference between the 2 logits. Note that because the difference between 2 logarithms is equivalent to the logarithm of a ratio, the difference between the logit(tpr) and the logit(fpr) derived in step 3 is equivalent to the log of the DOR. The fourth step is the calculation of the sum (S) of the logits. The fifth step is a linear regression of the difference (D), as the dependent variable, versus the S of these 2 logits, as the independent variable, using standard linear regression methods: D 5 a + bs, where a is the estimate of the intercept, and b is the estimate of the slope. As the final step, these logit-transformed values and the regression line are back-transformed to proportions. The resulting back-transformed SROC regression will now appear curved, not linear, in the original ROC space. The SROC curve so derived describes the expected relationship of sensitivity and specificity as it varies with test threshold across the range of studies. As the test threshold increases (indicated by the left to right arrow in Figure 1), the truepositive proportion and the false-positive proportion would be expected to decrease across a typical ROC plot (indicated by the right to left dotted arrow in Figure 2). The regression methods in the derivation of the SROC curve (step 5) facilitate additional statistical tests as part of the meta-analysis. The first test is a test for symmetry in the SROC curve, which is indicated by a near-zero slope (b) of the linear regression between D and S. A near-zero slope indicates that the estimate for D remains close to the value for the intercept (a) throughout the range of the regression. 13,14 The second test is the test for that intercept (a) value in the linear regression. Symmetry indicates that the DOR is equivalent to the intercept (a) value throughout the range of the regression. This finding implies that the logarithm of the DOR is constant through different test thresholds across different studies. This is an important finding. RECENT DEVELOPMENTS IN ROC METHODOLOGY In the past 10 years, bivariate 13 and hierarchic 14 methods for deriving the SROC curve have emerged, and the supporting software is downloadable. 15,16 These methods, which give near identical results to each other in the metaanalyses of diagnostic tests, involve the analysis of statistical distributions at multiple levels At the lower level, they model the cell counts in the tables derived from each of the individual studies by using binomial or logit transformations of proportions. The 2 methods differ at the upper level of the model. Although these regression methods require complex computation routines and the summary measures are derived by iteration, the methods are well within the capability of modern personal computers with readily available statistical software and downloads. 15,16,19 The advantage of these newer methods is that they can generate bivariate (ie, separate) summations of both sensitivity and specificity across the panel of included studies, in contrast to the univariate summaries (ie, the AUC or the Q* value) produced from earlier methods. Being bivariate, the imprecision in the summary estimates is now represented as 95% confidence and prediction regions, which on an ROC plot are each presented as an area (ie, 2 dimensional) instead of a 95% confidence and prediction intervals (ie, 1 dimensional). As a consequence, the appearance of the SROC plot as a graphic summary in meta-analyses of CSDTs is strikingly different than the forest plots used as graphic summaries in meta-analyses of RCTs. Together with the SROC curve, these confidence and prediction regions provide a better description of the uncertainty in the relationship between sensitivity and specificity across 1588 Arch Pathol Lab Med Vol 135, December 2011 How the ROC Works Hurley

5 Study No. Table 3. Example Derivation of a Summary Receiver Operating Characteristic (SROC) Curve for 10 Studies of a Diagnostic Test a No. of Patients D + D 2 D + = g TP FN FP TN (TP + FN) the range of studies in the analysis. Moreover, with these newer regression methods, the effect of different test versions and study populations can be modeled as study level covariates. This enables comparisons of the application of a diagnostic test to different target patient populations, such as adult or pediatric populations, or to newer versus older versions of a test, for example. Recent work has given attention to the optimal conditions for obtaining reliable estimates in the SROC plot. 21 An SROC curve may fail because the model is inappropriate because of the absence of a threshold effect, for example, or the absence of heterogeneity among the TPR or FPR of the studies. An SROC curve may also fail because of insufficient data available to estimate the model parameters. The number of studies required to establish reliable estimates is dependent on the number of individual patients contributing to the TPR and FPR estimates in each study. The sample size requirements for estimating variances and correlations are more exacting than they are for estimating means. A sample size of 30 is required to be 95% certain that a standard deviation of a normal distribution will be within 25% of its true value. 22 Chappell et al 21 provide estimates of the sample size requirements for estimating correlations accurately. Other more recent developments have explored alternative transformations to the logistic transformation of sensitivity and specificity within the bivariate random effects models used to estimate the SROC curve, 23 such as the probit and complementary log-log transformations, which have a long history of application in bioassay studies. 20 These transformations are also used as alternative link functions within generalized linear mixed models. 24 AN EXAMPLE OF META-ANALYSIS TECHNIQUES FOR A DIAGNOSTIC TEST An evaluation of the limulus assay for endotoxin (a cellwall component of gram-negative bacteria) as a test in patients with suspected gram-negative sepsis provides an illustrative example of the application of meta-analysis techniques to the evaluation of a diagnostic test Gramnegative bacteremia was used as the gold standard in all D 2 = g (FP + TN) (TP + FN + FP + TN) = g (TP + FN + FP + TN) Descriptive and Summary Statistics Sensitivity = TP/(TP + FN), % Specificity = TN/(FP + TN), % DOR = (TP/FN) /(FP/TN) b Total c 82 c 4.96 c Abbreviations: D +, subpopulation with disease present by reference to gold standard; D 2, subpopulation with disease absent by reference to gold standard; DOR, diagnostic odds ratio; FN, false-negative result; FP, false-positive result; TN, true-negative result; TP, true-positive result. a The 10 studies are plotted in Figure 2 for study numbers 1 5. b Calculated after the addition of 0.5 to all zero cells. c Because this value is derived from the column totals for TP, FN, FP, and TN, it is incorrect as a summary (see text). the studies. However, the clinical value of the assay had been unresolved despite more than 50 studies being published during a period of 40 years A meta-analysis of those studies was done to compare newer versus older versions of the limulus assay, which had not been done in any of the individual studies. An additional objective of the meta-analysis summary was to determine whether studies that gave seemingly contradictory results, as reflected in their study titles, were reconcilable with the overall experience reported in the literature or explainable by variations in test threshold. Table 3 and Figure 2 summarize studies taken from this meta-analysis using only 10 of the 58 studies for simplicity and demonstration purposes. 27 Note that the size of the largest study in the table is 23 times that of the smallest study. It would be inaccurate to simply use the TP, FN, FP, and TN column totals of Table 3 to derive summary estimates of DOR, sensitivity, and specificity, as indicated in the table. If that had been done, then the summary estimates of DOR, sensitivity, and specificity would be biased by the 3 largest studies, simply because of the disparity in study sizes. The hierarchic SROC method uses a more-uniform weighting between larger and smaller studies in deriving the summary estimates. Note that, in Table 3, the D + population accounts for between 10% and 50% of each study population. With differences as large as this, the influence of spectrum bias on the results might need to be considered. 31,32 In this case, it might be appropriate to repeat the analysis restricted to those studies for which the proportion of patients with disease (D + ) was in the range thought to be clinically relevant. A typical, curved SROC regression line is derived from the 10 example studies in Table 3 through the regression method described above (Figure 2). There is no evidence of asymmetry in the SROC curve in that the intercept obtained in the regression of D on S (step 5, Derivation of SROC Curve, Table 2) does not significantly differ from 0, P The summary DOR (95% confidence interval) in this example is 6.2 (1.7 23), a level which is below that generally considered to be clinically useful. The bivariate method also generates summary measures for sensitivity Arch Pathol Lab Med Vol 135, December 2011 How the ROC Works Hurley 1589

6 and specificity, which in this example are 71% (43% 89%) and 71% (48% 87%), respectively. Note that these summary and descriptive measures differ versus those derived (erroneously) from the column totals of Table 2. Beyond these summary measures, however, 6 studies with estimates of sensitivity as disparate as 35% and 82% are inside the prediction region, and 4 small studies with greater than 95% sensitivity are outside that region. The latter finding could represent publication bias because small studies that produce sensitivity results that were presumed by the authors (and journal editors) to be unexpectedly low may be less likely to be published than larger studies with the same level of sensitivity. 33,34 The degree of precision in the modeling is limited in this example with only 10 studies, of which 3 have 0 cells, and hence, the confidence region and prediction region are wide. In the published meta-analyses with 58 studies from which this example of 10 studies is drawn, 26,27 it was possible to model study-level factors, such as study size and type of patient population (ie, adult versus pediatric). Surprisingly, in the analysis of all 58 studies, the mean DOR was inferior for the newer limulus assay versus the older gelation version. None of these questions had been adequately addressed within any one of the single studies. The application of the limulus assay among 46 studies for which details of the type of gram-negative bacteria were available was further studied using meta-analytic techniques. 35 Here, the question was whether the type of gram-negative bacteremia detected, either an Enterobacteriaceae type or not, was an important factor in the ability of the assay to detect endotoxemia, a question that no single study had been large enough to address. All 46 studies used the limulus assay method, but the sensitivity limits of that method differed by up to a 1000-fold. In this series of CSDT, the gold standard was the type of gramnegative bacteremia, either Enterobacteriaceae or not. In this example, there was no significant heterogeneity in the OR across the 46 studies. For this reason, meta-analytic tests like those applied to the evaluation of RCTs were used in this instance, and there was no additional information to be gained from the use of SROC regression methods. CONCLUSION Recently developed, multilevel regression methods for the meta-analysis of CSDTs can be applied to research questions that could not be readily answered at the level of the single study. These questions include whether there is an optimal test threshold, which patient population is most suited to testing, and how do newer versions compare with existing versions of clinical diagnostic tests? Important conceptual and computational differences apply to different methodologies for meta-analysis of RCTs. References 1. Deeks JJ. Systematic reviews of evaluations of diagnostic and screening tests. In: Egger M, Smith GD, Altman DG, eds. Systematic Reviews in Health Care: Meta-analysis in Context. 2nd ed. London, England: BMJ Books; 2001: Moses LE, Shapiro D, Littenberg B. Combining independent studies of a diagnostic test into a summary ROC curve: data-analytic approaches and some additional considerations. Stat Med. 1993;12(14): Vamvakas EC. Meta-analysis of studies of the diagnostic accuracy of laboratory tests: a review of the concepts and methods. Arch Pathol Lab Med. 1998;122(8): Beck JR, Shultz EK. The use of relative operating characteristic (ROC) curves in test performance evaluation. Arch Pathol Lab Med. 1986;110(1): Greenland S, O Rourke K: Meta-analysis. In: Rothman KJ, Greenland S, Lash TL, eds. Modern Epidemiology. 3rd ed. Philadelphia, PA: Lippincott, Williams & Wilkins; 2008: Hurley JC. Bob Hope, pneumonia and the counterfactual. Chest. 2010; 138(2): Pepe MS, Janes H, Longton G, Leisenring W, Newcomb P. Limitations of the odds ratio in gauging the performance of a diagnostic, prognostic, or screening marker. Am J Epidemiol. 2004;159(9): Swets JA. Measuring the accuracy of diagnostic systems. Science. 1988; 240(4857): Obuchowski NA. Receiver operating characteristic curves and their use in radiology. Radiology. 2003;229(1): Schwartz MM, Bernstein J, Hill GS, Holley K, Phillips EA; Lupus Nephritis Collaborative Study Group. Predictive value of renal pathology in diffuse proliferative lupus glomerulonephritis. Kidney Int. 1989;36(5): Zamora J, Abraira V, Muriel A, Khan K, Coomarasamy A. Meta-DiSc: a software for meta-analysis of test accuracy data. BMC Med Res Methodol. 2006; 6: Whiting PF, Sterne JAC, Westwood ME, et al. Graphical presentation of diagnostic information. BMC Med Res Methodol. 2008;8: Reitsma JB, Glas AS, Rutjes AW, Scholten RJ, Bossuyt PM, Zwinderman AH. Bivariate analysis of sensitivity and specificity produces informative summary measures in diagnostic reviews. J Clin Epidemiol. 2005;58(10): Rutter CM, Gatsonis CA. A hierarchical regression approach to metaanalysis of diagnostic test accuracy evaluations. Stat Med. 2001;20(19): Wallace BC, Schmid CH, Lau J, Trikalinos TA. Meta-Analyst: a software for meta-analysis of binary, continuous and diagnostic data. BMC Med Res Methodol. 2009;9: Harbord R, Whiting P. Metandi: meta-analysis of diagnostic accuracy using hierarchical logistic regression. Stata J. 2009;9(2): Harbord RM, Deeks JJ, Egger M, Whiting P, Sterne JA. A unification of models for meta-analysis of diagnostic accuracy studies. Biostatistics. 2007;8(2): Irwig LI, Tosteson ANA, Gatsonis C, et al. Guidelines for meta-analyses evaluating diagnostic tests. Ann Intern Med. 1994;120 (8): Harbord RM, Whiting P. Metandi: meta-analysis of diagnostic accuracy using hierarchical logistic regression. In: Sterne JAC, ed. Meta-Analysis in Stata: An Updated Collection From the Stata Journal. College Station, TX: Stata Press; 2009: Agresti A. Categorical Data Analysis. 2nd ed. Hoboken, NJ: Wiley- Interscience; Chappell FM, Raab GM, Wardlaw JM. When are summary ROC curves appropriate for diagnostic meta-analyses? Stat Med. 2009;28(21): Greenwood JA, Sandomire MM. Sample size required for estimating the standard deviation as a percent of its true value. J Am Stat Assoc. 1950;45(2): Chu H, Nie L, Cole SR, Poole C. Meta-analysis of diagnostic accuracy studies accounting for disease prevalence: alternative parameterizations and model selection. Stat Med. 2009;28(18): Chu H, Guo H, Zhou Y. Bivariate random effects meta-analysis of diagnostic test studies using generalized linear mixed models. Med Decis Making. 2010;30(4): Hurley JC. Endotoxemia: methods of detection and clinical relevance. Clin Microbiol Rev. 1995;8(2): Hurley JC. Concordance of endotoxemia with gram-negative bacteremia: a meta-analysis using receiver operating characteristic curves. Arch Pathol Lab Med. 2000;124(8): Hurley JC. Does gram negative bacteremia occur without endotoxemia?: a meta-analysis using hierarchical summary ROC curves. Eur J Clin Microbiol Infect Dis. 2010;29(2): Elin RJ, Robinson RA, Levine AS, Wolff SM. Lack of clinical usefulness of the limulus test in the diagnosis of endotoxemia. N Engl J Med. 1975;293(11): Levin J, Poore TE, Zauber NP, Oser RS. Detection of endotoxin in the blood of patients with sepsis due to gram-negative bacteria. N Engl J Med. 1970; 283(24): Stumacher RJ, Kovnat MJ, McCabe WR. Limitations of the usefulness of the limulus assay for endotoxin. N Engl J Med. 1973;288(24): Mulherin SA, Miller WC. Spectrum bias or spectrum effect?: subgroup variation in diagnostic test evaluation. Ann Intern Med. 2002;137(7): Lachs MS, Nachamkon I, Edelstein PH, Goldman J, Feinstein AR, Schwartz JS. Spectrum bias in the evaluation of diagnostic tests: lessons from the rapid dipstick test for urinary tract infection. Ann Intern Med. 1992;117(2): Deeks JJ, Macaskill P, Irwig L. The performance of tests of publication bias and other sample size effects in systematic reviews of diagnostic test accuracy was assessed. J Clin Epidemiol. 2005;58(9): Rutjes AWS, Reitsma JB, Di Nisio M, et al. Evidence of bias and variation in diagnostic accuracy studies. CMAJ. 2006;174(4): Hurley JC. Diagnosis of endotoxemia with gram-negative bacteremia is bacterial species dependent: a meta-analysis of clinical studies. J Clin Microbiol. 2009;47(12): Arch Pathol Lab Med Vol 135, December 2011 How the ROC Works Hurley

Introduction to diagnostic accuracy meta-analysis. Yemisi Takwoingi October 2015

Introduction to diagnostic accuracy meta-analysis. Yemisi Takwoingi October 2015 Introduction to diagnostic accuracy meta-analysis Yemisi Takwoingi October 2015 Learning objectives To appreciate the concept underlying DTA meta-analytic approaches To know the Moses-Littenberg SROC method

More information

Meta-analyses evaluating diagnostic test accuracy

Meta-analyses evaluating diagnostic test accuracy THE STATISTICIAN S PAGE Summary Receiver Operating Characteristic Curve Analysis Techniques in the Evaluation of Diagnostic Tests Catherine M. Jones, MBBS, BSc(Stat), and Thanos Athanasiou, MD, PhD, FETCS

More information

Introduction to Meta-analysis of Accuracy Data

Introduction to Meta-analysis of Accuracy Data Introduction to Meta-analysis of Accuracy Data Hans Reitsma MD, PhD Dept. of Clinical Epidemiology, Biostatistics & Bioinformatics Academic Medical Center - Amsterdam Continental European Support Unit

More information

Systematic Reviews and meta-analyses of Diagnostic Test Accuracy. Mariska Leeflang

Systematic Reviews and meta-analyses of Diagnostic Test Accuracy. Mariska Leeflang Systematic Reviews and meta-analyses of Diagnostic Test Accuracy Mariska Leeflang m.m.leeflang@amc.uva.nl This presentation 1. Introduction: accuracy? 2. QUADAS-2 exercise 3. Meta-analysis of diagnostic

More information

Meta-analysis of diagnostic test accuracy studies with multiple & missing thresholds

Meta-analysis of diagnostic test accuracy studies with multiple & missing thresholds Meta-analysis of diagnostic test accuracy studies with multiple & missing thresholds Richard D. Riley School of Health and Population Sciences, & School of Mathematics, University of Birmingham Collaborators:

More information

Supplementary Online Content

Supplementary Online Content Supplementary Online Content Wu HY, Peng YS, Chiang CK, et al. Diagnostic performance of random urine samples using albumin concentration vs ratio of albumin to creatinine for microalbuminuria screening

More information

Meta-analysis of diagnostic research. Karen R Steingart, MD, MPH Chennai, 15 December Overview

Meta-analysis of diagnostic research. Karen R Steingart, MD, MPH Chennai, 15 December Overview Meta-analysis of diagnostic research Karen R Steingart, MD, MPH karenst@uw.edu Chennai, 15 December 2010 Overview Describe key steps in a systematic review/ meta-analysis of diagnostic test accuracy studies

More information

An Empirical Assessment of Bivariate Methods for Meta-analysis of Test Accuracy

An Empirical Assessment of Bivariate Methods for Meta-analysis of Test Accuracy Number XX An Empirical Assessment of Bivariate Methods for Meta-analysis of Test Accuracy Prepared for: Agency for Healthcare Research and Quality U.S. Department of Health and Human Services 54 Gaither

More information

Choice of axis, tests for funnel plot asymmetry, and methods to adjust for publication bias

Choice of axis, tests for funnel plot asymmetry, and methods to adjust for publication bias Technical appendix Choice of axis, tests for funnel plot asymmetry, and methods to adjust for publication bias Choice of axis in funnel plots Funnel plots were first used in educational research and psychology,

More information

Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy

Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy Cochrane Handbook for Systematic Reviews of Diagnostic Test Accuracy Chapter 10 Analysing and Presenting Results Petra Macaskill, Constantine Gatsonis, Jonathan Deeks, Roger Harbord, Yemisi Takwoingi.

More information

Multivariate Mixed-Effects Meta-Analysis of Paired-Comparison Studies of Diagnostic Test Accuracy

Multivariate Mixed-Effects Meta-Analysis of Paired-Comparison Studies of Diagnostic Test Accuracy Multivariate Mixed-Effects Meta-Analysis of Paired-Comparison Studies of Diagnostic Test Accuracy Ben A. Dwamena, MD The University of Michigan & VA Medical Centers, Ann Arbor SNASUG - July 24, 2008 Diagnostic

More information

Methods Research Report. An Empirical Assessment of Bivariate Methods for Meta-Analysis of Test Accuracy

Methods Research Report. An Empirical Assessment of Bivariate Methods for Meta-Analysis of Test Accuracy Methods Research Report An Empirical Assessment of Bivariate Methods for Meta-Analysis of Test Accuracy Methods Research Report An Empirical Assessment of Bivariate Methods for Meta-Analysis of Test Accuracy

More information

Multivariate Mixed-Effects Meta-Analysis of Paired-Comparison Studies of Diagnostic Test Accuracy

Multivariate Mixed-Effects Meta-Analysis of Paired-Comparison Studies of Diagnostic Test Accuracy Multivariate Mixed-Effects Meta-Analysis of Paired-Comparison Studies of Diagnostic Test Accuracy Ben A. Dwamena, MD The University of Michigan & VA Medical Centers, Ann Arbor SNASUG - July 24, 2008 B.A.

More information

Meta-analysis of diagnostic accuracy studies. Mariska Leeflang (with thanks to Yemisi Takwoingi, Jon Deeks and Hans Reitsma)

Meta-analysis of diagnostic accuracy studies. Mariska Leeflang (with thanks to Yemisi Takwoingi, Jon Deeks and Hans Reitsma) Meta-analysis of diagnostic accuracy studies Mariska Leeflang (with thanks to Yemisi Takwoingi, Jon Deeks and Hans Reitsma) 1 Diagnostic Test Accuracy Reviews 1. Framing the question 2. Identification

More information

Observed Differences in Diagnostic Test Accuracy between Patient Subgroups: Is It Real or Due to Reference Standard Misclassification?

Observed Differences in Diagnostic Test Accuracy between Patient Subgroups: Is It Real or Due to Reference Standard Misclassification? Clinical Chemistry 53:10 1725 1729 (2007) Overview Observed Differences in Diagnostic Test Accuracy between Patient Subgroups: Is It Real or Due to Reference Standard Misclassification? Corné Biesheuvel,

More information

Bayesian meta-analysis of Papanicolaou smear accuracy

Bayesian meta-analysis of Papanicolaou smear accuracy Gynecologic Oncology 107 (2007) S133 S137 www.elsevier.com/locate/ygyno Bayesian meta-analysis of Papanicolaou smear accuracy Xiuyu Cong a, Dennis D. Cox b, Scott B. Cantor c, a Biometrics and Data Management,

More information

SYSTEMATIC REVIEWS OF TEST ACCURACY STUDIES

SYSTEMATIC REVIEWS OF TEST ACCURACY STUDIES Biomarker & Test Evaluation Program SYSTEMATIC REVIEWS OF TEST ACCURACY STUDIES Patrick MM Bossuyt Structure 1. Clinical Scenarios 2. Test Accuracy Studies 3. Systematic Reviews 4. Meta-Analysis 5.

More information

Meta-analysis using RevMan. Yemisi Takwoingi October 2015

Meta-analysis using RevMan. Yemisi Takwoingi October 2015 Yemisi Takwoingi October 2015 Contents 1 Introduction... 1 2 Dataset 1 PART I..2 3 Starting RevMan... 2 4 Data and analyses in RevMan... 2 5 RevMan calculator tool... 2 Table 1. Data for derivation of

More information

Meta-analysis of Diagnostic Test Accuracy Studies

Meta-analysis of Diagnostic Test Accuracy Studies GUIDELINE Meta-analysis of Diagnostic Test Accuracy Studies November 2014 Copyright EUnetHTA 2013. All Rights Reserved. No part of this document may be reproduced without an explicit acknowledgement of

More information

EVIDENCE-BASED GUIDELINE DEVELOPMENT FOR DIAGNOSTIC QUESTIONS

EVIDENCE-BASED GUIDELINE DEVELOPMENT FOR DIAGNOSTIC QUESTIONS EVIDENCE-BASED GUIDELINE DEVELOPMENT FOR DIAGNOSTIC QUESTIONS Emily Vella, Xiaomei Yao Cancer Care Ontario's Program in Evidence-Based Care, Department of Oncology, McMaster University, Ontario, Canada

More information

MIDAS RETOUCH REGARDING DIAGNOSTIC ACCURACY META-ANALYSIS

MIDAS RETOUCH REGARDING DIAGNOSTIC ACCURACY META-ANALYSIS MIDAS RETOUCH REGARDING DIAGNOSTIC ACCURACY META-ANALYSIS Ben A. Dwamena, MD University of Michigan Radiology and VA Nuclear Medicine, Ann Arbor 2014 Stata Conference, Boston, MA - July 31, 2014 Dwamena

More information

Assessing variability in results in systematic reviews of diagnostic studies

Assessing variability in results in systematic reviews of diagnostic studies Naaktgeboren et al. BMC Medical Research Methodology (2016) 16:6 DOI 10.1186/s12874-016-0108-4 RESEARCH ARTICLE Open Access Assessing variability in results in systematic reviews of diagnostic studies

More information

Studies reporting ROC curves of diagnostic and prediction data can be incorporated into meta-analyses using corresponding odds ratios

Studies reporting ROC curves of diagnostic and prediction data can be incorporated into meta-analyses using corresponding odds ratios Journal of Clinical Epidemiology 60 (2007) 530e534 BRIEF REPORT Studies reporting ROC curves of diagnostic and prediction data can be incorporated into meta-analyses using corresponding odds ratios S.D.

More information

Systematic Reviews of Studies Quantifying the Accuracy of Diagnostic Tests and Markers

Systematic Reviews of Studies Quantifying the Accuracy of Diagnostic Tests and Markers Papers in Press. Published September 18, 2012 as doi:10.1373/clinchem.2012.182568 The latest version is at http://hwmaint.clinchem.org/cgi/doi/10.1373/clinchem.2012.182568 Clinical Chemistry 58:11 000

More information

Department of Epidemiology, Rollins School of Public Health, Emory University, Atlanta GA, USA.

Department of Epidemiology, Rollins School of Public Health, Emory University, Atlanta GA, USA. A More Intuitive Interpretation of the Area Under the ROC Curve A. Cecile J.W. Janssens, PhD Department of Epidemiology, Rollins School of Public Health, Emory University, Atlanta GA, USA. Corresponding

More information

METHODS FOR DETECTING CERVICAL CANCER

METHODS FOR DETECTING CERVICAL CANCER Chapter III METHODS FOR DETECTING CERVICAL CANCER 3.1 INTRODUCTION The successful detection of cervical cancer in a variety of tissues has been reported by many researchers and baseline figures for the

More information

Review. Imagine the following table being obtained as a random. Decision Test Diseased Not Diseased Positive TP FP Negative FN TN

Review. Imagine the following table being obtained as a random. Decision Test Diseased Not Diseased Positive TP FP Negative FN TN Outline 1. Review sensitivity and specificity 2. Define an ROC curve 3. Define AUC 4. Non-parametric tests for whether or not the test is informative 5. Introduce the binormal ROC model 6. Discuss non-parametric

More information

Statistical methods for the meta-analysis of full ROC curves

Statistical methods for the meta-analysis of full ROC curves Statistical methods for the meta-analysis of full ROC curves Oliver Kuss (joint work with Stefan Hirt and Annika Hoyer) German Diabetes Center, Leibniz Institute for Diabetes Research at Heinrich Heine

More information

Statistical methods for the meta-analysis of full ROC curves

Statistical methods for the meta-analysis of full ROC curves Statistical methods for the meta-analysis of full ROC curves Oliver Kuss (joint work with Stefan Hirt and Annika Hoyer) German Diabetes Center, Leibniz Institute for Diabetes Research at Heinrich Heine

More information

Fixed Effect Combining

Fixed Effect Combining Meta-Analysis Workshop (part 2) Michael LaValley December 12 th 2014 Villanova University Fixed Effect Combining Each study i provides an effect size estimate d i of the population value For the inverse

More information

Accuracy of enzyme-linked immunospot assay for diagnosis of pleural tuberculosis: a meta-analysis

Accuracy of enzyme-linked immunospot assay for diagnosis of pleural tuberculosis: a meta-analysis Accuracy of enzyme-linked immunospot assay for diagnosis of pleural tuberculosis: a meta-analysis Z.Z. Li 1, W.Z. Qin 1, L. Li 1, Q. Wu 1 and Y.J. Wang 1,2 1 West China School of Medicine/West China Hospital,

More information

1 Introduction. st0020. The Stata Journal (2002) 2, Number 3, pp

1 Introduction. st0020. The Stata Journal (2002) 2, Number 3, pp The Stata Journal (22) 2, Number 3, pp. 28 289 Comparative assessment of three common algorithms for estimating the variance of the area under the nonparametric receiver operating characteristic curve

More information

The recommended method for diagnosing sleep

The recommended method for diagnosing sleep reviews Measuring Agreement Between Diagnostic Devices* W. Ward Flemons, MD; and Michael R. Littner, MD, FCCP There is growing interest in using portable monitoring for investigating patients with suspected

More information

23.45 (95%CI ) 0.11 (95%CI ) (95%CI ) (pleural effusion);

23.45 (95%CI ) 0.11 (95%CI ) (95%CI ) (pleural effusion); CHEST -γ -γ -γ -γ 22 -γ 0.89 (95%CI 0.87 0.91) 0.97 (95%CI 0.96 0.98) 23.45 (95%CI 17.31 31.78) 0.11 (95%CI 0.07 0.16) 272.7 (95%CI 147.5 504.2) -γ -γ -γ (interferon); (pleural effusion); (tuberculosis)

More information

Atherosclerosis 220 (2012) Contents lists available at ScienceDirect. Atherosclerosis

Atherosclerosis 220 (2012) Contents lists available at ScienceDirect. Atherosclerosis Atherosclerosis 220 (2012) 128 133 Contents lists available at ScienceDirect Atherosclerosis journal homepage: www.elsevier.com/locate/atherosclerosis Carotid plaque, compared with carotid intima-media

More information

The conditional relative odds ratio provided less biased results for comparing diagnostic test accuracy in meta-analyses

The conditional relative odds ratio provided less biased results for comparing diagnostic test accuracy in meta-analyses Journal of Clinical Epidemiology 57 (2004) 461 469 The conditional relative odds ratio provided less biased results for comparing diagnostic test accuracy in meta-analyses Sadao Suzuki a,b, *, Takeo Moro-oka

More information

Index. Springer International Publishing Switzerland 2017 T.J. Cleophas, A.H. Zwinderman, Modern Meta-Analysis, DOI /

Index. Springer International Publishing Switzerland 2017 T.J. Cleophas, A.H. Zwinderman, Modern Meta-Analysis, DOI / Index A Adjusted Heterogeneity without Overdispersion, 63 Agenda-driven bias, 40 Agenda-Driven Meta-Analyses, 306 307 Alternative Methods for diagnostic meta-analyses, 133 Antihypertensive effect of potassium,

More information

Evaluating the results of a Systematic Review/Meta- Analysis

Evaluating the results of a Systematic Review/Meta- Analysis Open Access Publication Evaluating the results of a Systematic Review/Meta- Analysis by Michael Turlik, DPM 1 The Foot and Ankle Online Journal 2 (7): 5 This is the second of two articles discussing the

More information

Introduction to ROC analysis

Introduction to ROC analysis Introduction to ROC analysis Andriy I. Bandos Department of Biostatistics University of Pittsburgh Acknowledgements Many thanks to Sam Wieand, Nancy Obuchowski, Brenda Kurland, and Todd Alonzo for previous

More information

Detection of Unknown Confounders. by Bayesian Confirmatory Factor Analysis

Detection of Unknown Confounders. by Bayesian Confirmatory Factor Analysis Advanced Studies in Medical Sciences, Vol. 1, 2013, no. 3, 143-156 HIKARI Ltd, www.m-hikari.com Detection of Unknown Confounders by Bayesian Confirmatory Factor Analysis Emil Kupek Department of Public

More information

Biostatistics II

Biostatistics II Biostatistics II 514-5509 Course Description: Modern multivariable statistical analysis based on the concept of generalized linear models. Includes linear, logistic, and Poisson regression, survival analysis,

More information

Various performance measures in Binary classification An Overview of ROC study

Various performance measures in Binary classification An Overview of ROC study Various performance measures in Binary classification An Overview of ROC study Suresh Babu. Nellore Department of Statistics, S.V. University, Tirupati, India E-mail: sureshbabu.nellore@gmail.com Abstract

More information

Bivariate analysis of sensitivity and specificity produces informative summary measures in diagnostic reviews

Bivariate analysis of sensitivity and specificity produces informative summary measures in diagnostic reviews Journal of Clinical Epidemiology 58 (2005) 982 990 Bivariate analysis of sensitivity and specificity produces informative summary measures in diagnostic reviews Johannes B. Reitsma a, *, Afina S. Glas

More information

Accuracy of pulse oximetry in screening for congenital heart disease in asymptomatic newborns: a systematic review

Accuracy of pulse oximetry in screening for congenital heart disease in asymptomatic newborns: a systematic review Accuracy of pulse oximetry in screening for congenital heart disease in asymptomatic newborns: a systematic review Shakila Thangaratinam, Jane Daniels, Andrew K Ewer, Javier Zamora, Khalid S Khan Archives

More information

Performance of rapid influenza H1N1 diagnostic tests: a meta-analysis

Performance of rapid influenza H1N1 diagnostic tests: a meta-analysis DOI:10.1111/j.170-269.2011.00284.x www.influenzajournal.com Review Article Performance of rapid influenza H1N1 diagnostic tests: a meta-analysis Haitao Chu, a Eric T. Lofgren, b M. Elizabeth Halloran,

More information

Meta-Analysis Methods used in Radiology Journals

Meta-Analysis Methods used in Radiology Journals Meta-Analysis Methods used in Radiology Journals Trevor McGrath Medical Student University of Ottawa April 15, 2016 Disclosure Statement This project was funded by: University of Ottawa Department of Radiology

More information

Journal of Biostatistics and Epidemiology

Journal of Biostatistics and Epidemiology Journal of Biostatistics and Epidemiology Original Article Usage of statistical methods and study designs in publication of specialty of general medicine and its secular changes Swati Patel 1*, Vipin Naik

More information

Models for potentially biased evidence in meta-analysis using empirically based priors

Models for potentially biased evidence in meta-analysis using empirically based priors Models for potentially biased evidence in meta-analysis using empirically based priors Nicky Welton Thanks to: Tony Ades, John Carlin, Doug Altman, Jonathan Sterne, Ross Harris RSS Avon Local Group Meeting,

More information

Hayden Smith, PhD, MPH /\ v._

Hayden Smith, PhD, MPH /\ v._ Hayden Smith, PhD, MPH.. + /\ v._ Information and clinical examples provided in presentation are strictly for educational purposes, and should not be substituted for clinical guidelines or up-to-date medical

More information

Critical reading of diagnostic imaging studies. Lecture Goals. Constantine Gatsonis, PhD. Brown University

Critical reading of diagnostic imaging studies. Lecture Goals. Constantine Gatsonis, PhD. Brown University Critical reading of diagnostic imaging studies Constantine Gatsonis Center for Statistical Sciences Brown University Annual Meeting Lecture Goals 1. Review diagnostic imaging evaluation goals and endpoints.

More information

Summarising and validating test accuracy results across multiple studies for use in clinical practice

Summarising and validating test accuracy results across multiple studies for use in clinical practice Summarising and validating test accuracy results across multiple studies for use in clinical practice Richard D. Riley Professor of Biostatistics Research Institute for Primary Care & Health Sciences Thank

More information

Reporting and methods in systematic reviews of comparative accuracy

Reporting and methods in systematic reviews of comparative accuracy Reporting and methods in systematic reviews of comparative accuracy Yemisi Takwoingi, Richard Riley and Jon Deeks Public Health, Epidemiology and Biostatistics Which test is best? Example What is the evidence?

More information

Empirical assessment of univariate and bivariate meta-analyses for comparing the accuracy of diagnostic tests

Empirical assessment of univariate and bivariate meta-analyses for comparing the accuracy of diagnostic tests Empirical assessment of univariate and bivariate meta-analyses for comparing the accuracy of diagnostic tests Yemisi Takwoingi, Richard Riley and Jon Deeks Outline Rationale Methods Findings Summary Motivating

More information

What is indirect comparison?

What is indirect comparison? ...? series New title Statistics Supported by sanofi-aventis What is indirect comparison? Fujian Song BMed MMed PhD Reader in Research Synthesis, Faculty of Health, University of East Anglia Indirect comparison

More information

MicroRNAs are novel non-invasive diagnostic biomarkers for pulmonary embolism: a meta-analysis

MicroRNAs are novel non-invasive diagnostic biomarkers for pulmonary embolism: a meta-analysis Original Article MicroRNAs are novel non-invasive diagnostic biomarkers for pulmonary embolism: a meta-analysis Han-Yu Deng 1,2, Gang Li 1,2, Jun Luo 1,2, Zhi-Qiang Wang 1,2, Xiao-Yan Yang 3, Yi-Dan Lin

More information

Comparison of Meta-Analytic Results of Indirect, Direct, and Combined Comparisons of Drugs for Chronic Insomnia in Adults: A Case Study

Comparison of Meta-Analytic Results of Indirect, Direct, and Combined Comparisons of Drugs for Chronic Insomnia in Adults: A Case Study ORIGINAL ARTICLE Comparison of Meta-Analytic Results of Indirect, Direct, and Combined Comparisons of Drugs for Chronic Insomnia in Adults: A Case Study Ben W. Vandermeer, BSc, MSc, Nina Buscemi, PhD,

More information

The spectrum effect in tests for risk prediction, screening, and diagnosis

The spectrum effect in tests for risk prediction, screening, and diagnosis open access The spectrum effect in tests for risk prediction, screening, and diagnosis Juliet A Usher-Smith, Stephen J Sharp, Simon J Griffin, The Primary Care Unit, University of Cambridge, Strangeways

More information

Performance of the Trim and Fill Method in Adjusting for the Publication Bias in Meta-Analysis of Continuous Data

Performance of the Trim and Fill Method in Adjusting for the Publication Bias in Meta-Analysis of Continuous Data American Journal of Applied Sciences 9 (9): 1512-1517, 2012 ISSN 1546-9239 2012 Science Publication Performance of the Trim and Fill Method in Adjusting for the Publication Bias in Meta-Analysis of Continuous

More information

Fixed-Effect Versus Random-Effects Models

Fixed-Effect Versus Random-Effects Models PART 3 Fixed-Effect Versus Random-Effects Models Introduction to Meta-Analysis. Michael Borenstein, L. V. Hedges, J. P. T. Higgins and H. R. Rothstein 2009 John Wiley & Sons, Ltd. ISBN: 978-0-470-05724-7

More information

Does Body Mass Index Adequately Capture the Relation of Body Composition and Body Size to Health Outcomes?

Does Body Mass Index Adequately Capture the Relation of Body Composition and Body Size to Health Outcomes? American Journal of Epidemiology Copyright 1998 by The Johns Hopkins University School of Hygiene and Public Health All rights reserved Vol. 147, No. 2 Printed in U.S.A A BRIEF ORIGINAL CONTRIBUTION Does

More information

Introduction to systematic reviews/metaanalysis

Introduction to systematic reviews/metaanalysis Introduction to systematic reviews/metaanalysis Hania Szajewska The Medical University of Warsaw Department of Paediatrics hania@ipgate.pl Do I needknowledgeon systematicreviews? Bastian H, Glasziou P,

More information

What Is Evidence-Based Medicine? 1 Critical Thinking Skills Symposium

What Is Evidence-Based Medicine? 1 Critical Thinking Skills Symposium Special Report Radiology Alliance for Health Services Research What Is Evidence-Based Medicine? 1 Critical Thinking Skills Symposium Kelly H. Zou, PhD, Julia R. Fielding, MD, Silvia Ondategui-Parra, MD,

More information

Systematic Reviews. Simon Gates 8 March 2007

Systematic Reviews. Simon Gates 8 March 2007 Systematic Reviews Simon Gates 8 March 2007 Contents Reviewing of research Why we need reviews Traditional narrative reviews Systematic reviews Components of systematic reviews Conclusions Key reference

More information

GUIDELINE COMPARATORS & COMPARISONS:

GUIDELINE COMPARATORS & COMPARISONS: GUIDELINE COMPARATORS & COMPARISONS: Direct and indirect comparisons Adapted version (2015) based on COMPARATORS & COMPARISONS: Direct and indirect comparisons - February 2013 The primary objective of

More information

How to interpret results of metaanalysis

How to interpret results of metaanalysis How to interpret results of metaanalysis Tony Hak, Henk van Rhee, & Robert Suurmond Version 1.0, March 2016 Version 1.3, Updated June 2018 Meta-analysis is a systematic method for synthesizing quantitative

More information

Sensitivity, Specificity, and Relatives

Sensitivity, Specificity, and Relatives Sensitivity, Specificity, and Relatives Brani Vidakovic ISyE 6421/ BMED 6700 Vidakovic, B. Se Sp and Relatives January 17, 2017 1 / 26 Overview Today: Vidakovic, B. Se Sp and Relatives January 17, 2017

More information

Australian Dental Journal

Australian Dental Journal Australian Dental Journal The official journal of the Australian Dental Association Australian Dental Journal 2015; 60: 233 239 doi: 10.1111/adj.12326 Diagnostic value of panoramic radiography in predicting

More information

Unit 1 Exploring and Understanding Data

Unit 1 Exploring and Understanding Data Unit 1 Exploring and Understanding Data Area Principle Bar Chart Boxplot Conditional Distribution Dotplot Empirical Rule Five Number Summary Frequency Distribution Frequency Polygon Histogram Interquartile

More information

Assessment of performance and decision curve analysis

Assessment of performance and decision curve analysis Assessment of performance and decision curve analysis Ewout Steyerberg, Andrew Vickers Dept of Public Health, Erasmus MC, Rotterdam, the Netherlands Dept of Epidemiology and Biostatistics, Memorial Sloan-Kettering

More information

Clinical Epidemiology for the uninitiated

Clinical Epidemiology for the uninitiated Clinical epidemiologist have one foot in clinical care and the other in clinical practice research. As clinical epidemiologists we apply a wide array of scientific principles, strategies and tactics to

More information

Knowledge Discovery and Data Mining. Testing. Performance Measures. Notes. Lecture 15 - ROC, AUC & Lift. Tom Kelsey. Notes

Knowledge Discovery and Data Mining. Testing. Performance Measures. Notes. Lecture 15 - ROC, AUC & Lift. Tom Kelsey. Notes Knowledge Discovery and Data Mining Lecture 15 - ROC, AUC & Lift Tom Kelsey School of Computer Science University of St Andrews http://tom.home.cs.st-andrews.ac.uk twk@st-andrews.ac.uk Tom Kelsey ID5059-17-AUC

More information

Clinical Trials A Practical Guide to Design, Analysis, and Reporting

Clinical Trials A Practical Guide to Design, Analysis, and Reporting Clinical Trials A Practical Guide to Design, Analysis, and Reporting Duolao Wang, PhD Ameet Bakhai, MBBS, MRCP Statistician Cardiologist Clinical Trials A Practical Guide to Design, Analysis, and Reporting

More information

Meta Analysis. David R Urbach MD MSc Outcomes Research Course December 4, 2014

Meta Analysis. David R Urbach MD MSc Outcomes Research Course December 4, 2014 Meta Analysis David R Urbach MD MSc Outcomes Research Course December 4, 2014 Overview Definitions Identifying studies Appraising studies Quantitative synthesis Presentation of results Examining heterogeneity

More information

Comparison of 18 FDG PET/PET-CT and bone scintigraphy for detecting bone metastases in patients with nasopharyngeal cancer: a meta-analysis

Comparison of 18 FDG PET/PET-CT and bone scintigraphy for detecting bone metastases in patients with nasopharyngeal cancer: a meta-analysis /, 2017, Vol. 8, (No. 35), pp: 59740-59747 Comparison of FDG PET/PET-CT and bone scintigraphy for detecting bone metastases in patients with nasopharyngeal cancer: a meta-analysis Chuanhui Xu 1,*, Ruiming

More information

Diagnostic methods 2: receiver operating characteristic (ROC) curves

Diagnostic methods 2: receiver operating characteristic (ROC) curves abc of epidemiology http://www.kidney-international.org & 29 International Society of Nephrology Diagnostic methods 2: receiver operating characteristic (ROC) curves Giovanni Tripepi 1, Kitty J. Jager

More information

Overview. Goals of Interpretation. Methodology. Reasons to Read and Evaluate

Overview. Goals of Interpretation. Methodology. Reasons to Read and Evaluate Overview Critical Literature Evaluation and Biostatistics Ahl Ashley N. Lewis, PharmD, BCPS Clinical Specialist, Drug Information UNC Hospitals Background Review of basic statistics Statistical tests Clinical

More information

Interval Likelihood Ratios: Another Advantage for the Evidence-Based Diagnostician

Interval Likelihood Ratios: Another Advantage for the Evidence-Based Diagnostician EVIDENCE-BASED EMERGENCY MEDICINE/ SKILLS FOR EVIDENCE-BASED EMERGENCY CARE Interval Likelihood Ratios: Another Advantage for the Evidence-Based Diagnostician Michael D. Brown, MD Mathew J. Reeves, PhD

More information

Research and Evaluation Methodology Program, School of Human Development and Organizational Studies in Education, University of Florida

Research and Evaluation Methodology Program, School of Human Development and Organizational Studies in Education, University of Florida Vol. 2 (1), pp. 22-39, Jan, 2015 http://www.ijate.net e-issn: 2148-7456 IJATE A Comparison of Logistic Regression Models for Dif Detection in Polytomous Items: The Effect of Small Sample Sizes and Non-Normality

More information

MEASURES OF ASSOCIATION AND REGRESSION

MEASURES OF ASSOCIATION AND REGRESSION DEPARTMENT OF POLITICAL SCIENCE AND INTERNATIONAL RELATIONS Posc/Uapp 816 MEASURES OF ASSOCIATION AND REGRESSION I. AGENDA: A. Measures of association B. Two variable regression C. Reading: 1. Start Agresti

More information

11/18/2013. Correlational Research. Correlational Designs. Why Use a Correlational Design? CORRELATIONAL RESEARCH STUDIES

11/18/2013. Correlational Research. Correlational Designs. Why Use a Correlational Design? CORRELATIONAL RESEARCH STUDIES Correlational Research Correlational Designs Correlational research is used to describe the relationship between two or more naturally occurring variables. Is age related to political conservativism? Are

More information

Thyroid nodules are extremely common and are observed

Thyroid nodules are extremely common and are observed Systematic Review Contrast-Enhanced Ultrasound for Differentiation of Benign and Malignant Thyroid Lesions: Meta-analysis Otolaryngology Head and Neck Surgery 2014, Vol. 151(6) 909 915 Ó American Academy

More information

Appendix 1. Sensitivity analysis for ACQ: missing value analysis by multiple imputation

Appendix 1. Sensitivity analysis for ACQ: missing value analysis by multiple imputation Appendix 1 Sensitivity analysis for ACQ: missing value analysis by multiple imputation A sensitivity analysis was carried out on the primary outcome measure (ACQ) using multiple imputation (MI). MI is

More information

CLINICAL BIOSTATISTICS

CLINICAL BIOSTATISTICS 09/06/17 1 Overview and Descriptive Statistics a. Application of statistics in biomedical research b. Type of data c. Graphic representation of data d. Summary statistics: central tendency and dispersion

More information

Receiver operating characteristic

Receiver operating characteristic Receiver operating characteristic From Wikipedia, the free encyclopedia In signal detection theory, a receiver operating characteristic (ROC), or simply ROC curve, is a graphical plot of the sensitivity,

More information

Evidence-Based Medicine: Diagnostic study

Evidence-Based Medicine: Diagnostic study Evidence-Based Medicine: Diagnostic study What is Evidence-Based Medicine (EBM)? Expertise in integrating 1. Best research evidence 2. Clinical Circumstance 3. Patient values in clinical decisions Haynes,

More information

Biomarker adaptive designs in clinical trials

Biomarker adaptive designs in clinical trials Review Article Biomarker adaptive designs in clinical trials James J. Chen 1, Tzu-Pin Lu 1,2, Dung-Tsa Chen 3, Sue-Jane Wang 4 1 Division of Bioinformatics and Biostatistics, National Center for Toxicological

More information

Analysis of Rheumatoid Arthritis Data using Logistic Regression and Penalized Approach

Analysis of Rheumatoid Arthritis Data using Logistic Regression and Penalized Approach University of South Florida Scholar Commons Graduate Theses and Dissertations Graduate School November 2015 Analysis of Rheumatoid Arthritis Data using Logistic Regression and Penalized Approach Wei Chen

More information

Between-trial heterogeneity in meta-analyses may be partially explained by reported design characteristics

Between-trial heterogeneity in meta-analyses may be partially explained by reported design characteristics Journal of Clinical Epidemiology 95 (2018) 45e54 ORIGINAL ARTICLE Between-trial heterogeneity in meta-analyses may be partially explained by reported design characteristics Kirsty M. Rhodes a, *, Rebecca

More information

Systematic reviews of diagnostic test accuracy studies

Systematic reviews of diagnostic test accuracy studies Systematic reviews of diagnostic test accuracy studies McGill Summer Institute, Montreal June 2017 Karen R Steingart, MD, MPH Cochrane Infectious Diseases Group Liverpool School of Tropical Medicine karen.steingart@gmail.com

More information

ExperimentalPhysiology

ExperimentalPhysiology Exp Physiol 97.5 (2012) pp 557 561 557 Editorial ExperimentalPhysiology Categorized or continuous? Strength of an association and linear regression Gordon B. Drummond 1 and Sarah L. Vowler 2 1 Department

More information

Evidence Based Medicine

Evidence Based Medicine Course Goals Goals 1. Understand basic concepts of evidence based medicine (EBM) and how EBM facilitates optimal patient care. 2. Develop a basic understanding of how clinical research studies are designed

More information

Systematic reviews and meta-analyses of diagnostic test accuracy

Systematic reviews and meta-analyses of diagnostic test accuracy REVIEW 10.1111/1469-0691.12474 Systematic reviews and meta-analyses of diagnostic test accuracy M. M. G. Leeflang Clinical Epidemiology and Biostatistics and Bioinformatics, Academic Medical Centre, University

More information

Meta-analysis using HLM 1. Running head: META-ANALYSIS FOR SINGLE-CASE INTERVENTION DESIGNS

Meta-analysis using HLM 1. Running head: META-ANALYSIS FOR SINGLE-CASE INTERVENTION DESIGNS Meta-analysis using HLM 1 Running head: META-ANALYSIS FOR SINGLE-CASE INTERVENTION DESIGNS Comparing Two Meta-Analysis Approaches for Single Subject Design: Hierarchical Linear Model Perspective Rafa Kasim

More information

10. LINEAR REGRESSION AND CORRELATION

10. LINEAR REGRESSION AND CORRELATION 1 10. LINEAR REGRESSION AND CORRELATION The contingency table describes an association between two nominal (categorical) variables (e.g., use of supplemental oxygen and mountaineer survival ). We have

More information

SEED HAEMATOLOGY. Medical statistics your support when interpreting results SYSMEX EDUCATIONAL ENHANCEMENT AND DEVELOPMENT APRIL 2015

SEED HAEMATOLOGY. Medical statistics your support when interpreting results SYSMEX EDUCATIONAL ENHANCEMENT AND DEVELOPMENT APRIL 2015 SYSMEX EDUCATIONAL ENHANCEMENT AND DEVELOPMENT APRIL 2015 SEED HAEMATOLOGY Medical statistics your support when interpreting results The importance of statistical investigations Modern medicine is often

More information

New South Wales 2006; Australia

New South Wales 2006; Australia STATISTICS IN MEDICINE Statist. Med. 2002; 21:853 862 (DOI: 10.1002/sim.1066) Analytic methods for comparing two dichotomous screening or diagnostic tests applied to two populations of diering disease

More information

Systematic Reviews and Meta- Analysis in Kidney Transplantation

Systematic Reviews and Meta- Analysis in Kidney Transplantation Systematic Reviews and Meta- Analysis in Kidney Transplantation Greg Knoll MD MSc Associate Professor of Medicine Medical Director, Kidney Transplantation University of Ottawa and The Ottawa Hospital KRESCENT

More information

Study protocol v. 1.0 Systematic review of the Sequential Organ Failure Assessment score as a surrogate endpoint in randomized controlled trials

Study protocol v. 1.0 Systematic review of the Sequential Organ Failure Assessment score as a surrogate endpoint in randomized controlled trials Study protocol v. 1.0 Systematic review of the Sequential Organ Failure Assessment score as a surrogate endpoint in randomized controlled trials Harm Jan de Grooth, Jean Jacques Parienti, [to be determined],

More information