Issues in Meta-Analysis: An Overview
|
|
- Griselda Hunt
- 5 years ago
- Views:
Transcription
1 REVIEW Issues in Meta-Analysis: An Overview Alfred A. Bartolucci, Charles R. Katholi, Karan P. Singh, and Graciela S. Alarc6n A summary is provided for the relevant issues that investigators must address when wishing to synthesize data via a meta-analysis. The analogy is made to conducting a clinical trial and its components. Both design and statistical issues are discussed with the goal of assuring that the proper elements are included in the meta-analysis. Further, global concerns are presented so that the researcher is aware of the clinical controversies faced when conducting a meta-analysis. Finally a comparison is made of the advantages and disadvantages of a large-scale clinical trial versus a meta-analysis. Key Words: Meta-analysis; Clinical trial: Statistical issues; Controversies. Meta-analysis is defined as the statistical analysis of a collection of analytic results for the purpose of integrating the findings. In medical terms the focus is on combining the results of several different studies that may not render in themselves a definitive conclusion concerning the superiority of one treatment over another. One may then determine after combining their results if superiority in one direction does in fact exist. Such analyses are becoming increasingly popular in the medical literature, where information Alfred A. Bartolucci, PhD, Charles R. Katholi, PhD. Karan P. Singh, PhD, and Graciela S. Alarcbn, MD. MPH, are at the Department of Biostatistics, School of Public Health, and Division of Immunology and Rheumatology, School of Medicine, University of Alabama at Birmingham, Birmingham, Alabama. Address correspondence to Alfred A. Bartolucci, PhD, Department of Biostatistics, Room 101 Bishop Building th Street South, University of Alabama at Birmingham, Birmingham, AL Submitted for publication August 3, 1993; accepted March 15, by the American College of Rheumatology. 156 on the efficacy of a treatment is available from a number of clinical studies with similar treatment protocols. Dickersin and Berlin [l] believe that meta-analysis should be a standard tool. That is to say it should be performed when practical before new studies are undertaken (i.e., funded) to ensure that the investigators have a proper understanding of the gaps in the existing literature and of the relevant methodologic issues in a given area. The traditional rationale for combining studies is that separate studies may have insufficient sample size or be too limited in scope to be conclusive about treatment effect. Difficulties abound when one wishes to integrate various studies. Some studies may be better designed than others; some may be controlled and others may not be. Thus, the two main functions of meta-analysis, i.e., data summarization and recognition of the pattern of results from various studies, have inherent difficulties, some of which are subjective. This will be seen more readily in the sections below. Despite these concerns, Mosteller and Chalmers [2], in an overview of meta-analyses, indicated that 16 meta-analyses per year were found in scientific journals from 1983 to These include social and behavioral as well as medical journals. The method of meta-analysis has definitely reached a stage of general acceptance in the scientific literature. As one might expect, researchers are continuing to explore this technique and to improve on the validity of the method. Design Strategies There are two major concerns that come to mind should one wish to undertake a meta-analysis. Like a clinical trial, one has to consider the design and the analysis strategies. Sacks et al. [3] outlined over 20 points that must be considered in order to conduct a proper meta-analysis. As one might suspect for the abstracted studies or data, these include the following: /94/$5.00
2 Arthritis Care and Research Issues in Meta-Analysis 157 a clear statement of eligibility criteria, accountability of all admissions, methods of treatment allocation, clearly stated hypotheses or aims, proper tracking of all patients, properly presented statistical analysis and statistical methods, all possible sources of bias must be accounted for in the analysis [for example, if the results differ by sex or socioeconomic status are these categories analyzed separately?], a sample size sufficient to justify the conclusions, proper quality control of collected data, complete discussion of results, and complete and accurate citation of references. One then scores the individual studies or publications according to their ability to meet the proper number of requirements to be used in the meta-analysis. Only those studies meeting the requisite score are considered for this type of integration. In a survey by Colditz et al. [4], the articles having one of these required attributes ranged from 11% to 95%. One can thus see that not all of the published material on a particular treatment would qualify for the intended integrated analysis. This of course leads to another topic for the scientific literature on exactly what types of articles should be included in the meta-analysis. Usually that decision rests with more than one person doing the literature search. In other words it is the old saying that there is safety in numbers. The more individuals who agree on what articles should be included in the meta-analysis, then presumably the more valid will be the undertaking. Of course, this leaves the extra burden on investigators. They must determine an acceptable level of agreement among themselves before accepting material for the analysis. One must be aware that there are problems with the scoring or quality scheme. There has been no formal assessment of reliability between scorers, validity, or comparability of scoring methods. Jenicek [5] discusses a variety of scoring methods. The basic difficulty has been in translating the quality of articles into a valid scoring system. For example, a well-designed study that is reported poorly may receive a poor score and vice versa for a poorly designed study that is well reported, although the latter may seem a bit unlikely. Synthesizing the Studies Further problems and controversies concerning meta-analyses are discussed later. For the present, assume that one has sufficient articles and reports to conduct a meta-analysis. The next major concern, as in a clinical trial, is how to synthesize and analyze this information. The details of such an analysis are too extensive and technical to attempt to summarize in this short article. However, we will try to familiarize the reader with some of the concerns that have been expressed when approaching the data summary stage of this work. As stated earlier, the difficulty in the integration of the results stems mainly from the diverse nature of the studies both in terms of design and analysis methods employed. For example, some studies may be randomized and stratified, whereas others may have some combination of these features. Because of the differing sample sizes and patient populations [due to varying eligibility criteria], each study has a different level of sampling error. Also, because of the differing sample sizes of the studies, one may wish to consider a weighting scheme that attributes more importance to the larger studies. Various considerations have gone into the weighting scheme. For example, the weighted average of within-study differences, as is commonly used, may be constant, a function of the sample size of individual studies, a function of withinstrata sample size, or a function of within-study differences. Another important concern is that a metaanalysis by its very purpose is retrospective in nature. That is to say, it attempts to synthesize and analyze data that have already been observed. This in itself has left the methodology susceptible to the pitfalls of retrospective research. Another issue that has concerned individuals wishing to analyze combined studies may be labeled as heterogeneity. Chalmers [6] has addressed this issue. Heterogeneity occurs when the size or direction as well as variance between the treatment and control group differ across the studies being considered. This makes it difficult statistically to combine a function of these differences, called the effect size, and thus say they share a common variance so that some traditional statistical tests may be performed. Clinicians should be aware that heterogeneity can be the result of phenomenon, such as how patients vary across studies in their pretreatment characteristics. Thus, there could be a legitimate objection to combining data from two different studies with nonhomogeneous patient characteristics. As a result, one of the first steps in performing the meta-analysis is to test for the homogeneity of the effect sizes among the studies being considered. Work is currently being performed on the examination of ways of calculating sample variances in meta-analyses for valid analysis purposes [6]. Subjectivity may abound in this context in that one may ignore among-study variability in summarizing mean effect sizes, relative risks, or odds ratios. This is called assuming a fixed effects model. When one chooses to incorporate among-study variability or heterogeneity into these calculations, which may be more appropriate, one assumes a random effects model. The choice has consequences on the widths of the confidence limits of point estimates in a meta-analysis. This
3 158 Bartolucci et a]. Vol. 7, No. 3, September 1994 idea of heterogeneity also affects the interpretation of the final determination of the effect size. That is to say that this one result may be misleading, because if studies differ substantially in the size or direction of the effect, then one must recognize that the average results may not be representative of the individual components [studies] contributing to the average. Another problem is the issue of comprehensiveness. Simply stated, this concern presents itself in the literature search of relevant articles or reports on the treatment of interest if an insufficient amount of material is determined to be relevant for the meta-analysis. One would not want to perform a statistical analysis unless all the data were available, for fear of introducing a bias into the interpretation of the results due to missing or incomplete data. Naturally, in many cases this is a common phenomenon. However, clinicians and statisticians involved in modern-day clinical trials have learned to minimize this problem by such tactics as including all eligible patients and doing an intent-to-treat analysis. In a meta-analysis one does not have all the data at hand, either complete or incomplete. It is not a self-contained entity. As stated previously, one only has reports [published or unpublished]. As a result, one has to take the precaution to ensure that the bibliographical search is as comprehensive as possible. Even when dealing with abstracts, the person wishing to perform the meta-analysis should contact the authors of the abstract being considered for inclusion to determine whether appropriate information not covered in the abstract is available. This is also necessary to ensure that the original study was methodologically correct according to the criteria required for a proper meta-analysis. This in turn helps ensure that the results of the meta-analysis are statistically sound and not biased. Felson [7] summarizes in detail many aspects of bias, one of which is publication bias. The goal is to be comprehensive in the inclusion of articles. However, publication bias can interfere with that goal in that authors, sponsors of a study, and the editor or reviewers of the journal to which a paper is submitted may be selective. Authors are less likely to submit manuscripts if they are statistically null. Editors may encourage submission of manuscripts with positive results [see [8]]. Finally, a drug company may likely discourage publication of a null trial of a drug that it sponsors. Felson discusses these infractions and ways of assessing publication bias [7]. He also suggests that an important solution to publication bias may be the establishment of clinical trial registries that include published and unpublished studies. One way to overcome or at least honestly address the comprehensiveness issue is to include relevant experts in the field of study to which the meta- analysis is addressed so that they can help direct one s search for data, published or unpublished, from various centers of excellence in that research area. One also has to make full use of computerized database searches, bibliographic reviews, and reference lists. Hogue [9], recognizing the inherent problems of database sharing, discusses why data should be shared and how data sharing should be conducted. She further elaborates on the need to reduce bias leading to inaccurate results in data presentations. Felson also points out that the best protection against this source of bias is a thorough description of the procedures used to locate the studies so that the reader can make an intelligent assessment of the representativeness and completeness of the database for a meta-analysis [7]. Statistical Analysis Up to this point we have been discussing design issues that are important to the proper means of conducting a meta-analysis. One may also wish to know what types of statistical analyses are used to analyze the synthesized data. It would be inappropriate to go into all the details of what conditions call for what types of analyses. Suffice to say that there is nothing really magical or new in terms of statistical procedures in a meta-analysis. The standard statistical tests such as the chi-square, t-test, z-test, and analysis of variance [ANOVA) are commonly used. Several authors have discussed these procedures. Most notable is the work of Hedges and Olkin [lo]. These same two authors [ll] outline the use of regression methods in meta-analyses. They have made the material accessible to a nonstatistician by avoiding cumbersome mathematical proofs in the text and by presenting them in the commentary section at the end of each chapter [lo]. This type of presentation is certainly appealing to the clinician, who may wish to discover relevant analyses for various forms of the synthesized data without the burden of having to make one s way through a morass of mathematics. The book [lo] does not discuss techniques for the literature search or data gathering strategies. Alarc6n et al. [12] discuss techniques for a proper search and evaluation of articles for a meta-analysis. They discuss the comparison of methotrexate and nonmethotrexate disease-modifying anti-rheumatic drugs in rheumatoid arthritis patients. The materials and methods section deals with the selection of articles and the scoring of each for the meta-analysis. Validity and Replicability There are several general concerns beyond the scope of the design and analysis issues discussed here. One
4 Arthritis Care and Research Issues in Meta-Analysis 159 must be aware that meta-analysis is not designed to provide overwhelming evidence for or against a particular treatment modality. That may be the purpose of a large clinical trial. Meta-analysis deals with confirming that minor increments or advantages of one treatment over another are, in fact, real. The combination of several small studies to achieve this end may provide the overall sample size that tips the scale one way or the other. One may come to the conclusion that a large clinical trial may be more reliable than a meta-analysis combining many small trials, but there is no real evidence that this is the case. Both have their advantages. A meta-analysis of many small trials run in local clinics may have the reality of the heterogeneity that one would expect to find in the general population. Also, the large-scale clinical trial is usually made up of contributions of many clinics, so the pooling of all these results, assuming homogeneity in a cooperative setting, may or may not be appropriate. Another issue, discussed by Cook et. al. [13], is whether or not one should include unpublished data in metaanalysis. This is certainly a point of discussion, because some individuals believe that peer review brings validity and consistency to the material needed in the meta-analysis. Others will argue that as long as it can be confirmed that unpublished material came from methodologically sound research, then it should be included. Because of the recent surge in interest and application of meta-analysis, the replicability of the technique has at times been discouraging. Chalmers et al. [14] reviewed replicate analyses of 20 different research questions. They pointed out that meta-analyses agreed in terms of statistical results for 10 of the research questions and disagreed for 10. The reasons for disagreement between meta-analyses for the same research questions were not obvious. It was noted, however, that the meta-analyses that disagreed tended more often to use crude methods of pooling than did those that agreed. Likewise, Chalmers et al. [15] compared meta-analyses of several trials with a single large cooperative trial for three different topics. Depending on the particular trial and endpoint, the meta-analysis agreed with the cooperative trial in some cases and disagreed in others. The explanation, for example, when the meta-analysis favors the treatment and the large trial favors the control, could be some form of bias such as publication bias, in which small trials showing null results failed to reach publication. One would hope that the enhanced combination of scientific rigor during the conduct of studies and a drastic reduction in biases plaguing meta-analyses over time will result in more consistent results across meta-analyses of similar research questions. Another point of concern that affects the validity of the meta-analyzed result is recording error bias. This actual error rate of transfer of collected data to the actual trial report is fairly minimal, usually around 1%. Although this does not appear to be a problem of great magnitude in the actual observed trial, it is yet another element contributing to the validity of the nonobservational data-type meta-analysis. This obviously is something over which the meta-analyst has little or no control. In summary, we have discussed the major relevant quantitative issues facing meta-analysis. It is wise to assume that one wishing to embark on such a project should, as in conducting a clinical trial, gather a good team consisting of clinical, statistical, and other relevant scientific experts in the disease area of interest. REFERENCES 1. Dickersin K, Berlin JA: Meta-analysis: state-of-the-science. Epidemiol Rev 14: , Mosteller F, Chalmers C: Some progress and problems in meta-analysis of clinical trials. Stat Sci 7(2): , Sacks HS, Berrier J, Reitman D, Ancona-Berk VA, Chalmers TC: Meta-analysis of randomized controlled trials. N Engl J Med 8: Colditz GA, Miller JN, Mosteller F How study design affects outcomes in comparisons of therapy. I: medical. Stat Med 8: , Jenicek M: Meta-analysis in medicine: where we are and where we want to go. J Clin Epidemiol 42(1):35-44, Chalmers TC: Problems induced by meta-analysis. Stat Med 10: , Felson DT: Bias in meta-analytic research. J Clin Epidemiol 45(8]: , Dickersin K: The existence of publication bias and risk factors for its occurrence. JAMA 263: , Hogue CI: Ethical issues in sharing epidemiologic data. J Clin Epidemiol44(Suppl 1):103S-l07S, Hedges LV, Olkin I: Statistical Methods for Meta-Analysis. New York, Academic Press, Hedges LV, Olkin I: Regression models in research synthesis. The American Statistician 37(2): , Alarc6n GS, L6pez-Mkndez A, Walter J, Boerbooms A, Russel AS, Furst DE, Rau R, Drosos AA, Bartolucci AA: Radiographic evidence of disease progression in methotrexate treated and non-methotrexate disease modifying antirheumatic drug treated rheumatoid arthritis patients: a meta-analysis. J Rheumatol 19: , Cook DJ, Guyatt GH, Ryan G, Clifton J, Buckingham L, Wilan A, McIlroy W, Oxman AD: Should unpublished data be included in meta-analysis? JAMA 269(21): , 1993.
5 160 Bartolucci et al. Vol. 7, No. 3, September Chalmers TC, Berrier TC, Sacks HS, et al: Meta-analysis of clinical trials as a scientific discipline 11: replicate variability and comparison of studies that agree and disagree. Stat Med 6: , Chalmers TC, Levin H, Sacks HS, et al: Meta-analysis of clinical trials as a scientific discipline. I: control of bias and comparison with large cooperative trials. Stat Med 6: , 1987.
Systematic Reviews and Meta- Analysis in Kidney Transplantation
Systematic Reviews and Meta- Analysis in Kidney Transplantation Greg Knoll MD MSc Associate Professor of Medicine Medical Director, Kidney Transplantation University of Ottawa and The Ottawa Hospital KRESCENT
More informationLive WebEx meeting agenda
10:00am 10:30am Using OpenMeta[Analyst] to extract quantitative data from published literature Live WebEx meeting agenda August 25, 10:00am-12:00pm ET 10:30am 11:20am Lecture (this will be recorded) 11:20am
More informationResearch Synthesis and meta-analysis: themes. Graham A. Colditz, MD, DrPH Method Tuuli, MD, MPH
Research Synthesis and meta-analysis: themes Graham A. Colditz, MD, DrPH Method Tuuli, MD, MPH Today Course format Goals, competencies Overview of themes for the class SRMA M19-551 Course Format Lectures»
More informationEFFECTIVE MEDICAL WRITING Michelle Biros, MS, MD Editor-in -Chief Academic Emergency Medicine
EFFECTIVE MEDICAL WRITING Michelle Biros, MS, MD Editor-in -Chief Academic Emergency Medicine Why We Write To disseminate information To share ideas, discoveries, and perspectives to a broader audience
More informationA brief history of the Fail Safe Number in Applied Research. Moritz Heene. University of Graz, Austria
History of the Fail Safe Number 1 A brief history of the Fail Safe Number in Applied Research Moritz Heene University of Graz, Austria History of the Fail Safe Number 2 Introduction Rosenthal s (1979)
More informationSYSTEMATIC REVIEW: AN APPROACH FOR TRANSPARENT RESEARCH SYNTHESIS
SYSTEMATIC REVIEW: AN APPROACH FOR TRANSPARENT RESEARCH SYNTHESIS A Case Study By Anil Khedkar, India (Masters in Pharmaceutical Science, PhD in Clinical Research Student of Texila American University)
More informationPublishing Your Study: Tips for Young Investigators. Learning Objectives 7/9/2013. Eric B. Bass, MD, MPH
Publishing Your Study: Tips for Young Investigators Eric B. Bass, MD, MPH Learning Objectives To apply a logical approach to organizing & presenting your work in a manuscript To recognize the importance
More informationCRITICAL EVALUATION OF BIOMEDICAL LITERATURE
Chapter 9 CRITICAL EVALUATION OF BIOMEDICAL LITERATURE M.G.Rajanandh, Department of Pharmacy Practice, SRM College of Pharmacy, SRM University. INTRODUCTION Reviewing the Biomedical Literature poses a
More informationMeta-Analysis. Zifei Liu. Biological and Agricultural Engineering
Meta-Analysis Zifei Liu What is a meta-analysis; why perform a metaanalysis? How a meta-analysis work some basic concepts and principles Steps of Meta-analysis Cautions on meta-analysis 2 What is Meta-analysis
More informationTrials and Tribulations of Systematic Reviews and Meta-Analyses
Trials and Tribulations of Systematic Reviews and Meta-Analyses Mark A. Crowther and Deborah J. Cook St. Joseph s Hospital, Hamilton, Ontario, Canada; McMaster University, Hamilton, Ontario, Canada Systematic
More informationMeasuring and Assessing Study Quality
Measuring and Assessing Study Quality Jeff Valentine, PhD Co-Chair, Campbell Collaboration Training Group & Associate Professor, College of Education and Human Development, University of Louisville Why
More informationExperimental Psychology
Title Experimental Psychology Type Individual Document Map Authors Aristea Theodoropoulos, Patricia Sikorski Subject Social Studies Course None Selected Grade(s) 11, 12 Location Roxbury High School Curriculum
More informationCochrane Pregnancy and Childbirth Group Methodological Guidelines
Cochrane Pregnancy and Childbirth Group Methodological Guidelines [Prepared by Simon Gates: July 2009, updated July 2012] These guidelines are intended to aid quality and consistency across the reviews
More informationAssignment 4: True or Quasi-Experiment
Assignment 4: True or Quasi-Experiment Objectives: After completing this assignment, you will be able to Evaluate when you must use an experiment to answer a research question Develop statistical hypotheses
More informationStandards for the reporting of new Cochrane Intervention Reviews
Methodological Expectations of Cochrane Intervention Reviews (MECIR) Standards for the reporting of new Cochrane Intervention Reviews 24 September 2012 Preface The standards below summarize proposed attributes
More informationPubH 7470: STATISTICS FOR TRANSLATIONAL & CLINICAL RESEARCH
PubH 7470: STATISTICS FOR TRANSLATIONAL & CLINICAL RESEARCH Instructor: Chap T. Le, Ph.D. Distinguished Professor of Biostatistics Basic Issues: COURSE INTRODUCTION BIOSTATISTICS BIOSTATISTICS is the Biomedical
More informationWhat is indirect comparison?
...? series New title Statistics Supported by sanofi-aventis What is indirect comparison? Fujian Song BMed MMed PhD Reader in Research Synthesis, Faculty of Health, University of East Anglia Indirect comparison
More informationEvaluating the results of a Systematic Review/Meta- Analysis
Open Access Publication Evaluating the results of a Systematic Review/Meta- Analysis by Michael Turlik, DPM 1 The Foot and Ankle Online Journal 2 (7): 5 This is the second of two articles discussing the
More informationEvidence-Based Medicine and Publication Bias Desmond Thompson Merck & Co.
Evidence-Based Medicine and Publication Bias Desmond Thompson Merck & Co. Meta-Analysis Defined A meta-analysis is: the statistical combination of two or more separate studies In other words: overview,
More informationEvaluation Models STUDIES OF DIAGNOSTIC EFFICIENCY
2. Evaluation Model 2 Evaluation Models To understand the strengths and weaknesses of evaluation, one must keep in mind its fundamental purpose: to inform those who make decisions. The inferences drawn
More information2. How do different moderators (in particular, modality and orientation) affect the results of psychosocial treatment?
Role of psychosocial treatments in management of schizophrenia: a meta-analytic review of controlled outcome studies Mojtabai R, Nicholson R A, Carpenter B N Authors' objectives To investigate the role
More informationConfidence Intervals On Subsets May Be Misleading
Journal of Modern Applied Statistical Methods Volume 3 Issue 2 Article 2 11-1-2004 Confidence Intervals On Subsets May Be Misleading Juliet Popper Shaffer University of California, Berkeley, shaffer@stat.berkeley.edu
More information1 The conceptual underpinnings of statistical power
1 The conceptual underpinnings of statistical power The importance of statistical power As currently practiced in the social and health sciences, inferential statistics rest solidly upon two pillars: statistical
More informationC2 Training: August 2010
C2 Training: August 2010 Introduction to meta-analysis The Campbell Collaboration www.campbellcollaboration.org Pooled effect sizes Average across studies Calculated using inverse variance weights Studies
More informationHow to interpret results of metaanalysis
How to interpret results of metaanalysis Tony Hak, Henk van Rhee, & Robert Suurmond Version 1.0, March 2016 Version 1.3, Updated June 2018 Meta-analysis is a systematic method for synthesizing quantitative
More informationThe Royal College of Pathologists Journal article evaluation questions
The Royal College of Pathologists Journal article evaluation questions Previous exam questions Dorrian CA, Toole, BJ, Alvarez-Madrazo S, Kelly A, Connell JMC, Wallace AM. A screening procedure for primary
More informationClinical research in AKI Timing of initiation of dialysis in AKI
Clinical research in AKI Timing of initiation of dialysis in AKI Josée Bouchard, MD Krescent Workshop December 10 th, 2011 1 Acute kidney injury in ICU 15 25% of critically ill patients experience AKI
More informationChapter Three: Sampling Methods
Chapter Three: Sampling Methods The idea of this chapter is to make sure that you address sampling issues - even though you may be conducting an action research project and your sample is "defined" by
More informationCausation, the structural engineer, and the expert witness
Causation, the structural engineer, and the expert witness This article discusses how expert witness services can be improved in construction disputes where the determination of the cause of structural
More informationCochrane Breast Cancer Group
Cochrane Breast Cancer Group Version and date: V3.2, September 2013 Intervention Cochrane Protocol checklist for authors This checklist is designed to help you (the authors) complete your Cochrane Protocol.
More informationGUIDELINE COMPARATORS & COMPARISONS:
GUIDELINE COMPARATORS & COMPARISONS: Direct and indirect comparisons Adapted version (2015) based on COMPARATORS & COMPARISONS: Direct and indirect comparisons - February 2013 The primary objective of
More informationApplying Evidence-Based Practice with Meta-Analysis
Applying Evidence-Based Practice with Meta-Analysis Mike W.-L. Cheung, PhD 1 1 March 2018 1 Department of Psychology, National University of Singapore (NUS) 1 A little bit background about me (1) PhD:
More informationClinical Guidelines And Primary Care
Clinical Guidelines And Primary Care Alfred O. Berg, MD, MPH, Series Editor Screening For Adolescent Idiopathic Scoliosis: A Report From The United States Preventive Services Task Force Alfred O. Berg,
More informationBasic Concepts in Research and DATA Analysis
Basic Concepts in Research and DATA Analysis 1 Introduction: A Common Language for Researchers...2 Steps to Follow When Conducting Research...2 The Research Question...3 The Hypothesis...3 Defining the
More informationCHAPTER LEARNING OUTCOMES
EXPERIIMENTAL METHODOLOGY CHAPTER LEARNING OUTCOMES When you have completed reading this article you will be able to: Define what is an experiment Explain the role of theory in educational research Justify
More informationA Short Primer on Power Calculations for Meta-analysis
A Short Primer on Power Calculations for Meta-analysis Terri Pigott, Associate Provost for Research, Loyola University Chicago Editor, Methods Coordinating Group, Campbell Collaboration A webcast series
More informationCritical appraisal: Systematic Review & Meta-analysis
Critical appraisal: Systematic Review & Meta-analysis Atiporn Ingsathit MD.PhD. Section for Clinical Epidemiology and biostatistics Faculty of Medicine Ramathibodi Hospital Mahidol University What is a
More informationGlossary. Ó 2010 John Wiley & Sons, Ltd
Glossary The majority of the definitions within this glossary are based on, but are only a selection from, the comprehensive list provided by Day (2007) in the Dictionary of Clinical Trials. We have added
More informationSystematic reviews and meta-analyses of observational studies (MOOSE): Checklist.
Systematic reviews and meta-analyses of observational studies (MOOSE): Checklist. MOOSE Checklist Infliximab reduces hospitalizations and surgery interventions in patients with inflammatory bowel disease:
More informationResearch Prospectus. Your major writing assignment for the quarter is to prepare a twelve-page research prospectus.
Department of Political Science UNIVERSITY OF CALIFORNIA, SAN DIEGO Philip G. Roeder Research Prospectus Your major writing assignment for the quarter is to prepare a twelve-page research prospectus. A
More information3 CONCEPTUAL FOUNDATIONS OF STATISTICS
3 CONCEPTUAL FOUNDATIONS OF STATISTICS In this chapter, we examine the conceptual foundations of statistics. The goal is to give you an appreciation and conceptual understanding of some basic statistical
More informationEpidemiologic Methods and Counting Infections: The Basics of Surveillance
Epidemiologic Methods and Counting Infections: The Basics of Surveillance Ebbing Lautenbach, MD, MPH, MSCE University of Pennsylvania School of Medicine Nothing to disclose PENN Outline Definitions / Historical
More informationClinical Epidemiology for the uninitiated
Clinical epidemiologist have one foot in clinical care and the other in clinical practice research. As clinical epidemiologists we apply a wide array of scientific principles, strategies and tactics to
More informationCHAPTER III METHODOLOGY AND PROCEDURES. In the first part of this chapter, an overview of the meta-analysis methodology is
CHAPTER III METHODOLOGY AND PROCEDURES In the first part of this chapter, an overview of the meta-analysis methodology is provided. General procedures inherent in meta-analysis are described. The other
More informationGuidelines for Writing and Reviewing an Informed Consent Manuscript From the Editors of Clinical Research in Practice: The Journal of Team Hippocrates
Guidelines for Writing and Reviewing an Informed Consent Manuscript From the Editors of Clinical Research in Practice: The Journal of Team Hippocrates 1. Title a. Emphasize the clinical utility of the
More informationLec 02: Estimation & Hypothesis Testing in Animal Ecology
Lec 02: Estimation & Hypothesis Testing in Animal Ecology Parameter Estimation from Samples Samples We typically observe systems incompletely, i.e., we sample according to a designed protocol. We then
More informationIntroduction On Assessing Agreement With Continuous Measurement
Introduction On Assessing Agreement With Continuous Measurement Huiman X. Barnhart, Michael Haber, Lawrence I. Lin 1 Introduction In social, behavioral, physical, biological and medical sciences, reliable
More informationAmbiguous Data Result in Ambiguous Conclusions: A Reply to Charles T. Tart
Other Methodology Articles Ambiguous Data Result in Ambiguous Conclusions: A Reply to Charles T. Tart J. E. KENNEDY 1 (Original publication and copyright: Journal of the American Society for Psychical
More informationIntroduction to Meta-Analysis
Introduction to Meta-Analysis Nazım Ço galtay and Engin Karada g Abstract As a means to synthesize the results of multiple studies, the chronological development of the meta-analysis method was in parallel
More informationEssential Skills for Evidence-based Practice Understanding and Using Systematic Reviews
J Nurs Sci Vol.28 No.4 Oct - Dec 2010 Essential Skills for Evidence-based Practice Understanding and Using Systematic Reviews Jeanne Grace Corresponding author: J Grace E-mail: Jeanne_Grace@urmc.rochester.edu
More informationIntroducing Meta-Analysis. and Meta-Stat. This chapter introduces meta-analysis and the Meta-Stat program.
1 Introducing Meta-Analysis and Meta-Stat This chapter introduces meta-analysis and the Meta-Stat program. Introducing Meta-Analysis Research literature, it is often pointed out, is growing at an exponential
More informationREPRODUCTIVE ENDOCRINOLOGY
FERTILITY AND STERILITY VOL. 74, NO. 2, AUGUST 2000 Copyright 2000 American Society for Reproductive Medicine Published by Elsevier Science Inc. Printed on acid-free paper in U.S.A. REPRODUCTIVE ENDOCRINOLOGY
More informationIn many healthcare situations, it is common to find
Interpreting and Using Meta-Analysis in Clinical Practice Cheryl Holly Jason T. Slyer Systematic reviews, which can include a meta-analysis, are considered the gold standard for determination of best practice.
More informationReview of Veterinary Epidemiologic Research by Dohoo, Martin, and Stryhn
The Stata Journal (2004) 4, Number 1, pp. 89 92 Review of Veterinary Epidemiologic Research by Dohoo, Martin, and Stryhn Laurent Audigé AO Foundation laurent.audige@aofoundation.org Abstract. The new book
More informationUSE AND MISUSE OF MIXED MODEL ANALYSIS VARIANCE IN ECOLOGICAL STUDIES1
Ecology, 75(3), 1994, pp. 717-722 c) 1994 by the Ecological Society of America USE AND MISUSE OF MIXED MODEL ANALYSIS VARIANCE IN ECOLOGICAL STUDIES1 OF CYNTHIA C. BENNINGTON Department of Biology, West
More information9.0 L '- ---'- ---'- --' X
352 C hap te r Ten 11.0 10.5 Y 10.0 9.5 9.0 L...- ----'- ---'- ---'- --' 0.0 0.5 1.0 X 1.5 2.0 FIGURE 10.23 Interpreting r = 0 for curvilinear data. Establishing causation requires solid scientific understanding.
More informationMethodology for Non-Randomized Clinical Trials: Propensity Score Analysis Dan Conroy, Ph.D., inventiv Health, Burlington, MA
PharmaSUG 2014 - Paper SP08 Methodology for Non-Randomized Clinical Trials: Propensity Score Analysis Dan Conroy, Ph.D., inventiv Health, Burlington, MA ABSTRACT Randomized clinical trials serve as the
More informationBiostatistics 3. Developed by Pfizer. March 2018
BROUGHT TO YOU BY Biostatistics 3 Developed by Pfizer March 2018 This learning module is intended for UK healthcare professionals only. Job bag: PP-GEP-GBR-0986 Date of preparation March 2018. Agenda I.
More informationMeta Analysis. David R Urbach MD MSc Outcomes Research Course December 4, 2014
Meta Analysis David R Urbach MD MSc Outcomes Research Course December 4, 2014 Overview Definitions Identifying studies Appraising studies Quantitative synthesis Presentation of results Examining heterogeneity
More informationSTATS8: Introduction to Biostatistics. Overview. Babak Shahbaba Department of Statistics, UCI
STATS8: Introduction to Biostatistics Overview Babak Shahbaba Department of Statistics, UCI The role of statistical analysis in science This course discusses some biostatistical methods, which involve
More informationGLOSSARY OF GENERAL TERMS
GLOSSARY OF GENERAL TERMS Absolute risk reduction Absolute risk reduction (ARR) is the difference between the event rate in the control group (CER) and the event rate in the treated group (EER). ARR =
More informationA Guide to Reviewing Manuscripts
Maternal and Child Health Journal, Vol. 9, No. 1, March 2005 ( C 2005) DOI: 10.1007/s10995-005-2423-y A Guide to Reviewing Manuscripts Greg R. Alexander, MPH, ScD 1 MCH professionals play an invaluable
More informationMeta-Analysis De-Mystified: A Step-by-Step Workshop
Meta-Analysis De-Mystified: A Step-by-Step Workshop Eval Café Evaluation Center at WMU January, 2012 Robert McCowen, IDPE robert.h.mccowen@wmich.edu Overview Background and Context Applications of Meta-Analysis
More informationTitle: Who does not participate in a follow-up postal study? A survey of infertile couples treated by in vitro fertilization
Author's response to reviews Title: Who does not participate in a follow-up postal study? A survey of infertile couples treated by in vitro fertilization Authors: Pénélope Troude (penelope.troude@inserm.fr)
More informationPEER REVIEW HISTORY ARTICLE DETAILS VERSION 1 - REVIEW. Ball State University
PEER REVIEW HISTORY BMJ Open publishes all reviews undertaken for accepted manuscripts. Reviewers are asked to complete a checklist review form (see an example) and are provided with free text boxes to
More informationTHE DEPARTMENT OF VETERANS AFFAIRS HEALTH SERVICES RESEARCH & DEVELOPMENT SERVICE 810 Vermont Avenue Washington, DC 20420
HSR&D Service Document #39-32-010 THE DEPARTMENT OF VETERANS AFFAIRS HEALTH SERVICES RESEARCH & DEVELOPMENT SERVICE 810 Vermont Avenue Washington, DC 20420 June 1989 Common Methodological Problems in Investigator
More informationINTRODUCTION. Evidence standards for justifiable evidence claims, June 2016
EVIDENCE STANDARDS: A DIMENSIONS OF DIFFERENCE FRAMEWORK FOR APPRAISING JUSTIFIABLE EVIDENCE CLAIMS 1 David Gough, EPPI-Centre, SSRU, UCL Institute of Education, University College London INTRODUCTION
More informationQuality Digest Daily, March 3, 2014 Manuscript 266. Statistics and SPC. Two things sharing a common name can still be different. Donald J.
Quality Digest Daily, March 3, 2014 Manuscript 266 Statistics and SPC Two things sharing a common name can still be different Donald J. Wheeler Students typically encounter many obstacles while learning
More informationVocabulary. Bias. Blinding. Block. Cluster sample
Bias Blinding Block Census Cluster sample Confounding Control group Convenience sample Designs Experiment Experimental units Factor Level Any systematic failure of a sampling method to represent its population
More informationThe Efficacy of Paroxetine and Placebo in Treating Anxiety and Depression: A Meta-Analysis of Change on the Hamilton Rating Scales
The Efficacy of Paroxetine and Placebo in Treating Anxiety and Depression: A Meta-Analysis of Change on the Hamilton Rating Scales The Harvard community has made this article openly available. Please share
More informationUses and misuses of the STROBE statement: bibliographic study
Uses and misuses of the STROBE statement: bibliographic study Bruno R. da Costa 1, Myriam Cevallos 1, 2, Douglas G. Altman 3, Anne W.S. Rutjes 1, Matthias Egger 1 1. Institute of Social & Preventive Medicine
More informationChapter Three Research Methodology
Chapter Three Research Methodology Research Methods is a systematic and principled way of obtaining evidence (data, information) for solving health care problems. 1 Dr. Mohammed ALnaif METHODS AND KNOWLEDGE
More informationTitle: Intention-to-treat and transparency of related practices in randomized, controlled trials of anti-infectives
Author s response to reviews Title: Intention-to-treat and transparency of related practices in randomized, controlled trials of anti-infectives Authors: Robert Beckett (rdbeckett@manchester.edu) Kathryn
More informationIntroduction to systematic reviews/metaanalysis
Introduction to systematic reviews/metaanalysis Hania Szajewska The Medical University of Warsaw Department of Paediatrics hania@ipgate.pl Do I needknowledgeon systematicreviews? Bastian H, Glasziou P,
More informationEmpirical Research Methods for Human-Computer Interaction. I. Scott MacKenzie Steven J. Castellucci
Empirical Research Methods for Human-Computer Interaction I. Scott MacKenzie Steven J. Castellucci 1 Topics The what, why, and how of empirical research Group participation in a real experiment Observations
More informationChapter 5: Field experimental designs in agriculture
Chapter 5: Field experimental designs in agriculture Jose Crossa Biometrics and Statistics Unit Crop Research Informatics Lab (CRIL) CIMMYT. Int. Apdo. Postal 6-641, 06600 Mexico, DF, Mexico Introduction
More informationAuthors face many challenges when summarising results in reviews.
Describing results Authors face many challenges when summarising results in reviews. This document aims to help authors to develop clear, consistent messages about the effects of interventions in reviews,
More informationFixed Effect Combining
Meta-Analysis Workshop (part 2) Michael LaValley December 12 th 2014 Villanova University Fixed Effect Combining Each study i provides an effect size estimate d i of the population value For the inverse
More informationChecking the counterarguments confirms that publication bias contaminated studies relating social class and unethical behavior
1 Checking the counterarguments confirms that publication bias contaminated studies relating social class and unethical behavior Gregory Francis Department of Psychological Sciences Purdue University gfrancis@purdue.edu
More informationMeta-analysis of safety thoughts from CIOMS X
CIOMS Working Group X Meta-analysis of safety thoughts from CIOMS X Stephen.Evans@Lshtm.ac.uk Improving health worldwide www.lshtm.ac.uk Evans: ENCePP CIOMS Meta Analysis 1 Acknowledgements, conflicts,
More informationStatistical Essentials in Interpreting Clinical Trials Stuart J. Pocock, PhD
Statistical Essentials in Interpreting Clinical Trials Stuart J. Pocock, PhD June 03, 2016 www.medscape.com I'm Stuart Pocock, professor of medical statistics at London University. I am going to take you
More informationThe Logic of Data Analysis Using Statistical Techniques M. E. Swisher, 2016
The Logic of Data Analysis Using Statistical Techniques M. E. Swisher, 2016 This course does not cover how to perform statistical tests on SPSS or any other computer program. There are several courses
More informationUnderstanding Science Conceptual Framework
1 Understanding Science Conceptual Framework This list of conceptual understandings regarding the nature and process of science are aligned across grade levels to help instructors identify age-appropriate
More informationChapter 8. Learning Objectives 9/10/2012. Research Principles and Evidence Based Practice
1 Chapter 8 Research Principles and Evidence Based Practice 2 Learning Objectives Explain the importance of EMS research. Distinguish between types of EMS research. Outline 10 steps to perform research
More informationCritical Appraisal of a Meta-Analysis: Rosiglitazone and CV Death. Debra Moy Faculty of Pharmacy University of Toronto
Critical Appraisal of a Meta-Analysis: Rosiglitazone and CV Death Debra Moy Faculty of Pharmacy University of Toronto Goal To provide practitioners with a systematic approach to evaluating a meta analysis
More informationDownloaded from:
Arnup, SJ; Forbes, AB; Kahan, BC; Morgan, KE; McKenzie, JE (2016) The quality of reporting in cluster randomised crossover trials: proposal for reporting items and an assessment of reporting quality. Trials,
More informationOutcomes assessed in the review
The effectiveness of mechanical compression devices in attaining hemostasis after removal of a femoral sheath following femoral artery cannulation for cardiac interventional procedures Jones T Authors'
More informationTitle:The role of BRCA1 and BRCA2 mutations in prostate, pancreatic and stomach cancers.
Author's response to reviews Title:The role of BRCA1 and BRCA2 mutations in prostate, pancreatic and stomach cancers. Authors: Helen Cavanagh (helen21987@hotmail.com) Katherine MA Rogers (k.rogers@qub.ac.uk)
More informationEPSE 594: Meta-Analysis: Quantitative Research Synthesis
EPSE 594: Meta-Analysis: Quantitative Research Synthesis Ed Kroc University of British Columbia ed.kroc@ubc.ca March 14, 2019 Ed Kroc (UBC) EPSE 594 March 14, 2019 1 / 39 Last Time Synthesizing multiple
More informationPLANNING THE RESEARCH PROJECT
Van Der Velde / Guide to Business Research Methods First Proof 6.11.2003 4:53pm page 1 Part I PLANNING THE RESEARCH PROJECT Van Der Velde / Guide to Business Research Methods First Proof 6.11.2003 4:53pm
More informationUnit 1 Exploring and Understanding Data
Unit 1 Exploring and Understanding Data Area Principle Bar Chart Boxplot Conditional Distribution Dotplot Empirical Rule Five Number Summary Frequency Distribution Frequency Polygon Histogram Interquartile
More informationSystematic reviews: From evidence to recommendation. Marcel Dijkers, PhD, FACRM Icahn School of Medicine at Mount Sinai
Systematic reviews: From evidence to recommendation Session 1 - June 4, 2014 A blast from the past: Systematic reviews and the traditional evidence pyramid Marcel Dijkers, PhD, FACRM Icahn School of Medicine
More informationStatistical probability was first discussed in the
COMMON STATISTICAL ERRORS EVEN YOU CAN FIND* PART 1: ERRORS IN DESCRIPTIVE STATISTICS AND IN INTERPRETING PROBABILITY VALUES Tom Lang, MA Tom Lang Communications Critical reviewers of the biomedical literature
More information26:010:557 / 26:620:557 Social Science Research Methods
26:010:557 / 26:620:557 Social Science Research Methods Dr. Peter R. Gillett Associate Professor Department of Accounting & Information Systems Rutgers Business School Newark & New Brunswick 1 Overview
More informationAOTA S EVIDENCE EXCHANGE CRITICALLY APPRAISED PAPER (CAP) GUIDELINES Annual AOTA Conference Poster Submissions Critically Appraised Papers (CAPs) are
AOTA S EVIDENCE EXCHANGE CRITICALLY APPRAISED PAPER (CAP) GUIDELINES Annual AOTA Conference Poster Submissions Critically Appraised Papers (CAPs) are at-a-glance summaries of the methods, findings and
More informationPHO MetaQAT Guide. Critical appraisal in public health. PHO Meta-tool for quality appraisal
PHO MetaQAT Guide Critical appraisal in public health Critical appraisal is a necessary part of evidence-based practice and decision-making, allowing us to understand the strengths and weaknesses of evidence,
More informationThe Meta on Meta-Analysis. Presented by Endia J. Lindo, Ph.D. University of North Texas
The Meta on Meta-Analysis Presented by Endia J. Lindo, Ph.D. University of North Texas Meta-Analysis What is it? Why use it? How to do it? Challenges and benefits? Current trends? What is meta-analysis?
More informationRecent developments for combining evidence within evidence streams: bias-adjusted meta-analysis
EFSA/EBTC Colloquium, 25 October 2017 Recent developments for combining evidence within evidence streams: bias-adjusted meta-analysis Julian Higgins University of Bristol 1 Introduction to concepts Standard
More informationNeuroscience and Generalized Empirical Method Go Three Rounds
Bruce Anderson, Neuroscience and Generalized Empirical Method Go Three Rounds: Review of Robert Henman s Global Collaboration: Neuroscience as Paradigmatic Journal of Macrodynamic Analysis 9 (2016): 74-78.
More information