Συστηματικές ανασκοπήσεις Μετα-αναλύσεις. Β Κώτσης Επίκουρος Καθηγητής Παθολογίας ΑΠΘ

Transcription

1 Συστηματικές ανασκοπήσεις Μετα-αναλύσεις Β Κώτσης Επίκουρος Καθηγητής Παθολογίας ΑΠΘ

2 Systema>c reviews of health promo>on and public health interven>ons

3 Overview Overview of systemaec reviews Outline of The Cochrane CollaboraEon FuncEon of systemaec reviews in informing policy and pracece

4 Types of reviews Reviews (narrative/literature/ traditional) Systematic reviews Meta-analysis

5 NarraEve reviews Usually writen by experts in the field Use informal and subjeceve methods to collect and interpret informaeon Usually narraeve summaries of the evidence Read: Klassen et al. Guides for Reading and Interpreting Systematic Reviews. Arch Pediatr Adolesc Med 1998;152:

6 What is a systema>c review? A review of the evidence on a clearly formulated queseon that uses systemaec and explicit methods to idenefy, select and criecally appraise relevant primary research, and to extract and analyse data from the studies that are included in the review* *Undertaking Systematic Reviews of Research on Effectiveness. CRD s Guidance for those Carrying Out or Commissioning Reviews. CRD Report Number 4 (2nd Edition). NHS Centre for Reviews and Dissemination, University of York. March 2001.

7 Key elements of a systema>c review Structured, systemaec process involving several steps : 1. Formulate the queseon 2. Plan the review 3. Comprehensive search 4. Unbiased seleceon and abstraceon process 5. CriEcal appraisal of data 6. Synthesis of data (may include meta-analysis) 7. InterpretaEon of results All steps described explicitly in the review

8 Systema>c vs. Narra>ve reviews ScienEfic approach to a review arecle Criteria determined at outset Comprehensive search for relevant arecles Explicit methods of appraisal and synthesis Meta-analysis may be used to combine data Depend on authors inclinaeon (bias) Author gets to pick any criteria Search any databases Methods not usually specified Vote count or narraeve summary Can t replicate review

9 Advantages of systema>c reviews Reduce bias Replicability Resolve controversy between confliceng studies IdenEfy gaps in current research Provide reliable basis for decision making

10 Limita>ons of systema>c reviews specific to health promo>on Results may sell be inconclusive There may be no trials/evidence The trials may be of poor quality The interveneon may be too complex to be tested by a trial PracEce does not change just because you have the evidence of effect/effeceveness

11 Consider these interven>ons IntervenEons to promote smoke alarm ownership and funceon School-based driver educaeon for the preveneon of traffic crashes Helmets for preveneng head and facial injuries in bicyclists Do you think the results idenefied in SRs will be good, promising or absent (and poteneally harmful)?

12 Results from systema>c reviews Helmets reduce bicycle-related head and facial injuries for bicyclists of all ages involved in all types of crashes including those involving motor vehicles. The results provide no evidence that drive educa>on reduces road crash involvement, and suggest that it may lead to a modest but poteneally important increase in the proporeon of teenagers involved in traffic crashes. Results from this review suggest that area-wide traffic calming in towns and ci>es may be a promising interven>on for reducing the number of road traffic injuries and deaths. However, further rigorous evaluaeons of this interveneon are needed.

13 The Cochrane Collabora>on Interna>onal non-profit organisa>on that prepares, maintains, and disseminates systema>c up-to-date reviews of health care interven>ons

14 Cochrane Collabora>on Named in honour of Archie Cochrane, a BriEsh researcher In 1979: It is surely a great criticism of our profession that we have not organised a critical summary, by specialty or subspecialty, adapted periodically, of all relevant randomised controlled trials

15 The Cochrane Library Cochrane Systema>c reviews : Cochrane reviews and protocols Database of Reviews of Effects: Other systemaec reviews appraised by the Centre for Reviews and DisseminaEon. Cochrane Central Register of Controlled Trials: Bibliography of controlled trials (some not indexed in MEDLINE). Health Technology Assessment Database: HTA reports NHS Economic evalua>on database: Economic evaluaeons of health care interveneons.

16 The Cochrane Library

17

18 Cochrane CollaboraEon Structure Steering Group Review Groups Centres Fields Consumer Network Methods Groups

19 CollaboraEve Review Groups (50) Produce systemaec reviews relevant to a parecular disease or health issue for inclusion in the Cochrane Library Examples Airways Group Drug and Alcohol Group Heart Group Injuries Group Skin Group Pregnancy and Childbirth Group Stroke Group Breast Cancer Group

20 Methods Groups (12) Provide advice and support in the development of the methods of systemaec reviews Examples Non-Randomised Studies Screening and DiagnosEc Tests Empirical Methodological Studies QualitaEve Methods

21 Cochrane Centres (14) Work to assist all Cochrane eneees within a specific geographical area Examples Australasian Cochrane Centre (at Monash) South African Cochrane Centre Italian Cochrane Centre Chinese Cochrane Centre

22 Cochrane Fields/Networks (9) Represent an area of interest which spans a number of health problems - and hence a number of Review Groups Examples Health PromoEon and Public Health Field Primary Health Care Field Cancer Network Child Health Field

23 Asking an answerable ques>on

24 Ques>ons of interest EffecEveness: Does the interveneon work/not work? Who does it work/not work for? Other important queseons: How does the interveneon work? Is the interveneon appropriate? Is the interveneon feasible? Is the interveneon and comparison relevant?

25 Answerable ques>ons EFFECTIVENESS A descripeon of the populaeons P An idenefied interveneon I An explicit comparison C Relevant outcomes O

26 A PICO ques>on Time-consuming ques>on: What is the best strategy to prevent smoking in young people?

27 An answerable ques>on Q. Are mass media (or school-based or communitybased) interveneons effeceve in preveneng smoking in young people?

28 The PICO(T) chart Problem, population Intervention Comparison Outcome Types of studies Young people under 25 years of age a) Television b) Radio c) Newspapers d) Bill boards e) Posters f) Leaflets g) Booklets a) School-based interventions b) No intervention a) objective measures of smoking (saliva thiocyanate levels, alveolar CO) b) self-reported smoking behaviour a) RCT b) Controlled before and after studies c) Time series designs c) Intermediate measures (intentions, attitude, knowledge, skills) d) Media reach

29 Systema>c review process 1. Well formulated queseon 2. Comprehensive data search 3. Unbiased seleceon and abstraceon process 4. CriEcal appraisal of data 5. Synthesis of data 6. InterpretaEon of results

30 A good search ü Clear research queseon ü Comprehensive search All domains, no language restriceon, unpublished and published literature, up-to-date ü Document the search (replicability)

31 Components of electronic searching 1. Describe each PICO component 2. Start with primary concept 3. Find synonyms a) IdenEfy MeSH / descriptors / subject headings b) Add textwords 4. Add other components of PICO queseon to narrow citaeons (may use study filter) 5. Examine abstracts 6. Use search strategy in other databases (may need adapeng)

32 So you want to do a quick & dirty? DARE CENTRAL PubMed (clinical queries, related records) CDC NICE OrganisaEons who do work in your area google

33 The Cochrane Library:

34 Cochrane HPPH Field

35 Health-evidence.ca

36 The Guide to Community Preven>ve Services htp://

37 Na>onal Ins>tute for Health and Clinical Excellence htp://

38 Evidence for Policy and Prac>ce Informa>on and Coordina>ng Centre (EPPI-Centre) htp://eppi.ioe.ac.uk

39 Effec>ve Public Health Prac>ce Project (EPHPP) htp:// CityandGovernment/HealthandSocialServices/ Research/EPHPP/

40 Centre for Reviews and DisseminaEon htp://

41 Other sources of primary research

42 Searching The order of terms will effect the results so start with the obvious or key concept No need for and Google will ignore common words If they are important use + (e.g. policy + 3) Phrase searching is useful eg suicide preveneon Google searches for variaeons on words eg diet, dietary

43 Searching Where terms have muleple meanings you can direct google to remove sites you want to avoid (e.g. bass music) Keep your search strings brief Mental health promoeon inieaeves to prevent suicide in young people Compartmentalise your search strings mental health promoeon suicide Suicide preveneon young people PrevenEon and suicide and youth

44 Searching htp:// Use the same principles for google keep it short and sweet. Key features Journals Database MeSH Database Single CitaEon Matcher Clinical Queries

45 Searching

46 Searching

47 Searching Select the Limits tab just under the search string

48 Searching

49 These next few slides show you how to search MeSH terms in PubMed. Useful if you don t have access to electronic databases. You combine this method with the one for text words outlined above. The process for combining text words and MeSH terms is outlines below.

50

51

52

53 Example Mass media interveneons to prevent smoking in young people P= Young people STEP ONE: Find MeSH and textwords to describe young people

54 Example Mass media interveneons to prevent smoking in young people P= Young people MeSH: Adolescent Child Minors

55 Example Mass media interveneons to prevent smoking in young people P= Young people Textwords: Adolescent Girl Child Boy Juvenile Teenager Young people Young adult Student Youth

56 Textwords Trunca>on $: To pick up various forms of a word Teen$.tw Smok$.tw Teenage Smoke Teenager Smoking Teenagers Teens Teen Smoker Smokes Smokers

57 Textwords Wild cards? and #: To pick up different spellings Colo?r.tw (? Can be subs>tuted for one or no characters) Colour Color Wom#n.tw (# Subs>tutes for one character) Woman Women

58 Textwords Adjacent ADJn: retrieves two or more query terms within n words of each other, and in any order Great when you are not sure of phraseology Eg sport adj1 policy Sport policy Policy for sport Eg mental adj2 health Mental health Mental and physical health

59 Example con>nued Mass media interveneons to prevent smoking in young people I = Mass media interven>ons STEP TWO: Find MeSH and textwords to describe mass media interventions

60 Example con>nued MeSH Mass media Audiovisual aids Television MoEon pictures Radio TelecommunicaEons Newspapers Videotape recording AdverEsing

61 Example con>nued Mass media interveneons to prevent smoking in young people O = Preven>on of smoking STEP THREE: Find MeSH and textwords to describe prevention of smoking

62 Example of search P = YOUNG PEOPLE MeSH Textwords.. OR.... I = MASS MEDIA MeSH Textwords... OR... C = (if required) O = PREVENTION OF SMOKING MeSH Textwords. OR..... P AND I AND C AND O

63 Different bibliographic databases Databases use different types of controlled vocabulary Same citaeons indexed differently on different databases Medline and EMBASE use a different indexing system for study type PsycINFO and ERIC do not have specific terms to idenefy study types Need to develop search strategy for each database

64 htp:// Cinahl_eb_filters.pdf SystemaEc reviews/meta-analyses CINAHL: as above Medline htp:// OVID_eb_filters.pdf Medline and Embase htp:// PubMed Study design filters RCTs See Cochrane Reviewer s Handbook Non-RCTs Not yet developed, research in progress QualitaEve research Specific subject headings used in CINAHL, qualitaeve research used in Medline CINAHL Filter: Edward Miner Library

65 2. Unpublished literature Not all known published trials are idenefiable in Medline (depending on topic) Only 25% of all medical journals in Medline Non-English language arecles are underrepresented in Medline (and developing countries) PublicaEon bias tendency for invesegators to submit manuscripts and of editors to accept them, based on strength and direceon of results (Olsen 2001)

66 2. Unpublished literature Hand searching of key journals and conference proceedings Scanning bibliographies/reference lists of primary studies and reviews ContacEng individuals/agencies/ academic insetueons NeglecEng certain sources may result in reviews being biased

67 Librarians are your friends!

68 Principles of cri>cal appraisal

69 Cri>cal appraisal The process of systema>cally examining research evidence to assess its validity, results and relevance before using it to inform a decision. Alison Hill, CriEcal Appraisal Skills Programme, InsEtute of Health Sciences, Oxford htp://

70 Cri>cal appraisal I: Quan>ta>ve studies

71 Why appraise validity? Not all published and unpublished literature is of saesfactory methodological rigour Just because it is in a journal does not mean it is sound! Onus is on you to assess validity! Quality may be used as an explanaeon for differences in study results Guide the interpretaeon of findings and aid in determining the strength of inferences

72 Bias quality assessment tool 1. SelecEon bias 2. AllocaEon bias 3. Confounding 4. Blinding (deteceon bias) 5. Data colleceon methods 6. Withdrawals and drop-outs 7. StaEsEcal analysis 8. IntervenEon integrity

73 SelecEon bias Recruit parecipants IntervenEon Exposed to interveneon AllocaEon Confounding Integrity of interveneon AllocaEon of concealment Control Not exposed to interveneon Follow-up Outcome Analysis IntenEon-to-treat Withdrawals Blinding of outcome assessment Data colleceon methods StaEsEcal analysis Follow-up Outcome Analysis

74 CriEcal appraisal tools RCTs The Quality Assessment Tool for QuanEtaEve Studies ( htp:// Non-RCTs Cochrane EffecEve PracEce and OrganisaEon of Care Group (htp:// The Berkeley SystemaEc Reviews Group (htp://

75 Cri>cal appraisal II: Qualita>ve studies

76 Qualita>ve research explores the subjeceve world. It atempts to understand why people behave the way they do and what meaning experiences have for people. QualitaEve studies of experience Process evaluaeon Undertaking Systematic Reviews of Research on Effectiveness. CRD s Guidance for those Carrying Out or Commissioning Reviews. CRD Report Number 4 (2nd Edition). NHS Centre for Reviews and Dissemination, University of York. March 2001.

77 CASP appraisal checklist 1. Clear aims of research (goals, why it is important, relevance) 2. Appropriate methodology (what, how, why) 3. Sampling strategy 4. Data colleceon 5. RelaEonship between researcher and parecipants 6. Ethical issues 7. Data analysis 8. Findings 9. Value of research (context dependent)

78 Other qualita>ve checklist Quality framework Government Chief Social Researcher s Office, UK htp:// Quality_framework.pdf 19 queseon checklist for process evaluaeons (EPPI-Centre)

79 Appraisal of a systema>c review 10 queseons 1. Clearly-focused queseon 2. The right type of study included 3. IdenEfying all relevant studies 4. Assessment of quality of studies 5. Reasonable to combine studies 6. What were the results 7. Preciseness of results 8. ApplicaEon of results to local populaeon 9. ConsideraEon of all outcomes 10. Policy or pracece change as a result of evidence CASP

80 Interpreta>on of results

81 Factors influencing effec>veness Type III error (integrity of interveneon) TheoreEcal framework of interveneon Context For whom did the interveneon work, why, in what circumstances, at what cost

82 Difficul>es addressing inequali>es Studies rarely present informaeon on differeneal effects of interveneons Cannot locate studies addressing inequaliees May need original data from authors Low power to detect subgroup differences

83 Integra>on of evidence into prac>ce

84 Assessing the applicability and transferability of interveneons Applicability whether the interveneon process could be implemented in the local sežng, no mater what the outcome is. Is it possible to run this interveneon in this local sežng? Eg. provision of condoms in area where they are not acceptable for religious reasons Wang et al 2005

85 Assessing the applicability and transferability of interveneons Transferability if the interveneon were to be implemented in the local sežng, would the effeceveness of the program be similar to the level detected in the study sežng? E.g. if the interveneonists lack experience and have few skills in delivering the interveneon then its effeceveness in the local sežng may be lower than that demonstrated in the study sežng Wang et al 2005

86 IntroducEon to Meta-Analysis

87 ObjecEves Define Meta-Analysis Strengths and LimitaEons of Meta Analysis How to Conduct a Meta Analysis Asking a QuesEon Searching, EvaluaEng and Coding Studies Synthesizing with StaEsEcal Methods

88 What is meta-analysis? A staesecal analysis that combines or integrates the results of several independent clinical trials considered by the analyst to be combinable ASA, 1988 Meta-analysis clearly has advantages over conveneonal narraeve reviews and carries considerable promise as a tool in clinical research Eggers, Davey Smith, 1997 Νow widely accepted as a method of summarizing the results of empirical studies within the behavioral social and health sciences Lipsey and Wilson, 2000

89 Meta-analysis publicaeons

90

91 DefiniEon A set of methods to systemaecally and reproducibly search, sample and (staesecally) synthesize evidence from studies.

92 Strengths of Meta-Analysis Imposes Discipline Makes process explicit and systemaec Organized way of combining a lot of informaeon More differeneated and sophisecated than tradieonal reviews Combining studies increases power Find significant results

93 Weaknesses of Meta-Analysis Heterogeneity apples and oranges Biases Missing Studies May differ from published studies (publicaeon bias) Quality of Studies (GIGO) What consetutes quality? Requires a lot of effort and substance-area experese Mechanics of staesecs may obscure theory Good for closed-ended queseons only

94 Meta-Analysis: A 12-Step Program I. Specify Problem II. Search for and IdenEfy Studies III. Enter studies into database IV. Select Studies for Review V. Review Studies VI. Develop Coding Scheme VII. Abstract / Code Studies VIII. Select Effect Size StaEsEc IX. Transform and Weight Effect Sizes X. Assess heterogeneity XI. Assess Bias XII. Synthesize and Present Results

95 ????? I. Problem SpecificaEon

96 II. IdenEfy Studies Has a meta-analysis been done already? PubMed Clinical Queries Electronic / Online Resources PubMed, Medline, CumulaEve Index of Nursing and Allied Health Literature (CINAHL), Allied and Complementary Medicine (AMED), PsychINFO, Health and Psychosocial Instruments, ProQuest Digital DissertaEon Database, Papers First, Cochrane Reviews, ACP Journal Club, Database of Abstracts of Reviews of EffecEveness (DARE), the Cochrane Controlled Trials Register (CCTR), Sociologic Abstracts and Web of Science Search Terms Hand Search References of electronically idenefied arecles Contact invesegators

97 III. Enter Studies Endnote / Zotero / Bookends Search and enter directly through program Remove duplicates Review Etles and remove non-relevant studies Review abstracts and remove non-relevant studies Retrieve pdf s ATach to citaeon Review full text Divide into main categories

98 IV. / V. Review and Select Studies

99 VI. Develop Coding Scheme Interviewing a Study Conceptually Effect Size (outcome) and populaeons, methods, procedures, designs study methods, designs, procedures (explanatory) Examples of Data Elements IdenEfying informaeon (authors, journal, etc!) Sample size Type of populaeon Type of effect (e.g. odds raeo, prevalence) Result

100 Kinds of Effect StaEsEcs ProporEon Central tendencies Standardized mean difference (d) Group contrasts of conenuous measures CorrelaEon coefficient (r) Linear associaeons Odds-raEo Group contrasts of dichotomous measures

101 IX. Transform and Weight the Effect StaEsEc Transform magnitude and direceon of the effect same scale for all studies Weight inverse variance gives more weight to larger studies sample size is key ("n = "precision) standard error a. means, correlaeons, proporeons, odds - not well-suited to complex procedures like muleple logisec regression

102 Why Weight Effect Sizes? Studies vary in size. ES based on 100 subjects assumed more precise esemate of populaeon ES than ES based on 10 subjects. Larger studies should carry more weight. WeighEng by the inverse variance opbmal approach

103 An example of not weigheng!

104 Why weight by inverse variance? The standard error (SE) is a direct index of ES precision. influenced by sample size used to create confidence intervals. The smaller the SE, the more precise the ES. Op>mal weights for meta-analysis (Hedges):

105 Transforming a Con-nuous Measure: The Standardized Mean Difference (d or g) standardized group contrast on an conenuous measure uses a pooled standard deviaeon

106 WeighBng a ConBnuous Measure SE for a standardized mean difference: Weight for standardized mean difference:

107 Methods for Dichotomous Effect Sizes Variance-Based Method transform and work on log scale Lipsey and Wilson, CMA AKA Inverse Variance Methods Can be applied to OR s, RR s, RD s Can be applied when don t have complete 2x2 table info Mantel-Haenszel work on original scale Cochrane, PeEŽ, R packages Long history of experience StaEsEcally opemal Fixed Effects vs. Peto Method DerSimonian and Laird Cochrane, R packages Random Effects

108 Variance Method for Dichotomous Outcomes 1. Transform the result to the log scale 2. Weight the result by inverse variance 3. Calculate the weighted mean effect size 4. Calculate the standard error of the weighted mean effect size 5. Calculate a confidence interval for the weighted mean effect size

109 Variance Method for Odds RaEos

110 2. Weight the Logit s.e. for a Logit Weight for a Logit

111 3. Calculate the Weighted (Mean) Odds RaEo

112

113

114

115

116

117

118

119

120

121

122

123

124

125

126

127

128

129

130