Critical Appraisal of Scientific Literature. André Valdez, PhD Stanford Health Care Stanford University School of Medicine

Transcription

1 Critical Appraisal of Scientific Literature André Valdez, PhD Stanford Health Care Stanford University School of Medicine

2 Learning Objectives Understand the major components of a scientific article Learn about some of the most common study designs and analyses Become more familiar with literature appraisal tools 2

3 Why Does This Matter? Greater understanding of the literature = better advocacy for EBP Increased literacy on a given clinical topic and gaps in knowledge Increased awareness of a given study s potential and its limitations Grounding of clinical decision making in sound scientific reasoning 3

4 Sections of a Scientific Article Abstract Introduction Objectives/hypothesis Methods Results Discussion/Conclusion Two general types: structured and unstructured Structured contains headings for each major section of a study Unstructured does not contain headings 4

5 Sections of a Scientific Article Introduction Helps the reader become familiar with the topic Refers to recent literature and clearly maps out study s rationale Supports central argument with citations What question(s) does the study attempt to answer? Why is the study design appropriate? (duprel et al. 2009) 5

6 Sections of a Scientific Article Methods Highlights data for appraisal of study s validity Describes all phases of planning, study sample, procedures, and statistical tests Lays out study design, which is paramount Precise description of inclusion/exclusion criteria Thoroughly describes how data were collected, measurement scales of variables, etc. Sample size calculation (duprel et al. 2009) 6

7 Sections of a Scientific Article Results Clear and objective presentation of findings Summary statistics on all variables Inferential statistics on variables of interest Tables and figures to improve clarity (duprel et al. 2009) 7

8 Sections of a Scientific Article Discussion Compares findings with what s currently known What has the study added to the existing body of knowledge? What conclusions may the reader draw? Do the interpretations follow logically from the results? Analyzes study limitations Statistical significance not the same as clinical relevance (duprel et al. 2009) 8

9 Common Study Designs Quality Improvement/Process/Performance Evaluation Set up to enhance clinical practice at a local level (not generalized) Often but not always quasi/non-experimental (no random assignment) Uncontrolled pre-post comparison Controlled pre-post comparison Qualitative experience Cost-effectiveness Human Subject Research Set up to generate new knowledge (generalizable) Often but not always experimental (random assignment) Randomized controlled trial Quasi/non-experimental/observational Case-control Cohort Longitudinal Retrospective chart review (Portela et al. 2015; Topics in Biostatistics 2007) 9

10 Common Statistical Analyses Variable scale: continuous (parametric: assumptions about data distribution) Independent-samples t-test Paired t-test Analysis of Variance Linear regression Variable scale: categorical Chi-square test Logistic regression Variable scale: ordinal (non-parametric: distribution free) Signed-rank test Rank-sum test Kruskal-Wallis test ( 10

11 Literature Appraisal Tools Johns Hopkins Evidence Levels & Quality Guides Evidence Levels Level I Experimental study, randomized controlled trial (RCT) Systematic review of RCTs, with or without meta-analysis Level II Quasi-experimental study Systematic review of a combination of RCTs and quasi-experimental, or quasi-experimental studies only, with or without meta-analysis Level III Non-experimental study Systematic review of a combination of RCTs, quasi-experimental and non-experimental studies, or non-experimental studies only, with or without meta-analysis Qualitative study or systematic review with or without a meta-synthesis Quality Guides A High quality: Consistent, generalizable results; sufficient sample size for the study design; adequate control; definitive conclusions; consistent recommendations based on comprehensive literature review that includes thorough reference to scientific evidence B Good quality: Reasonably consistent results; sufficient sample size for the study design; some control, fairly definitive conclusions; reasonably consistent recommendations based on fairly comprehensive literature review that includes some reference to scientific evidence C Low quality or major flaws: Little evidence with inconsistent results; insufficient sample size for the study design; conclusions cannot be drawn ( 11

12 Literature Appraisal Tools Johns Hopkins Evidence Summary Table Article # Author & Date Evidence Type Sample, Sample Size & Setting Study findings that help answer the EBP question Limitations Evidence Level & Quality ( 12

13 Literature Appraisal Tools STROBE ( 13

14 Literature Appraisal Tools STROBE ( 14

15 Literature Appraisal Tools STROBE ( 15

16 Summary Understanding and appraising scientific literature is crucial for effective EBP Learning about study designs and analyses will help you interpret findings Using appraisal tools is a good way to organize your literature review Critical appraisal promotes critical thinking about your topic 16