Example: Treatment Question

Transcription

1 Example: Treatment Question EVIDENCE TABLE What: Tabular description of the studies. Compare studies by characteristics. Why: Prevent errors of interpretation. Increase clarity of analysis. Plan a statistical analysis. Citation Population Study Design Intervention (N in the Group) Comparison (N in the Group) Follow-Up Period Outcomes Measure Effect Estimate (CI or p) Study Quality Reviewer Comments Study 1 Study 2 Study 3

2 HDA Evidence Table -Therapy Studies Question: Study 1 Study 2 Study 3 Citation Population Study Design Intervention (N in the Group) Comparison (N in the Group) Follow-Up Period Outcomes Measure Effect Estimate (CI or p) Study Quality Reviewer Comments

3 Citation: CRITICAL REVIEW FORM FOR THERAPY STUDY Users Guide: Article: Are the Results Valid? Did experimental and control groups begin the study with a similar prognosis? Were patients randomized? Was randomization concealed? Were patients analyzed in the groups to which they were randomized? Were patients in the treatment and control groups similar with respect to known prognostic factors? Did experimental and control groups retain a similar prognosis after the study started? Were 5 important groups (patients, caregivers, collectors of outcome data, adjudicators of outcome, data analysts) aware of group allocation? Aside from the experimental intervention, were groups treated equally? What are the Results? How large was the treatment effect? How precise was the treatment effect? How can I apply the results to my patient care? Were the study patients similar to my patient? Were all patient- important outcomes considered? Are the likely benefits worth the potential harms and costs? From McMaster EBCP Workshop/Duke University Medical Center

4 CRITICAL REVIEW FORM FOR DIAGNOSIS STUDY Citation: Users Guide: Article: Are the Results Valid? Did the clinicians face diagnostic uncertainty? Was there blind comparison with an independent gold standard? Did the results of the test being evaluated influence the decision to perform the gold standard? What are the Results? What likelihood ratios are associated with the range of possible test results? How can I apply the results to my patient care? Will the reproducibility of the test result and its interpretation is satisfactory in my setting? Are the results applicable to my patient? Will the results change my management? Will patients be better off as a result of the test? Are the benefits worth the potential harms and costs? From McMaster EBCP Workshop/Duke University Medical Center

5 The HelpDesk Search Strategy HelpDesk Answers are intended to provide the same quality response to a clinical question as would be achieved by a search-savvy physician spending an hour or so on the Internet. Authors of HelpDesk Answers are directed to search Healthlinks ( and the TRIP database ( These portals provide access to more than a dozen sources of the highest quality evidencebased clinical information, including BMJ Clinical Evidence, the Guide to Clinical Preventive Services, AHRQ Evidence Reports, and others. Searches of the Cochrane Database, Medline, and other databases, are conducted as needed. Does early insertion of tympanostomy tubes improve developmental outcomes of children with chronic middle ear effusion? Evidence-Based Answer Early placement of tympanostomy tubes for chronic middle ear effusion does not improve developmental outcomes compared with delayed placement in children followed up to 11 years of age. (SOR A, based on a systematic review and RCTs.) A meta-analysis evaluated the effect of tympanostomy tubes on multiple outcomes in children with otitis media with effusion (OME), including the outcome of language and cognitive development. 1 The investigators searched multiple databases for RCTs comparing early insertion of tympanostomy tubes versus delayed insertion or no insertion and found 4 RCTs (841 preschool-aged children) measuring the outcomes of language and cognitive development. Children were included if OME was present for at least 3 to 4 months. Those randomized to early insertion received tubes as soon as possible, whereas children randomized to delayed treatment were given tubes if OME persisted for another 6 to 12 months. In all the studies the developmental assessors were blinded to treatment. In pooled results, 24% of the children in the delayed group had surgery and 15% of the children in the early group did not have surgery. Meta-analysis was performed on the standardized mean differences (SMD) for 3 developmental areas using validated assessment tools. No significant differences were found between the early and delayed groups for language comprehension (SMD 0.09; 95% CI, 0.21 to 0.17), expressive language (SMD 0.02; 95% CI, 0.45 to 0.49), or general development (SMD 0.03; 95% CI, 0.31 to 0.26). Clinical heterogeneity was noted among the studies for populations studied (1 study included children with developmental problems), the time to follow-up assessment (6 22 months), and the developmental tests used. 1 The children in 1 of the RCTs were subsequently reassessed at 6 years of age 2 and 9 to 11 years of age. 3 Before 3 years of age, these children had bilateral OME for more than 90 days or unilateral OME for more than 135 days and were randomized to prompt insertion of tympanostomy tubes or delayed insertion 9 months later if still needed. The original assessments of these children at 3 and 4 years of age were included in the above meta-analysis. Of the 429 children originally enrolled, 395 were reassessed at age 6, 2 and 391 were reassessed at ages 9 to Investigators assessing the patients development were blinded to treatment assignment and patients were analyzed in the groups to which they were randomized. In the 213 children randomized to delayed treatment, 79 had tubes inserted by age 6 and 88 by ages 9 to 11. The cognitive, language, speech, and psychosocial development of the children were assessed using 9 tests at 6 years of age, and their literacy, auditory processing, attention, behavior, social skills, and academic achievement were assessed using 12 tests at 9 to 11 years of age. The studies showed no significant difference in mean developmental test scores between early and delayed insertion of tympanostomy tubes, even though the studies had sufficient power to detect a difference of 0.33 standard deviations on any of the outcome measures. 2,3 Stephanie Hemmer, DO Thomas Satre, MD U of MN/St Cloud Hospital FMR St. Cloud, MN 1. Lous J, Burton MJ, Felding JU, Ovesen T, Rovers MM, Williamson I. Grommets (ventilation tubes) for hearing loss associated with otitis media with effusion in children. Cochrane Database Syst Rev. 2005; (1):CD [LOE 1a ] 2. Paradise JL, Campbell TF, Dollaghan CA, et al. Developmental outcomes after early or delayed insertion of tympanostomy tubes. N Engl J Med. 2005; 353(6): [LOE 1b] 3. Paradise JL, Felmand HM, Campbell TF, et al. Tympanostomy tubes and developmental outcomes at 9 to 11 years of age. N Engl J Med. 2007; 356(3): [LOE 1b] 8 Evidence-Based Practice / May 2010

6 Is cinnamon effective for reducing blood glucose in patients with type 2 diabetes? Evidence-Based Answer Use of Cinnamon cassia, in addition to usual care, may modestly lower blood glucose in patients with type 2 (No SOR, due to apparent conflict between RCTs and meta-analysis results.) Several studies have evaluated the effectiveness of cinnamon in reducing blood glucose in patients with type 2 diabetes mellitus. In a prospective RCT, 60 patients were divided into 6 groups: groups 1 3 received cinnamon at 1, 3, and 6 g/day for 40 days, respectively; groups 4 6 received the corresponding placebo. 1 Baseline fasting blood glucose (FBG) levels ranged from 205 to 300 mg/dl. Patients receiving insulin were excluded from the study. FBG reductions were noted in all 3 active groups, ranging from 18% to 29% (P<.05), compared to no significant differences in the placebo groups. 1 Another RCT included 79 patients with a mean baseline HbA1c of 6.8% and FBG level of 161 mg/dl. 2 Patients continued 1 or more oral antidiabetic medications or diet and received either 1 g aqueous cinnamon extract or placebo 3 times daily with meals for 4 months. Patients using insulin were excluded. A significant reduction was noted in FBG (10.35% ± 13.2%) compared with the placebo group (3.37% ± 14.2%; P<.038). 2 Two other smaller RCTs 3,4 failed to find any effect (TABLE). In the most recent RCT, 109 patients with an HbA1c >7% were randomized to usual care or usual care plus 1 g cinnamon daily. 5 After 90 days, HbA1c levels decreased in the cinnamon-treated group by an absolute 0.83%, compared with 0.37% in the control group (P<.04). 5 In stark contrast, a meta-analysis published as a brief report in 2008 identified 5 clinical trials (n=282) and reported data on FBG and HbA1c levels. 6 One of the trials included adolescents with type 1 (n=57) diabetes, whereas the rest included patients with type 2 diabetes. Subgroup analysis for type 2 trials alone (including the 2 later trials above and 2 negative trials not discussed) revealed that cinnamon was associated with a mean FBG reduction of mg/dl (95% CI, to 13.27) and a HbA1c increase of 0.01% (95% CI, 0.20 to 0.22). The authors concluded that the use of cinnamon did not significantly alter FBG or HbA1c levels. 6 EBP Laura Bowers, PharmD, MBA Robert Gauer, MD Womack FMR Clinic Fort Bragg, NC The opinions and assertions contained herein are the private views of the authors and are not to be construed as official or as reflecting the views of the Medical Department of the US Army or the US Army Service at large. 1. Khan A, et al. Diabetes Care. 2003; 26(12): [LOE 1c] 2. Mang B, et al. Eur J Clin Invest. 2006; 36(5): [LOE 1c] 3. Blevins SM, et al. Diabetes Care. 2007; 30(9): [LOE 1c] 4. Vanschoonbeek K, et al. J Nutr. 2006; 136(4): [LOE 1c] 5. Crawford P. J Am Board Fam Med. 2009; 22(5): [LOE 1b] 6. Baker WL, et al. Diabetes Care. 2008; 31(1): [LOE 1a] TABLE RCTs evaluating the effectiveness of cinnamon use in patients with type 2 diabetes Study design, n Dose Statistically length Reference (g) Results significant Comments RCT, 3 months Crawford, 2009 a Lowered HbA1c Yes External validity Meta-anaysis, Baker et al, No change in FBG or HbA1c No May be underpowered. 3 months No statistical heterogeneity RCT, 3 months Blevins et al, 2007 a No change in FBG or HbA1c No Many exclusions. Baseline HbA1c near goal RCT, 4 months Mang et al, Reduced FBG by 10.4%. Strong evidence. Moderate effect No effect on HbA1c Yes on FBG RCT, 6 weeks Vanschoonbeek et al, No change in FBG, insulin, HbA1c, No Good evidence. No men included or oral glucose tolerance Short study duration RCT, 40 days Khan et al, , 3, Reduced FBG by 18% 29%; Yes Strong effect on FBG or 6 did not measure HbA1c a RCTs conducted in US patients with type 2 diabetes. FBG=fasting blood glucose; HbA1c=glycosylated hemoglobin; RCT=randomized controlled trial. 12 Evidence-Based Practice / June 2010

7 DETERMINING LOEs AND SORs Level 1a Using the Center for Evidence Based Medicine (CEBM) table below, locate the articles type of study in the column headings. Within that column, identify the specifics of the study to determine the Level of Evidence (LOE.) Each article referenced will have its own LOE. Oxford Centre for Evidence based Medicine Levels of Evidence (March 2009) (See notes and glossary on the following pages.) Therapy / Prevention, etiology / Harm SR (with homogeneity*) of RCTs Prognosis inception cohort studies; CDR validated in different populations Diagnosis Level 1 diagnostic studies; CDR with 1b studies from different clinical centres Differential diagnosis / symptom prevalence study prospective cohort studies Economic and decision analyses Level 1 economic studies 1b Individual RCT (with narrow Confidence Interval ) Individual inception cohort study Validating** cohort study with > 80% follow-up; CDR with good reference validated in a single population standards; or CDR tested within one clinical centre Prospective cohort study with good follow-up**** Analysis based on clinically sensible costs or alternatives; systematic review(s) of the evidence; and including multiway sensitivity analyses 1c All or none All or none case-series Absolute SpPins and SnNouts All or none case-series Absolute better-value or worse-value analyses 2a SR (with homogeneity*) of cohort studies either retrospective cohort studies or untreated control groups in RCTs Level >2 diagnostic studies 2b and better studies Level >2 economic studies 2b Individual cohort study (including low quality RCT; e.g., <80% followup) Retrospective cohort study or follow-up of untreated control patients in an RCT; Derivation of CDR or validated on splitsample only Exploratory** cohort study Retrospective cohort with good reference study, or poor follow-up standards; CDR after derivation, or validated only on split-sample or databases Analysis based on clinically sensible costs or alternatives; limited review(s) of the evidence, or single studies; and including multi-way sensitivity analyses 2c 3a "Outcomes" Research; Ecological studies SR (with homogeneity*) of case-control studies "Outcomes" Research Ecological studies Audit or outcomes research 3b and better studies 3b and better studies 3b and better studies 3b 4 Individual Case-Control Study Case-series (and poor quality cohort and casecontrol studies ) Case-series (and poor quality prognostic cohort studies***) Non-consecutive study; or Non-consecutive cohort without consistently applied study, or very limited reference standards population Case-control study, poor or Case-series or non-independent reference superseded reference standard standards Analysis based on limited alternatives or costs, poor quality estimates of data, but including sensitivity analyses incorporating clinically sensible variations. Analysis with no sensitivity analysis 5 Expert opinion without explicit critical appraisal, or based on physiology, bench research or "first principles" Expert opinion without explicit critical appraisal, or based on physiology, bench research or "first principles" Expert opinion without explicit critical appraisal, or based on physiology, bench research or "first principles" Expert opinion without explicit critical appraisal, or based on physiology, bench research or "first principles" Expert opinion without explicit critical appraisal, or based on economic theory or "first principles" This table was produced by Bob Phillips, Chris Ball, Dave Sackett, Doug Badenoch, Sharon Straus, Brian Haynes, Martin Dawes since November Updated by Jeremy Howick, in March HDA Document # 205 Page 1 of

8 Notes to CEBM table of Medicine Levels of Evidence (previous page): * By homogeneity we mean a systematic review that is free of worrisome variations (heterogeneity) in the directions and degrees of results between individual studies. Not all systematic reviews with statistically significant heterogeneity need be worrisome, and not all worrisome heterogeneity need be statistically significant. As noted above, studies displaying worrisome heterogeneity should be tagged with a " " at the end of their designated level. Clinical Decision Rule. (These are algorithms or scoring systems that lead to a prognostic estimation or a diagnostic category.) See note above for advice on how to understand, rate and use trials or other studies with wide confidence intervals. Met when all patients died before the Rx became available, but some now survive on it; or when some patients died before the Rx became available, but none now die on it. By poor quality cohort study we mean one that failed to clearly define comparison groups and/or failed to measure exposures and outcomes in the same (preferably blinded), objective way in both exposed and nonexposed individuals and/or failed to identify or appropriately control known confounders and/or failed to carry out a sufficiently long and complete follow up of patients. By poor quality case control study we mean one that failed to clearly define comparison groups and/or failed to measure exposures and outcomes in the same (preferably blinded), objective way in both cases and controls and/or failed to identify or appropriately control known confounders. Split sample validation is achieved by collecting all the information in a single tranche, then artificially dividing this into "derivation" and "validation" samples. ** *** An "Absolute SpPin" is a diagnostic finding whose Specificity is so high that a Positive result rules in the diagnosis. An "Absolute SnNout" is a diagnostic finding whose Sensitivity is so high that a Negative result rules out the diagnosis. Good, better, bad and worse refer to the comparisons between treatments in terms of their clinical risks and benefits. Good reference standards are independent of the test, and applied blindly or objectively to applied to all patients. Poor reference standards are haphazardly applied, but still independent of the test. Use of a nonindependent reference standard (where the 'test' is included in the 'reference', or where the 'testing' affects the 'reference') implies a level 4 study. Better value treatments are clearly as good but cheaper, or better at the same or reduced cost. Worse value treatments are as good and more expensive, or worse and the equally or more expensive. Validating studies test the quality of a specific diagnostic test, based on prior evidence. An exploratory study collects information and trawls the data (e.g. using a regression analysis) to find which factors are 'significant'. By poor quality prognostic cohort study we mean one in which sampling was biased in favor of patients who already had the target outcome, or the measurement of outcomes was accomplished in <80% of study patients, or outcomes were determined in an unblinded, non objective way, or there was no correction for confounding factors. **** Good follow up in a differential diagnosis study is >80%, with adequate time for alternative diagnoses to emerge (for example 1 6 months acute, 1 5 years chronic) HDA Document # 205 Page 2 of

9 Strength of Recommendation Taxonomy The Strength of Recommendation is determined based upon the LOEs of the articles used to determine the Evidence Based Answer. This is also described as Grading the Evidence. accessed on Strength of recommendation A B C SORT GRADES Definition Recommendation based on consistent and good quality patient oriented evidence.* Recommendation based on inconsistent or limited quality patient oriented evidence.* Recommendation based on consensus, usual practice, opinion, disease oriented evidence,* or case series for studies of diagnosis, treatment, prevention, or screening Consistent Inconsistent Definition of 'Consistency across studies' Most studies found similar or at least coherent conclusions (coherence means that differences are explainable); or If high quality and up to date systematic reviews or meta analyses exist, they support the recommendation Considerable variation among study findings and lack of coherence; or If high quality and up to date systematic reviews or meta analyses exist, they do not find consistent evidence in favor of the recommendation *Patient-oriented evidence measures outcomes that matter to patients: morbidity, mortality, symptom improvement, cost reduction, and quality of life. Disease-oriented evidence measures intermediate, physiologic, or surrogate end points that may or may not reflect improvements in patient outcomes (ie, blood pressure, blood chemistry, physiologic function, and pathologic findings). For a more detailed table and examples, please refer to The flow chart on the following pages is extracted from an example on the JFP online web page. HDA Document # 205 Page 3 of

10 CHART A HDA Document # 205 Page 4 of

11 CHART B Note: The JFP SORT Taxonomy is easily applied by authors and physicians, and explicitly addresses the issue of patient oriented versus disease oriented evidence. These SORs also create some limitations. On the JFP website, you can read about the walkover between the two methods of grading the evidence. HDA Document # 205 Page 5 of