Effective implementation of guidelines requires 3 interrelated processes:

Similar documents
Washington, DC, November 9, 2009 Institute of Medicine

Pharmaceutical Poetry 2018: Recent Updates in Clinical Research for the Practicing Pharmacist

Evaluating the Strength of Clinical Recommendations in the Medical Literature: GRADE, SORT, and AGREE

Essential Skills for Evidence-based Practice Understanding and Using Systematic Reviews

Guideline development in TB diagnostics. Karen R Steingart, MD, MPH McGill University, Montreal, July 2011

GRADE. Grading of Recommendations Assessment, Development and Evaluation. British Association of Dermatologists April 2014

Systematic reviews: From evidence to recommendation. Marcel Dijkers, PhD, FACRM Icahn School of Medicine at Mount Sinai

GRADE. Grading of Recommendations Assessment, Development and Evaluation. British Association of Dermatologists April 2018

ACR OA Guideline Development Process Knee and Hip

Clinical Policy: Fractional Exhaled Nitric Oxide Reference Number: CP.MP.103

Evidence- and Value-based Solutions for Health Care Clinical Improvement Consults, Content Development, Training & Seminars, Tools

Critical appraisal: Systematic Review & Meta-analysis

Evidence-based medicine and guidelines: development and implementation into practice

Overview and Comparisons of Risk of Bias and Strength of Evidence Assessment Tools: Opportunities and Challenges of Application in Developing DRIs

Appraising the Literature Overview of Study Designs

SSAI Clinical Practice Committee guideline work flow v2. A. Formal matters

Introduzione al metodo GRADE

Overview of Study Designs in Clinical Research

SkillBuilder Shortcut: Levels of Evidence

Surveillance report Published: 6 April 2016 nice.org.uk. NICE All rights reserved.

GRADE tables to assist guideline development and recommendations. Plain Language Summary of Results

The QUOROM Statement: revised recommendations for improving the quality of reports of systematic reviews

APPLYING EVIDENCE-BASED METHODS IN PSYCHIATRY JOURNAL CLUB: HOW TO READ & CRITIQUE ARTICLES

10801 Sixth St, Rancho Cucamonga, CA Tel (909) Fax (909) Visit our web site at:

GRADE Evidence Profiles on Long- and Rapid-Acting Insulin Analogues for the treatment of Diabetes Mellitus [DRAFT] October 2007

Critical Appraisal of a Meta-Analysis: Rosiglitazone and CV Death. Debra Moy Faculty of Pharmacy University of Toronto

Cost-effectiveness analysis of budesonide aqueous nasal spray and fluticasone propionate nasal spray in the treatment of perennial allergic rhinitis

Int. J. Pharm. Sci. Rev. Res., 36(1), January February 2016; Article No. 06, Pages: JNC 8 versus JNC 7 Understanding the Evidences

The National Asthma Education and Prevention Program s

EVIDENCE AND RECOMMENDATION GRADING IN GUIDELINES. A short history. Cluzeau Senior Advisor NICE International. G-I-N, Lisbon 2 November 2009

The problem with critical and non-critical inhaler errors

TORCH: Salmeterol and Fluticasone Propionate and Survival in COPD

SYSTEMATIC REVIEW: AN APPROACH FOR TRANSPARENT RESEARCH SYNTHESIS

CHAPTER 5. Weekly self-monitoring and treatment adjustment benefit patients with partly controlled and uncontrolled asthma

PGY1 Learning activities-ebcp Scripts

Objectives. Information proliferation. Guidelines: Evidence or Expert opinion or???? 21/01/2017. Evidence-based clinical decisions

Introduction to systematic reviews/metaanalysis

CRITICAL APPRAISAL OF CLINICAL PRACTICE GUIDELINE (CPG)

Controversies in Clinical Trials

1. Draft checklist for judging on quality of animal studies (Van der Worp et al., 2010)

The methodology behind GINA and EPR-3 medication recommendations: Stepwise treatment in asthma

2. The effectiveness of combined androgen blockade versus monotherapy.

Transparency and accuracy in reporting health research

MINI SYMPOSIUM - EUMASS - UEMASS European Union of Medicine in Assurance and Social Security

Live WebEx meeting agenda

Surveillance report Published: 13 April 2017 nice.org.uk. NICE All rights reserved.

U.S. Preventive Services Task Force Methods and Processes. Alex R. Kemper, MD, MPH, MS June 16, 2014

Varenicline and cardiovascular and neuropsychiatric events: Do Benefits outweigh risks?

Are the likely benefits worth the potential harms and costs? From McMaster EBCP Workshop/Duke University Medical Center

Case-Compare Impact Report

Evaluation of Asthma Management in Middle EAst North Africa Adult population

CRITICAL APPRAISAL WORKSHEET 1

Systematic Reviews and Meta- Analysis in Kidney Transplantation

The Joanna Briggs Institute Reviewers Manual 2014

USDA Nutrition Evidence Library: Systematic Review Methodology

#1 cause of school absenteeism in children 13 million missed days annually

Reporting the effects of an intervention in EPOC reviews. Version: 21 June 2018 Cochrane Effective Practice and Organisation of Care Group

Mapping from SORT to GRADE. Brian S. Alper, MD, MSPH, FAAFP Editor-in-Chief, DynaMed October 31, 2013

W e have shown in a previous meta-analysis of placebo

NG80. Asthma: diagnosis, monitoring and chronic asthma management (NG80)

Interventions to improve adherence to inhaled steroids for asthma. Respiratory department

Do current treatment protocols adequately prevent airway remodeling in children with mild intermittent asthma?

Improving reporting for observational studies: STROBE statement

Critical Appraisal Practicum. Fabio Di Bello Medical Implementation Manager

Using Number Needed to Treat to Interpret Treatment Effect

Q: Should patients with mild asthma

Systematic reviews and meta-analyses of observational studies (MOOSE): Checklist.

Evidence-Based Medicine and Publication Bias Desmond Thompson Merck & Co.

Which asthma patient has allergic airway inflammation?

2018 American Academy of Neurology

Clinical efficacy of montelukast in anti-inflammatory treatment of asthma and allergic rhinitis

Accepted refereed manuscript of:

Identifying Biologic Targets to Attenuate or Eliminate Asthma Exacerbations

Trials and Tribulations of Systematic Reviews and Meta-Analyses

In 2002, it was reported that 72 of 1000

Suffolk PCT Drug & Therapeutics Committee New Medicine Report (Adopted by the CCG until review and further notice)

2014 International Pressure Ulcer Guideline Methodology

Guidelines for Writing and Reviewing an Informed Consent Manuscript From the Editors of Clinical Research in Practice: The Journal of Team Hippocrates

Outline. What is Evidence-Based Practice? EVIDENCE-BASED PRACTICE. What EBP is Not:

GRADE, Summary of Findings and ConQual Workshop

Online supplementary material

Setting The setting was secondary care. The economic study was carried out in France.

Clinical Guidelines And Primary Care

The SPIRIT Initiative: Defining standard protocol items

Module 5. The Epidemiological Basis of Randomised Controlled Trials. Landon Myer School of Public Health & Family Medicine, University of Cape Town

Study selection Study designs of evaluations included in the review Diagnosis.

POEMS Patient Oriented Evidence that Matters

Month/Year of Review: March 2014 Date of Last Review: April 2012

Item Number: 6 NHS VALE OF YORK CLINICAL COMMISSIONING GROUP GOVERNING BODY MEETING. Meeting Date: 7 November Report Author: Report Sponsor:

An evidence-based laboratory medicine approach to evaluate new laboratory tests

Current Asthma Management: Opportunities for a Nutrition-Based Intervention

Literature Scan: Analgesics for Gout. Month/Year of Review: April 2015 Date of Last Review: January 2014

CADET: Clinical & Cost Effectiveness of Collaborative Care for Depression in UK Primary Care: A Cluster Randomized Controlled Trial

Is reslizumab effective in improving quality of life and asthma control in adolescent and adult patients with poorly controlled eosinophilic asthma?

Standard Methods for Quality Assessment of Evidence

Evidence Briefing for NHS Bradford and Airedale. Physical health monitoring for people with schizophrenia or other serious mental illness

POEMs P ATIENT O RIENTED E VIDENCE THAT M ATTERS. Statins prevent strokes in high-risk patients. What is a POEM?

Evidence-based recommendations or Show me the patients selected and I will tell you the results

Transcription:

Importance of Evidence Grading for Guideline Implementation: The Example of Asthma David L. Hahn, MD, MS University of Wisconsin School of Medicine and Public Health, Madison, Wisconsin ABSTRACT The goal of evidence-based clinical guidelines is to improve the value of health care by recommending treatments with favorable benefit/harm ratios. Achieving this goal requires use of evidence-grading systems that explicitly address strength of evidence in terms of external validity (generalizability), internal validity, and patient-oriented outcomes. To be clinically useful, guidelines should also incorporate patient preferences, particularly when evidence is weak. The National Heart, Lung and Blood Institute recently published Expert Panel Report 3: Guidelines for the Diagnosis and Management of Asthma (EPR-3). This special report addresses the extent to which current guidelines adhere to the principles enunciated above by using EPR-3 as the prime example. EPR-3 used an unconventional evidence-grading system that emphasized precision and consistency (statistical significance, large sample sizes, and/or consistency of results) at the expense of patient-oriented outcomes and generalizability (applicability to the general population). EPR-3 did not report information on numbers needed to treat or numbers needed to harm, which are useful in eliciting patient preferences via shared decision making. Asthma guidelines (and others) are limited by lack of a generalizable research base, flawed evidence grading, and lack of attention to patient preferences. An evidence-grading system based on applicable populations, patient-oriented outcomes, and shared decision making might improve physician and patient guideline adherence and improve asthma outcomes. Ann Fam Med 2009;7:364-369. doi:10.1370/afm.995. Annals Journal Club selection; see inside back cover or http://www. annfammed.org/ajc/. Confl icts of interest: none reported CORRESPONDING AUTHOR David L. Hahn, MD, MS Dean East Clinic 1821 S. Stoughton Road Madison, WI 53716 dlhahn@wisc.edu INTRODUCTION Effective implementation of guidelines requires 3 interrelated processes: (1) an explicit assessment of the strength of the best available medical evidence (evidence quality), (2) application of clinical judgment in the care of individual patients, and (3) elicitation of patient preferences via shared decision making. Most current evidence-based guidelines address only the fi rst of these 3 processes (evidence quality). Implementation requires the addition of tailoring guidelines to individuals because evidence alone is never suffi cient to make a clinical decision. 1 With a few notable exceptions, 2,3 current versions of evidence-based guidelines do not address patient preferences or include tools for shared decision making. Although patient preferences have not historically been at the heart of evidence-based medicine, 4 an emerging consensus seems to be developing that patient preferences should be included, 5-7 particularly when evidence is weak. 8 The growing body of evidence-based shared decision-making tools should facilitate elicitation of patient preferences. 9 This line of reasoning suggests that guidelines should present all relevant benefi ts (as numbers needed to treat [NNT]) and harms (as numbers needed to harm [NNH]) of the best medical evidence in terms understandable to physicians (and patients) in order to facilitate the shared decision-making process. The National Heart, Lung and Blood Institute recently published its updated guidelines on asthma: Expert Panel Report 3: Guidelines for the Diagnosis and Management of Asthma (EPR-3). 10 The fi rst expert panel report, published 364

in 1991, 11 was criticized by Berg and Moy 12 for lack of an evidence-based process for developing and communicating the guidelines. The EPR-3, the third such report, states that it has overcome these critical fl aws by adopting an evidence-based methodology. This special report aims to illustrate strengths and defi ciencies of current evidence-based guidelines using EPR-3 as the prime example. It should be noted, however, that some or all of the guideline limitations discussed in this analysis may apply to other disease guidelines as well as to asthma. 13 ASSESSING EVIDENCE QUALITY: 4 IMPORTANT QUESTIONS Practitioners and patients assessing evidence-based guidelines should ask themselves at least 4 questions: Are study results true, correct, or valid (internal validity); do the results relate to something that I or my patients value (patient-oriented outcomes); are the results applicable to me or to my patients (generalizability or external validity); and are all clinically relevant benefits (and harms) considered? 1 Evidence-grading systems include a hierarchy of evidence that usually acknowledges the primacy of randomized controlled trials (RCTs), then nonexperimental (observational) studies, then expert opinion. 1 Some systems, including the Strength of Recommendation Taxonomy (SORT), 14-16 additionally emphasize the primacy of patient-oriented (eg, morbidity, mortality, quality of life) outcomes, then surrogate (eg, laboratory, histopathologic) outcomes. Regarding generalizability, evidence-grading systems and guidelines may neglect to emphasize that the results of internally valid RCTs using patient-oriented outcomes may not be applicable to the populations specified in a guideline recommendation and should not always be assumed to provide high-quality evidence for therapy recommendations. 13 Clinicians must therefore remain vigilant about the validity of recommendations of guidelines that claim to be evidence based. In addition, clinicians should be aware that some guidelines contain considerable material that has not been critically assessed for evidence quality (see Supplemental Appendix 1, available online at http://www.annfammed.org/ cgi/content/full/7/4/364/dc1 for further details). EVIDENCE GRADING FOR GUIDELINE IMPLEMENTATION EVIDENCE-GRADING SYSTEMS The EPR-3 described an explicit approach to a literature retrieval, review, and evidence-grading process for these sections of the guideline: the 4 components of asthma management: (1) measures of asthma assessment and monitoring, (2) education for a partnership in asthma care, (3) control of environmental factors and comorbid conditions that affect asthma, and (4) medications. The EPR-3 ranked newer evidence using the evidence grading system described in Table 1. An evidence category A recommendation for randomized trials required a consistent pattern of fi ndings in the population for which the recommendation was made and on substantial numbers of studies involving substantial numbers of participants. The system assigned primacy to quantity and consistency of evidence without mentioning generalizability or patient-oriented outcomes. Inexplicably, the EPR-3 system downgraded evidencebased meta-analyses, including those produced by the Cochrane Collaboration, to B-level evidence. In support of this strategy, EPR-3 cited a study by Jadad et al, 17 but this citation actually reported that Cochrane reviews were more rigorous and better reported compared with reviews published in peer-reviewed journals or funded by industry. Importantly, the EPR-3 grading system did not explicitly assess the quality of individual studies, and there was no transparency in the linkage between individual study quality and strength of recommendation. These defi ciencies are illustrated in example 1. The EPR-3 grading system also insuffi ciently addressed generalizability, as shown in example 2. The SORT evidence-grading system has been adopted by most family medicine journals. 14-16 SORT Table 1. Ranking the Evidence: Expert Panel Report 3 (EPR-3): Guidelines for the Diagnosis and Management of Asthma Evidence Category A: Randomized controlled trials (RCTs), rich body of data Description Evidence is from end points of well-designed RCTs that provide a consistent pattern of findings in the population for which the recommendation is made. Category A requires substantial numbers of studies involving substantial numbers of participants B: RCTs, limited body of evidence Evidence is from end points of intervention studies that include only a limited number of patients, post hoc or subgroup analysis of RCTs, or meta-analyses of RCTs. In general, category B pertains when few randomized trials exist; they are small in size, they were undertaken in a population that differs from the target population of the recommendation, or the results are somewhat inconsistent C: Nonrandomized trials and observational studies Evidence is from outcomes of uncontrolled or nonrandomized trials or from observational trials D: Panel consensus judgment This category is used only in cases where the provision of some guidance was deemed valuable, but the clinical literature addressing the subject was insufficient to justify placement in one of the other categories. The panel consensus is based on clinical experience and does not meet the criteria for categories A through C Adapted from: Expert Panel Report 3. Guidelines for the Diagnosis and Management of Asthma. 10(p7) 365

Figure 1. Strength of Recommendation Taxonomy (SORT): evidence-grading system for individual studies. In general, only key recommendations for readers require a grade of the Strength of Recommendation. Recommendations should be based on the highest quality evidence available. For example, Vitamin E was found in some cohort studies (Level 2 study quality) to have a benefit for cardiovascular protection, but good-quality randomized trials (Level 1) have not confirmed this effect. It is therefore preferable to base clinical recommendations in a manuscript on the level 1 studies. Strength of Recommendation A B C Definition Recommendation based on consistent and good quality patient-oriented evidence. a Recommendation based on inconsistent or limited quality patient-oriented evidence. a Recommendation based on consensus, usual practice, opinion, disease-oriented evidence, a and case series for studies of diagnosis, treatment, prevention, or screening. Use the table below to determine whether a study measuring patient-oriented outcomes is of good or limited quality, and whether the results are consistent or inconsistent between studies. Diagnosis Type of Study Treatment/Prevention/ Screening Prognosis Study Quality Level 1 Good quality patient-oriented evidence Level 2 Limited quality patient-oriented evidence Level 3 Other evidence Validated clinical decision rule Systematic review (SR)/meta-analysis of high quality studies High quality diagnostic cohort study b Unvalidated clinical decision rule SR/meta-analysis of lower quality studies or studies with inconsistent findings Lower quality diagnostic cohort study or diagnostic case-control study b SR/meta-analysis of RCTs with consistent findings High quality individual randomized controlled trial (RCT) c All or none study d SR/meta-analysis of lower quality clinical trials or of studies with inconsistent findings Lower quality clinical trial c Cohort study Case-control study SR/meta-analysis of good quality cohort studies Prospective cohort study with good follow-up SR/meta-analysis of lower quality cohort studies or with inconsistent results Retrospective cohort study or prospective cohort study with poor follow-up Case-control study Case series Consensus guidelines, extrapolations from bench research, usual practice, opinion, disease-oriented evidence (intermediate or physiologic outcomes only), and case series for studies of diagnosis, treatment, prevention, or screening. Consistency Across Studies Consistent Inconsistent Most studies found similar or at least coherent conclusions (coherence means that differences are explainable) or If high quality and up-to-date systematic reviews or meta-analyses exist, they support the recommendation. Considerable variation among study findings and lack of coherence or If high quality and up-to-date systematic reviews or meta-analyses exist, they do not find consistent evidence in favor of the recommendation. a Patient-oriented evidence measures outcomes that matter to patients: morbidity, mortality, symptom improvement, cost reduction, quality of life. Disease-oriented evidence measures intermediate, physiologic, or surrogate endpoints that may or may not reflect improvements in patient outcomes (i.e., blood pressure, blood chemistry, physiological function, and pathological findings). b High quality diagnostic cohort study: cohort design, adequate size, adequate spectrum of patients, blinding, and a consistent, well-defined reference standard. c High quality RCT: allocation concealed, blinding if possible, intention-to-treat analysis, adequate statistical power, adequate follow-up (>80%). d An all-or-none study is one where the treatment causes a dramatic change in outcomes, such as antibiotics for meningitis or surgery for appendicitis, which precludes study in a controlled trial. Reproduced by permission from the American Board of Family Medicine. emphasizes the primacy of patient-oriented over surrogate outcomes and explicitly describes criteria for internal validity (Figure 1). In keeping with wellaccepted principles of evidence-based medicine, SORT also acknowledges the primacy of evidence-based meta-analyses of high-quality RCTs. 1 366

Here are some examples of how the EPR-3 guideline recommendations appear differently when viewed through the SORT lens: Example 1: The authors recommend various allergen-avoidance measures, including encasing mattresses in allergen-impermeable covers. 1(p171) They cite 10 supporting references, including an editorial, a review, a before-after study, a study on rhinitis, a study involving multiple interventions and allergens, and a study excluded from a Cochrane review 18 because it included some patients who did not have dust mite sensitivity. The remaining 5 trials did not show a benefi t for mattress encasings. 18 The EPR-3 did not cite the Cochrane meta-analysis of 49 RCTs (2,733 patients) fi nding no evidence for effectiveness. 18 Example 2: Despite using the phrase in the populations for which the recommendation was made (Table 1), the EPR-3 does not acknowledge that evidencebased recommendations for inhaled corticosteroids (ICS) in asthma apply exclusively to nonsmokers. First, current smokers and those who have consumed more than 10 pack-years are routinely excluded from pharmaceutical-sponsored trials in pursuit of Food and Drug Administration approval for ICS in asthma. Second, there are several non pharmaceutical-funded trials showing that ICS treatment in asthmatic smokers does not improve patient-oriented outcomes. 19-21 Furthermore, in nonsmoking asthmatics with less than a 10 pack-year smoking history, a recent trial sponsored by the National Heart, Lung and Blood Institute found that ICS did not improve patient-oriented outcomes in 33 (46%) of 72 asthma patients. 22 The EPR-3 guidelines were produced by a group of American experts. A very similar guideline (Global Initiative for Asthma [GINA]) was produced by a group of experts worldwide. 23 The GINA report uses the identical EPR-3 grading system (Table 1) but states that it is considering adoption of a different grading system (Grading of Recommendations Assessment, Development and Evaluation [GRADE]) 5 in future guidelines. GRADE was developed by a widely representative group of international guideline developers. 6 As does SORT, GRADE clearly separates evidence quality assessment from strength of recommendation and includes patient-oriented outcomes. Importantly, GRADE also specifi es generalizability and explicitly acknowledges values and preferences. 8,24-27 For the clinician, GRADE endorses elicitation of patient preferences, particularly when evidence is weak. 6 For the policy maker, GRADE incorporates resource allocation and consensus building. 26,27 GRADE has been endorsed by the American Thoracic Society but has not yet found its way into contemporary asthma guidelines. 28 PATIENT PREFERENCES The EPR-3 guidelines recommend daily low-dose ICS for mild persistent asthma (p. 343), yet prescribing of ICS for mild persistent asthma is controversial. 29 There are few generalizable effectiveness trials that yield information for shared decision making, but those that are available appear to support a weak recommendation. The only available SORT level-1 trial of the population effectiveness of ICS in mild persistent asthma of recent onset is START, 30 a large, multinational, randomized effectiveness trial in children and adults that did not exclude smokers. START randomized 7,241 patients in 32 countries to inhaled budesonide or placebo once daily for 3 years. The primary outcome was time to fi rst severe asthma-related event defi ned as admission, emergency treatment, or death from asthma. The 3 components were not reported separately, so it is not possible to judge the actual clinical severity of outcome events, although no deaths were reported. One hundred ninety-eight (5.5%) of 3,568 patients assigned placebo and 117 (3.3%) of 3,597 assigned budesonide had a severe asthma-related event (NNT = 44 to prevent 1 exacerbation over 3 years, P <.001). No information was provided on whether smokers and nonsmokers responded differentially to treatment. Regarding harms, low-dose budesonide caused growth retardation in children aged 5 to 15 years ( 0.43 cm/y, P <.001). Growth retardation in children taking ICS has also been reported in another study. 31 START provides the kind of information needed for shared decision making. Nowhere in the EPR-3 guidelines is there mention of the number needed to treat (NNT), number needed to harm (NNH), or lack of effectiveness of ICS in smokers. Including this kind of information in future guidelines would facilitate shared decision making and would allow individual patients (or their parents) to apply their personal values. Clinicians should realize, however, that NNTs derived from poor-quality asthma RCTs may not be valid for shared decision making (see Supplemental Appendix 2, available online at http://www.annfammed.org/cgi/content/ full/7/4/364/dc1 for an example). Shared decision making depends on the quality and completeness of the underlying research, as well as on the completeness of its presentation in guidelines. An analytical review of 504 trials of ICS found signifi cant associations between specifi c study design characteristics and the likelihood of reporting statistically signifi cant results for adverse effects. 32 Compared with other study designs, the likelihood of reporting adverse effects was lower in randomized trials, in those with an effi cacy design, in those studying only nonspecifi c clinical adverse effects, and in studies of children; stud- 367

ies focusing on specifi c adverse effects ie, growth, cortisol, and higher steroid doses were more likely to report adverse effects. 32 In a comparison of 275 industry-funded and 229 non industry-funded studies, industry funding was signifi cantly associated with many of the study designs reporting fewer adverse effects, including more randomized trials, more studies stating effi cacy as an aim, more studies of only nonspecifi c clinical or laboratory results, fewer studies of specifi c adverse effects, and fewer studies stating safety as an aim or as the only aim. 32 In addition to a decreased likelihood for industry-funded studies to report statistically signifi cant adverse effects, the authors of industry-funded studies that did report such adverse effects were less likely than other authors to conclude that these were clinically important. For example, of studies that found statistically signifi cant adverse effects, 41.8% of 79 industry-funded studies vs 11.4% of 132 non industry-funded studies concluded that the treatment was safe (prevalence ratio 3.68; 95% CI, 2.14-6.33). 32 The authors of the analytical review stated: [E]xtrapolation of statistical to clinical signifi cance is based on subjective criteria, so we cannot estimate if (industryfunded studies) are too benevolent or (non industry-funded) studies are too cautious. However, we postulate that having information on source of funding will help readers of these studies have a better informed and balanced judgment on the authors interpretations. 32 A current characteristic of the National Institues of Health asthma expert panel report guideline development process is that many of the same experts who sit on the panel also perform the research often industry funded that forms the basis for the guidelines. IMPLICATIONS FOR PRIMARY CARE PRACTICE Do the identifi ed defi ciencies negate the guidelines entirely? ICS treatment is currently the cornerstone of asthma control, but there is growing awareness that ICS treatment has signifi cant limitations, as described above. Will patients who stop smoking regain ICS responsiveness? This important clinical question cannot be answered at this time because this particular research question will never be recognized by an asthma research agenda that continues to exclude smokers. Other parts of the guideline, including asthma self-management, are supported by better evidence. One wonders, however, how much better management outcomes would be if the limitations of ICS treatments were recognized and addressed. The implications for primary care are in general are as follows: (1) be vigilant and skeptical when reading evidence-based guidelines; (2) in particular, recognize the limitations of current asthma treatments; (3) share this information in a balanced way with your patients; and (4) at every opportunity be an advocate for more clinically relevant research. Look for shared decisionmaking tools that are unbiased and that can be used in the offi ce. Consider whether any guideline recommendations made without such information are useful. CONCLUDING REMARKS Less than one-third of asthma patients nationwide adhere to the expert panel report guideline recommendations for asthma treatment. 33 Some believe that limitation in primary care physicians knowledge is largely responsible for lack of guideline adherence. More attention should be paid to the limitations of the guidelines and underlying evidence base themselves as causes for the current gap between recommendation and practice. An Agency for Health Research and Quality technology assessment on asthma treatment commissioned by the expert panel 34 states: The overriding priority is to develop a national research agenda for long-term studies to improve the effectiveness of asthma management. Short-term drug effi cacy studies are over-represented in the current literature. It is imperative to develop an evidence base that supports clinical decisionmaking on the intensity of treatment, optimization of medication regimens, and utility of disease management interventions for various asthma populations. 34(p10) This special report advocates that future asthma guidelines (1) should adopt a conventional evidencegrading system that accounts for generalizability and patient-oriented outcomes, and (2) should present all relevant outcomes as NNT and NNH to facilitate shared decision making. GRADE appears to be the most comprehensive grading system available to account for evidence quality, patient-important outcomes, and patient values and preferences. Adoption of GRADE or its equivalent would serve to highlight the limitations of current asthma research, promote effectiveness research in a broad sample of the kind of asthma patients encountered daily, and discourage efficacy research in highly selected nonrepresentative subsamples. When coupled with effective decision aids, 9 such guidelines might improve the overall effectiveness of care delivery. To read or post commentaries in response to this article, see it online at http://www.annfammed.org/cgi/content/full/7/4/364. Submitted June 30, 2008; submitted, revised, November 14, 2008; accepted December 1, 2008. Key words: Asthma; clinical guidelines; evidence-based medicine 368

References 1. Guyatt G, Rennie D, eds. Users Guides to the Medical Literature: Essentials of Evidence-Based Clinical Practice. Chicago, IL: AMA Press; 2002. 2. National Collaborating Centre for Chronic Conditions. Type 2 Diabetes: National Clinical Guideline for Management in Primary and Secondary Care (update). London: Royal College of Physicians; 2008. 3. National Institute for Health and Clinical Excellence (NICE). Lipid Modification: Cardiovascular Risk Assessment and the Modification of Blood Lipids for the Primary and Secondary Prevention of Cardiovascular Disease. NICE Clinical Guideline 67 2008. http://www.nice.org. uk/nicemedia/pdf/cg67niceguideline.pdf. 4. Krahn M, Naglie G. The next step in guideline development: incorporating patient preferences. JAMA. 2008;300(4):436-438. 5. Guyatt G, Vist G, Falck-Ytter Y, Kunz R, Magrini N, Schunemann H. An emerging consensus on grading recommendations? Evid Based Med. 2006;11(1):2-4. 6. Guyatt GH, Oxman AD, Vist GE, et al.; GRADE Working Group. GRADE: an emerging consensus on rating quality of evidence and strength of recommendations. BMJ. 2008;336(7650):924-926. 7. Montori VM, Guyatt GH. Progress in evidence-based medicine. JAMA. 2008;300(15):1814-1816. 8. Guyatt GH, Oxman AD, Kunz R, et al; GRADE Working Group. Going from evidence to recommendations. BMJ. 2008;336(7652): 1049-1051. 9. O Connor AM, Stacey D, Entwistle V, et al. Decision aids for people facing health treatment or screening decisions. Cochrane Database Syst Rev. 2003;(2):CD001431. 10. US Department of Health and Human Services, National Institutes of Health, National Heart, Lung, and Blood Institute. Expert Panel Report 3. Guidelines for the Diagnosis and Management of Asthma. Bethesda, MD: National Institutes of Health; 2007:1-417. 11. US Department of Health and Human Services, National Institutes of Health, National Heart, Lung, and Blood Institute. Expert Panel Report. National Asthma Education Program: Guidelines for the Diagnosis and Management of Asthma. Office of Prevention, Education, and Control. National Heart. Lung, and Blood Institute. National Institutes of Health. Bethesda, MD: National Institutes of Health; 1991:1-136. Publication No. 91-3042. 12. Berg AO, Moy JG. Guidelines for the diagnosis and management of asthma. J Am Board Fam Pract. 1992;5(6):629-634. 13. McAlister FA, van Diepen S, Padwal RS, Johnson JA, Majumdar SR. How evidence-based are the recommendations in evidence-based guidelines? PLoS Med. 2007;4(8):e250. 14. Ebell MH, Siwek J, Weiss BD, et al. Simplifying the language of evidence to improve patient care: Strength of recommendation taxonomy (SORT): a patient-centered approach to grading evidence in medical literature. J Fam Pract. 2004;53(2):111-120. 15. Ebell MH, Siwek J, Weiss BD, et al. Strength of recommendation taxonomy (SORT): a patient-centered approach to grading evidence in the medical literature. J Am Board Fam Pract. 2004;17(1):59-67. 16. Ebell MH, Siwek J, Weiss BD, et al. Strength of recommendation taxonomy (SORT): a patient-centered approach to grading evidence in the medical literature. Am Fam Physician. 2004;69(3):548-556. 17. Jadad AR, Moher M, Browman GP, et al. Systematic reviews and meta-analyses on treatment of asthma: critical evaluation. BMJ. 2000;320(7234):537-540. 18. Gøtzsche PC, Johansen HK, Schmidt LM, Burr ML. House dust mite control measures for asthma. Cochrane Database Syst Rev. 2004;(4): CD001187. 19. Lazarus SC, Chinchilli VM, Rollings NJ, et al; National Heart Lung and Blood Institute s Asthma Clinical Research Network. Smoking affects response to inhaled corticosteroids or leukotriene receptor antagonists in asthma. Am J Respir Crit Care Med. 2007;175(8):783-790. 20. Tomlinson JEM, McMahon AD, Chaudhuri R, Thompson JM, Wood SF, Thomson NC. Efficacy of low and high dose inhaled corticosteroid in smokers versus non-smokers with mild asthma. Thorax. 2005;60(4):282-287. 21. Chaudhuri R, Livingston E, McMahon AD, Thomson L, Borland W, Thomson NC. Cigarette smoking impairs the therapeutic response to oral corticosteroids in chronic asthma. Am J Respir Crit Care Med. 2003;168(11):1308-1311. 22. Martin RJ, Szefler SJ, King TS, et al; the National Heart, Lung, and Blood Institute s Asthma Clinical Research Center. The predicting response to inhaled corticosteroid efficacy (PRICE) trial. J Allergy Clin Immunol. 2007;119(1):73-80. 23. Global-Initiative-for-Asthma-(GINA)-2007. Global strategy for asthma management and prevention. http://www.ginasthma.org. 24. Guyatt GH, Oxman AD, Kunz R, Vist GE, Falck-Ytter Y, Schünemann HJ; GRADE Working Group. What is quality of evidence and why is it important to clinicians? BMJ. 2008;336(7651):995-998. 25. Schünemann HJ, Oxman AD, Brozek J, et al; GRADE Working Group. Grading quality of evidence and strength of recommendations for diagnostic tests and strategies. BMJ. 2008;336(7653):1106-1110. 26. Guyatt GH, Oxman AD, Kunz R, et al; GRADE Working Group. Incorporating considerations of resources use into grading recommendations. BMJ. 2008;336(7654):1170-1173. 27. Jaeschke R, Guyatt GH, Dellinger P, et al. Use of GRADE grid to reach decisions on clinical practice guidelines when consensus is elusive. BMJ. 2008;337:a744. 28. Schünemann HJ, Jaeschke R, Cook DJ, et al.; ATS Documents Development and Implementation Committee. An official ATS statement: grading the quality of evidence and strength of recommendations in ATS guidelines and recommendations. Am J Respir Crit Care Med. 2006;174(5):605-614. 29. O Byrne P, Boushey HA. Pro/Con Editorials: Daily inhaled corticosteroid treatment should be prescribed for mild persistent asthma. Am J Respir Crit Care Med. 2005;172(4):410-416. 30. Pauwels RA, Pedersen S, Busse WW, et al; START Investigators Group. Early intervention with budesonide in mild persistent asthma: a randomised, double-blind trial. Lancet. 2003;361(9363):1071-1076. 31. Guilbert TW, Morgan WJ, Zeiger RS, et al. Long-term inhaled corticosteroids in preschool children at high risk for asthma. N Engl J Med. 2006;354(19):1985-1997. 32. Nieto A, Mazon A, Pamies R, et al. Adverse effects of inhaled corticosteroids in funded and nonfunded studies. Arch Intern Med. 2007;167(19):2047-2053. 33. Fuhlbrigge AL, Adams RJ, Guilbert TW, et al. The burden of asthma in the United States: level and distribution are dependent on interpretation of the national asthma education and prevention program guidelines. Am J Respir Crit Care Med. 2002;166(8):1044-1049. 34. Aronson N, Lefevre F, Piper M, et al. Management of Chronic Asthma. Evidence Report/Technology Assessment Number 44. (Prepared by Blue Cross and Blue Shield Association Technology Evaluation Center under Contract No. 290-97-0015). Rockville, MD: Agency for Healthcare Research and Quality: AHRQ Publication No. 01-E044, September 2001. 369