Case 1 Agency for Healthcare Research and Quality www.ahrq.gov Preventive Medicine 2009: Understanding the US Preventive Services Task Force Guidelines George F. Sawaya, MD Associate Professor Department of Obstetrics, Gynecology and Reproductive Sciences Department of Epidemiology and Biostatistics University of California, San Francisco Director, Cervical Dysplasia Clinic, San Francisco General Hospital Some slides courtesy of AHRQ/USPSTF Your 74-year-old father is found to have a slightly elevated serum PSA drawn as part of a routine exam. He asks you what this means. He says his doctor recommends that since he smokes, he be screened for an abdominal aortic aneurysm, COPD and lung cancer. He was also told that he has a carotid bruit. Who is the USPSTF? Current Members of the USPSTF Experts in primary care, prevention, research methods Government supported by AHRQ but independent Family medicine, internal medicine, pediatrics, obstetrics/gynecology, nursing, behavioral health Scientific support from an Evidence-Based Practice Center (EPC) Non-member liaisons from primary care clinician associations, Federal agencies Bruce N. (Ned) Calonge, M.D., M.P.H. (Chair) Diana B. Petitti, M.D., M.P.H. (Vice Chair) Thomas G. DeWitt, MD Allen Dietrich, MD, MPH Kimberly D. Gregory, MD, MPH David Grossman, MD, MPH George Isham, MD, M.P.H. Michael LeFevre, MD, MSPH Rosanne Leipzig, MD, PhD Bernadette Melnyk, PhD, RN, CPNP/NPP Lucy N. Marion, PhD, RN Virginia A. Moyer, MD, MPH *Judith K. Ockene, PhD, MSEd *George F. Sawaya, MD J. Sanford Schwartz, MD, AB Timothy Wilt, MD, MPH 1
Task Force Activities Why Evidence-Based? Provide evidence-based scientific reviews of preventive health services for use in primary healthcare delivery settings for patients without recognized signs or symptoms of target condition Age- and risk-factor specific recommendations for routine practice Recommendations include: Screening tests Counseling Preventive medications Need transparent systematic process to obtain and distill best available (or best feasible) evidence to support decision making Identifying, evaluating and summarizing scientific evidence about outcomes or interventions or policies Translating evidence into practice recommendations Certainty of Net Benefit USPSTF Grades of Recommendations Substantial Magnitude of Net Benefit Moderate Small Zero/negative High A B C D Moderate B B C D Low Insufficient Grade A B C D Communicating USPSTF Recommendations Grade Definition The USPSTF recommends the service. There is high certainty that the net benefit is substantial. The USPSTF recommends the service. There is high certainty that the net benefit is moderate or there is moderate certainty that the net benefit is moderate to substantial. The USPSTF recommends against routinely providing the service. There may be considerations that support providing the service in an individual patient. There is moderate or high certainty that the net benefit is small. The USPSTF recommends against the service. There is moderate or high certainty that the service has no net benefit or that the harms outweigh the benefits. Suggestion for Practice Offer or provide this service. Offer or provide this service. Offer or provide this service only if there are other considerations in support of the offering or providing the service in an individual patient. Discourage the use of this service. I The USPSTF concludes that the current evidence is insufficient to assess the balance of benefits and harms of the service. Evidence is lacking, of poor quality, or conflicting, and the balance of benefits and harms cannot be determined. Read Clinical Considerations section of USPSTF Recommendation Statement. If offered the service, patients should understand the uncertainty about the balance of benefits and harms. 2
Case 2 Steps in the Process Your 40-year-old sister has received an e-mail encouraging her to get a CA-125 for ovarian cancer screening. 3. Evaluate QUALITY of individual studies 4. Synthesize and judge STRENGTH of available evidence 6. Link recommendation to judgment about net benefits Step 1: Analytic Framework on Screening for a Disease Steps in the Process 8 Key Questions (KQ) 3. Evaluate QUALITY of individual studies 4. Synthesize and judge STRENGTH of available evidence 6. Link recommendation to judgment about net benefits 3
Define and Retrieve Relevant Evidence For each KQ developed from AF: Create inclusion/exclusion criteria based on the key questions defined from the analytic framework PubMed search Cochrane search Other database search (CINAHL, etc) References from key articles, editorials, review articles Expert consultation (others, TF members) 8 Key Questions (KQ) Step 1: Analytic Framework on Screening for a Disease Steps in the Process 3. Evaluate quality of individual studies 4. Synthesize and judge STRENGTH of available evidence 6. Link recommendation to judgment about net benefits Evaluate Quality of Individual Studies Good: Uses a credible reference standard Reliability of test assessed Includes large number of subjects Fair: Uses reasonable although not best standard Interprets reference std independent of screening test Moderate sample size Poor: Has fatal flaw such as: Uses inappropriate reference standards Biased ascertainment of reference standard Very small sample size or very selected patients. 4
Steps in the Process 3. Evaluate quality of individual studies 4. Synthesize and judge STRENGTH of overall evidence 6. Link recommendation to judgment about net benefits Critical Appraisal Questions Do the studies have the appropriate research design to answer the key questions? To what extent are the existing studies high quality? To what extent are the results of the studies generalizable (or applicable ) to the general US primary care population and situation? How many studies have been conducted that address the key questions? How large are the studies? How consistent/coherent are the results of the studies? Are there additional factors that assist us in drawing conclusions about the certainty of the evidence? (e.g., presence or absence of dose-response effects; fit within a biologic model) Step 4: Synthesize and Judge Strength of Overall Evidence Step 4: Synthesize & Judge Strength of Key Question Evidence Evidence reports Evidence tables summarizing studies Narrative discussing overall strength of evidence Meta-analysisanalysis Modeling Systematic reviews from others Cochrane, etc. Convincing: Well-designed, well-conducted studies in representative populations that directly assess effects on health outcomes Adequate: Evidence sufficient to determine effects on health outcomes, but limited by number, quality, or consistency of studies, generalizability to routine practice, or indirect nature of the evidence. Inadequate : Insufficient evidence to determine effect on health outcomes due to limited number or power of studies, important flaws in their design or conduct, gaps in the chain of evidence, or lack of information on important health outcomes 5
Case 3 Steps in USPSTF Process Your 28-year-old patient saw a television commercial encouraging her to get an HPV test. She wants one. Now. 3. Evaluate quality of individual studies 4. Synthesize and judge STRENGTH of overall evidence 6. Link recommendation to net benefits Step 5: Determine Balance of Benefits and Harms Step 5: Determine Balance of Benefits and Harms: Assessing Harms Estimate Magnitude of Net Benefit Benefits Harms = Net Benefit Potential harms real but hard to quantify Include psychological and physical consequences of false-positives, false-negatives, labeling, over treatment Opportunity costs Magnitude and duration of harm subjective, hard to compare to benefits May translate into QALYs to compare NNH 6
Step 5: Determine Balance of Benefits & Harms: Assessing Magnitude of Net Benefit Certainty No explicit criteria for magnitude (substantial, moderate, small, zero/negative) Substantial benefit : impact on high burden or major effect on uncommon outcome Problems: requires evidence on harms and common metric for benefit and harms Definition: The U.S. Preventive Services Task Force defines certainty as likelihood that the USPSTF assessment of the net benefit of a preventive service is correct. The net benefit is defined as benefit minus harm of the preventive service as implemented in a general, primary care population. The USPSTF assigns a certainty level based on the nature of the overall evidence available to assess the net benefit of a preventive service. Sawaya GF, Guirguis-Blake J, LeFevre M. Barton M, Harris R, Petitti D, US Preventive Services Task Force. Update on the methods of the US Preventive Services Task Force: estimating certainty and magnitude of net benefit. Ann Intern Med. 2007;147:871-875. Levels of Certainty High: The available evidence usually includes consistent results from well- designed, well-conducted studies in representative primary care populations. These studies assess the effects of the preventive service on health This outcomes. conclusion is therefore unlikely to be strongly affected by the results of future studies. Moderate: The available evidence is sufficient to determine the effects of the preventive service on health outcomes, but confidence in the estimate is constrained by factors such as: the number, size, or quality of individual studies; inconsistency of findings across individual studies; limited generalizability of findings to routine primary care practice; or lack of coherence in the chain of evidence. As more information becomes available, the magnitude or direction of the observed effect could change, and this change may be large enough to alter the conclusion. Low: The available evidence is insufficient to assess effects on health outcomes. Evidence is insufficient because of: the limited number or size of studies; Important flaws in study design or methods; inconsistency of findings across individual studies gaps in the chain of evidence; findings not generalizable to routine primary care practice; or a lack of information on important health outcomes. More information may allow an estimation of effects on health outcomes. Case 4 Your 40 year old male patient is considering aspirin therapy for the prevention of cardiovascular events. He is otherwise healthy. Yes? No? Maybe? 7
Steps in the Process Step 6: Link recommendation to net benefits: USPSTF Grades of Recommendations 3. Evaluate quality of individual studies 4. Synthesize and judge STRENGTH of overall evidence Certainty of Net Benefit Substantial Magnitude of Net Benefit Moderate Small Zero/negative High A B C D Moderate B B C D Low Insufficient 6. Link recommendation to net benefits Step 6: Link recommendation to net benefits: Insufficient Evidence Electronic Resources for Clinicians Lack of evidence on harms or benefits Poor quality of existing studies Good quality studies with conflicting results epss electronic Preventive Services Selector Tool Search USPSTF recommendations by age, sex and risk factors Available as a web-based based tool or can be downloaded to your PDA www.epss.ahrq.gov 8
Cases: A, B, C, D or I? Your 74-year-old father is found to have a slightly elevated serum PSA drawn as part of a routine exam I He says his doctor also recommends he be screened for an abdominal aortic aneurysm B lung cancer I COPD D carotid artery stenosis D Asymptomatic Men Analytic Framework - Prostate Cancer Screening Screen: PSA, DRE 5 2 Adverse effects of screening: false positive, false negative, inconvenience, labeling 1 Early Prostate Cancer Treat radiation, prostatectomy 4 Adverse effects of Rx: Impotence, incontinence, death, overtreatment 3 Reduced prostate cancer morbidity, mortality 9
Cases: A, B, C, D or I? Your 40-year-old sister has received an e-mail encouraging her to get a CA-125 for ovarian cancer screening. D Your 28-year-old patient saw a television commercial encouraging her to get an HPV test. I Your 40 year old male patient is considering aspirin therapy for the prevention of cardiovascular events. A Resources for Clinicians and Excellence in Consumers Based on recommendations of the USPSTF. At-a-glance wall chart for appropriate preventive services based on age, sex, and risk status. To be used in prompting shared decision-making between consumers and their primary care clinician. Resources for Consumers AHRQPubs@ahrq.hhs.gov 10
Summary Preventive interventions require a high burden of proof: the do no harm principle. The Task Force takes this seriously. Guidelines are designed to maximize population screening benefits and minimize population screening harms Guidelines involve a lot of evidence, but a certain degree of judgment. Task Force Mantra Questions? An ounce of prevention is a whole lot of work. ---- Paul Frame, MD (Task Force Member Emeritus) 11