Slide 1: Introduction. Slide 2: Acknowledgements. Slide 3: Presentation Objectives

Transcription

1 Slide 1: Introduction Developed through APTR Initiative to Enhance Prevention and Population Health Developed Education through in collaboration APTR Initiative with to Brody Enhance School Prevention of Medicine and Population at East Carolina Health University Education in with collaboration funding from with Centers Brody for School Disease of Medicine Control and at East Prevention Carolina University with funding from Centers for Disease Control and Prevention The screening module is one of a series created through funding from Centers for Disease Control and Prevention and Association for Prevention Teaching and Research. Slide 2: Acknowledgements APTR APTR wishes wishes to to acknowledge following following individuals individuals that that developed developed this this module: module: Anna Anna Zendell, Zendell, PhD, PhD, MSW MSW Center for Public Health Continuing Education Center for Public Health Continuing Education University at Albany School of Public Health University at Albany School of Public Health Joseph Joseph Nicholas, Nicholas, MD, MD, MPH MPH University of Rochester School of Medicine University of Rochester School of Medicine Mary Mary Applegate, Applegate, MD, MD, MPH MPH University at Albany School of Public Health University at Albany School of Public Health Cheryl Cheryl Reeves, Reeves, MS, MS, MLS MLS Center for Public Health Continuing Education Center for Public Health Continuing Education University at Albany School of Public Health University at Albany School of Public Health This education module is made possible through Centers for Disease Control and Prevention (CDC) and Association This education for Prevention module is Teaching made possible and Research through (APTR) Centers Cooperative for Disease Agreement, Control and No. Prevention 5U50CD (CDC) and The module represents Association for opinions Prevention of Teaching author(s) and and Research does not (APTR) necessarily Cooperative represent Agreement, views of No. 5U50CD Centers for Disease The module Control represents and Prevention opinions or of Association author(s) and for Prevention does not necessarily Teaching and represent Research. views of Centers for Disease Control and Prevention or Association for Prevention Teaching and Research. Slide 3: Presentation Objectives 1. The objectives of this presentation are as 1. Define screening and and identify appropriate conditions for for screening Evaluate screening tests tests in in terms of of ir ir validity, follows: Define screening and appropriate results and and generalizability Evaluate effectiveness of of a screening program and and conditions for screening; evaluate screening discuss common biases Discuss ethical considerations in in screening tests in terms of validity, results, and generalizability; evaluate effectiveness of a screening program; discuss common biases in evaluating screening programs; and discuss common ethical considerations in screening. Slide 4: Introduction to Screening We will begin with an overview of screening. 1

2 Slide 5: Oprah s Full Body Scan Take a few minutes to watch a video clip related As As you you watch watch this this clip clip and and complete module, think think about about implications for for patient screening based based on on this to screening. Oprah Winfrey has completed a this technology Medical concerns? full body scan. Fully body scans are one of Ethical considerations? newer fads in healthcare. Some are done in a Access issues? medical office and ors are conducted out of Informed decision-making after after screening? mobile vans. Oprah is discussing results of her scan on national TV with her doctor. As you watch clip and complete this module think about implications for patient screening. Are re medical concerns that may result from using this technology? How about any ethical issues? What barriers to accessing full body scans may exist and for what groups of people? In particular, think about what happens after scan. Every human body has some imperfections. Do se imperfections necessarily require medical intervention, or may uncovering m cause patient unnecessary physical or psychological harm? Slide 6: Preventive Medicine & Public Health Preventive medicine and public health share Share common goals Enhance quality of of life life of of patients many common goals. First and foremost, both Health Health promotion Disease Disease and and injury injury prevention strive to promote quality of life by preventing Preventive medicine promotes se goals at at disease. This goal is accomplished through individual and population levels, while public health focuses on on populations. health promotion programs and prevention of common diseases, like diabetes, cancer, and injuries. The difference is that preventive medicine works toward se goals at both individual patient level and population level. Public health s focus is at population level. [The distinguishing factor about preventive medicine that makes it special is its focus on populations!] 2

3 Slide 7: Prevention Brief Overview Prevention occurs at three levels. The first level Tertiary is primary prevention. These are preventive Prevention Secondary measures that are undertaken to prevent Prevention Primary onset of illness and injury. This is done through Prevention elimination of causal risk factors or by increasing resistance to condition. An McKenzie et al.: 2008 McKenzie et al.: 2008 example of primary prevention is childhood vaccination against infectious disease. Secondary prevention entails measures that lead to early diagnosis and prompt treatment of illness or injury. Here we try to interrupt disease process by detecting and treating it before symptoms emerge. For a test to have screening characteristic it must be done in non-symptomatic phase of disease. Cancer screening is an example of secondary prevention. Tertiary prevention involves measures aimed at minimizing disability after disease symptoms have appeared. Cardiac rehabilitation following a diagnosis of heart failure is an example of tertiary prevention. Slide 8: Screening Defined Presumptive identification of of an an unrecognized disease through tests, examinations, or or or procedures which can can be be applied rapidly Screening tests sort out out apparently well persons who probably have a disease from those who probably do do not. Screening is defined as presumptive identification of an unrecognized disease or defect through tests, exams, or or procedures that can be applied rapidly and easily. Screening tests differentiate apparently healthy persons who may have a disease from those who probably don t have disease. 3

4 Slide 9: Importance of Screening Screening is widely considered bedrock of Early Early detection Leads Leads to to early early treatment treatment secondary prevention. Periodic health screening Can Can lead lead to to a a decrease decrease in in morbidity morbidity and and mortality mortality can lead to early detection and diagnosis of a Can Can break break chain chain of of transmission transmission and and development development of of new new cases cases disease. This early detection n leads to Is Is often often cost-effective The The human body body is is continually changing earlier treatment with a goal of decreasing mortality and morbidity related to that disease. Jekel et al:, 1996; McKenzie et al:, 2008; Londrigan & Lewenson: Jekel et al:, ; McKenzie et al:, 2008; Londrigan & Lewenson: 2011 In case of infectious diseases, screening can also break chain of transmission and prevent development of new cases. Screenings can be cost-effective if disease is common enough and test is accurate enough. It s also cost-effective if affordable treatments that work are accessible to those patients whom test positive. Because we may develop diseases at many points throughout our lifespan, many screenings are only effective if done periodically. Slide 10: Screening-Diagnosis Connection Screening starts before diagnosis History questions Physical exam exam findings Lab Lab tests tests Pre-test probability Results of of screening trigger diagnostic work-up and and preventive interventions interventions. Jekel et al:, 1996; McKenzie et al:, 2008 Jekel et al:, 1996; McKenzie et al:, 2008 The best screening protocols generally incorporate a patient history, physical exam, and laboratory test. Pre-test probability, also known as demographically determined risk, is probably biggest reason to screen someone. Positive results of screening test will trigger a diagnostic work-up and preventive or treatment Slide 11: Screening versus Diagnostic Tests Screening Test Test Diagnostic Test Test Many people use terms screening and diagnosis interchangeably. They are not same. 4

5 Slide 12: Screening versus Diagnostic Tests Screening Test Test Identifies asymptomatic people people who who may may have have a disease disease Diagnostic Test Test Screening tests are used for a presumptive identification of an unrecognized disease or illness. Slide 13: Screening versus Diagnostic Tests Screening Test Test Identifies asymptomatic people people who who may may have have a disease disease Diagnostic Test Test Determines presence presence or or absence absence of of disease disease when when patient patient shows shows signs signs or or symptoms Diagnostic tests, on or hand, are used to determine presence or absence of a disease when patient is showing symptoms of disease OR has been targeted through a positive screening test. In some circumstances, same test can function as eir a screening or diagnostic test. Slide 14: Characteristics of a Good Screening Test Simple A good screening test must meet several Rapid Inexpensive important criteria. It needs to be simple and Safe Available quick to administer. It should be inexpensive Acceptable and safe to use. It also needs to be readily available, along with an accessible plan of treatment in place in case of positive results. A good screen must be acceptable to population in which it will be used. It must also be well researched and proven to be valid, reliable, and to have good predictive values. We will discuss se criteria in more detail a bit later. Slide 15: Common Screening Tests Next, we will discuss some common screening tests. 5

6 Slide 16: Common Disease Screenings Pap Pap smear smear screens screens for for Fasting Fasting blood blood sugar sugar screens screens for for Fecal Fecal occult occult blood blood test test screens screens for for Blood Blood pressure pressure screens screens for for Bone Bone densitometry screens screens for for PSA PSA test test screens screens for for PPD PPD test test screens screens for for Mammography screens screens for for Take a few moments to review this list of screenings commonly conducted in US. Do you know what conditions y screen for? USPSTF: 2009 USPSTF: 2009 Slide 17: Common Disease Screenings Pap Pap smear smear screens screens for for cervical cervical cancer cancer Medical practitioners have access to numerous Fasting Fasting blood blood sugar sugar screens screens for for diabetes diabetes screenings for potentially life-threatening Fecal Fecal occult occult blood blood test test screens screens for for colorectal cancer cancer Blood Blood pressure pressure screens screens for for hypertension conditions. We will briefly discuss some Bone Bone densitometry screens screens for for osteoporosis & osteopenia PSA common ones and n focus on several in more PSA test test screens screens for for prostate prostate cancer cancer PPD PPD test test screens screens for for tuberculosis detail. All women age 21 or older should receive Mammography screens screens for for breast breast cancer. cancer. Pap smears to test for cervical cancer. Pap USPSTF: 2009 USPSTF: 2009 smears may be administered even earlier for younger women and girls who are sexually active. These are routine ongoing screens, usually performed every 1-3 years. Pap smears may be done more often if abnormalities are found, or if a family history is reported. A fasting blood sugar test is conducted for anyone at any age at risk for diabetes, for instance pregnant women, people who are overweight, or those with a family history of diabetes. The fecal occult blood test is a common screening for colorectal cancer. We will learn more about this a bit later. Blood pressure is taken at annual exams and during each visit to primary care provider. If hypertension is found, patients may be advised to come regularly for blood pressure tests. In some cases, people may receive blood pressure cuffs which enable m to check and log ir blood pressure daily. Osteoporosis or osteopenia are often diagnosed in adults over age fifty, or in adults of any age with metabolic or eating disorders. Bone densitometry is indicated for women over age of 65. People with a family history of se conditions or or risk factors may also have a bone density scan. Men over age of 50 can be screened for prostate cancer, through an annual PSA test (or prostate-specific antigen). The typical tuberculosis screening is a TB skin test, also called a purified protein derivative or PPD test. TB tests are mandated for children and adults attending educational institutions. Anyone working in a healthcare setting or correctional facility is also mandated to be tested annually. Mammography is used to screen for breast cancer. The guidelines for screening are quite controversial. We will discuss se more in a few moments. 6

7 Slide 18: Common Wellness Screenings There are numerous federal, state, and local Obesity Weight, Body Body Mass Mass Index Index Dental caries, oral oral cancer Oral Oral examination initiatives to fight obesity in children and adults. Drugs, Alcohol, and and Urine test, test, NMASSIST, or or Tobacco Weight is one of standard screens that occur Flagerstrom Tolerance Test Test for for Nicotine Dependency at each medical visit. In many cases, BMI or body mass index is also calculated. A standard oral examination can identify dental caries (cavities) as well as oral cancers and many or conditions. The NIDA-Modified Alcohol, Smoking, and Substance Involvement Screening Test is a commonly used screening to detect use and abuse of alcohol, tobacco, and or legal and illegal drugs. The screening test starts out with preliminary questions to assess use and n asks questions to identify abuse and/ or dependence. Now, let s delve into more detail on a few screenings. Slide 19: Breast Cancer Screening There has been a great deal of controversy Standard practice Annual Annual mammograms for for women women age age years years recently on protocols for breast cancer Start Start earlier earlier if if family family history history of of breast breast cancer cancer 2009 US US Preventive Services Task Task Force (USPSTF) screening. Standard protocol for decades has recommendations Mammograms not notuniversal for for women women age age years years been annual mammogram for women age 40 Bi-annual Bi-annual mammograms for for women women years years Cost-benefit analysis and older. Younger women are also screened False False positives positives Unnecessary Unnecessary invasive invasive procedures procedures annually if re is a family history of breast cancer. In 2009, United States Preventive Services Task Force reviewed medical evidence on regular use of mammograms. Issues such as population prevalence, costs, and consequences of false positive results were considered. Based on this review, recommendations changed in late Mammograms are no longer recommended for women under 50, unless y and ir providers feel it s necessary. Women 50 and older are now urged to get mammograms every or year. The opposition to revised mammogram recommendations has been strong. Many healthcare providers organizations, particularly radiologists and breast surgeons, have long advocated for yearly screenings. They have worked hard to promote importance of regular mammograms. For more information on reasons behind controversy, please see articles appended in resources section. 7

8 Slide 20: Colon Cancer Screening We have a number of screening options available Multiple screening options Colonoscopy Colonoscopy gold gold standard standard for colorectal cancer. The gold standard, or most Sigmoidoscopy Virtual Virtual colonoscopy colonoscopy CT CT colonoscopy colonoscopy accurate screening, is colonoscopy. The Barium Barium enema enema Fecal Fecal testing testing occult occult blood, blood, DNA DNA test test colonoscopy allows immediate biopsy and Recommended age, age, frequency vary vary by by test test and and family history immediate excision of any lesions. However, it s also most expensive and is very dependent on patient preparation and practitioner expertise. Sigmoidoscopy examines most distal part of colon. While this screening tool allows immediate biopsy and excision, it can only assess part of colon and misses polyps that are beyond its range. Virtual colonoscopies, commonly called CT colonoscopies and barium enemas both require same bowel preparation as colonoscopy; however, re is no ability to do biopsies or excisions during procedures. These screenings involve significant radiation exposure. Fecal occult blood testing (FOBT) is an easy, inexpensive, and non-invasive screening tool. Fecal testing is, however, less sensitive and specific than or tests we have mentioned. Occult blood may be due to or health conditions, leading to many false positive results. New fecal DNA tests remain less sensitive than or tests mentioned, but y have fewer false positive test results than FOBT. Recommendations for age of initial screening and frequency of screening vary by test and family history. Slide 21: Case Study You will find a case study on Colorectal Cancer Practice evaluation of of diagnostic test test characteristics and and screening programs Screening in small group activities section of Discuss prevention concepts this module. This case study, used in classes Apply this this at at patient and and across United States, will allow you to population level level practice evaluating diagnostic tests and screening programs. You will also discuss concepts related to preventive medicine. You will be able to apply se concepts at individual patient and population levels. 8

9 Slide 22: Newborn Screening Newborn screening is mandatory in all states. It Mandatory universal screen for for disorders, including metabolic, hormonal, hematologic, and and infectious conditions looks for serious metabolic, hormonal, States States vary vary in in what what diseases y y test test for for hematologic, and infectious conditions. States Heel Heel prick prick blood blood test test hours hours post post birth birth --if if done done too too early, early, metabolic disease may may not not show show up up in in blood blood vary widely in what diseases y test for and Family history may may indicate need need for for additional screens how many. For example, NY tests for over 40 conditions, including HIV. Newborn screening involves a heel prick blood test done hours after birth. Most states now use a tandem mass spectrometer to test blood, which tests for many metabolic conditions with one drop of blood. The 24 to 48-hour window of time is important. If testing is done too early, presence of diseases may not show up in baby s blood. Some of more common screens include: phenylketonuna(pku), galactosemia, medium-chain acylcarnitine deficiency (MCAD), sickle cell anemia, and HIV. In addition to a state s mandated tests, family history or ethnicity may indicate a need for additional screens. Non-mandatory screens are not paid for by state. They may or may not be covered under a family s personal health insurance. Slide 23: Evaluation of Screening Tests Now, we will briefly discuss how screening tests are evaluated for use. 9

10 Slide 24: Evaluating Tests Two central concepts in evaluation of tests are Reliability and and validity are arecentral concepts in in evaluating tests tests validity and reliability. Validity is defined as how Distinction between reliability and and validity well test result corresponds to true Reliability: Reliability: consistency consistency of of test test at at different different times times or or under under differing differing conditions conditions condition of patient, i.e. wher or not Validity: Validity: how how well well test test distinguishes distinguishes between between who who has has disease disease and and who who does does not not person has disease. Reliability, on or hand, evaluates how consistent or reproducible Fortune & Reid: 1998; Jekel et al:, 1996 Fortune & Reid: 1998; Jekel et al:, 1996 test results are over time or under different testing conditions. For this module, we will not discuss validity in detail, as concepts are more relevant to research studies than clinical tests. To learn more about validity, see Resources Section. Slide 25: Characteristics of a Screening Test Testing both validity and reliability is important. VALIDITY and RELIABILITY You can have a test that is quite reliable over time and across groups, but that isn t valid. It misses mark. Likewise, you can have a test that is valid, but not very reliable for certain groups or in certain testing situations. We want Fortune & Reid: 1998 Fortune & Reid: 1998 screening tests that are both valid measures of what is being tested and reliable in various groups and at all times. Slide 26: Reliability Also Also known as as consistency Ability to to yield yield same same results with with repeated measurements of of same same construct Degree to to which results are are free free from from random error error Jekel: 1996; Al-Eisa: 2009 Jekel: 1996; Al-Eisa: 2009 Reliability, commonly referred to as consistency, is ability of a test to yield same results with repeated measurements. Put anor way, reliability is degree to which results are free from error across testing occasions. If we have a large random error in a test, we see decreased reliability. 10

11 Slide 27: Common Types of Reliability Intra-subject Intra-subject Jekel: 1996; Al-Eisa: 2009 Jekel: 1996; Al-Eisa: 2009 Intra-subject reliability refers to consistency of measurement scores taken on same subject across testing occasions. We are evaluating degree of change in subject performance on test from one time to anor. Intra-subject Intra-subject Intra-rater Intra-rater Slide 28: Common Types of Reliability Intra-rater reliability refers to consistency of measurements taken by same tester on two or more testing occasions. Jekel: 1996; Al-Eisa: 2009 Jekel: 1996; Al-Eisa: 2009 Intra-subject Intra-subject Inter-rater Inter-rater Intra-rater Intra-rater Slide 29: Common Types of Reliability Inter-rater reliability is considered one of most important indices of reliability for screening tools. Here we are looking at consistency of measurement scores taken by two different testers. Jekel: 1996; Al-Eisa: 2009 Jekel: 1996; Al-Eisa: 2009 Slide 30: Common Types of Reliability Intra-rater Intra-rater Intra-subject Intra-subject Inter-rater Inter-rater Instrument Instrument Jekel: 1996; Al-Eisa: 2009 Jekel: 1996; Al-Eisa: 2009 Instrument reliability is anor important indicator of reliability. It refers to internal consistency of measurement tool itself. All of se forms of reliability serve an important purpose in evaluating screening and diagnostic tests. 11

12 Slide 31: Sensitivity We will now talk about sensitivity and specificity. Measures validity of of screening tests Ability to to identify those with disease correctly These concepts relate to validity, or how well a Minimizes false false negatives if if test test highly sensitive test result reflects reality. Sensitivity evaluates SNOUT Sensitive test test with with Negative result rules rules OUT OUTdisease ability to find patients who do have disease. In or words, sensitivity is percent of people with disease who are correctly identified. A sensitive test will reduce or eliminate false negatives. False negatives are test results that suggest absence of disease when disease is in fact present. These are also called beta or Type II errors. We want to aim for high sensitivity when consequences of a missed diagnosis are serious. An example might be screening donated blood for presence of HIV. An easy way to remember sensitivity is SNOUT acronym. Sensitive test with Negative results rules OUT disease. Slide 32: Specificity Specificity evaluates ability to identify which Ability to toidentify those without disease correctly patients do not have disease. Specificity Minimizes false false positives if if test test highly specific seeks to reduce or eliminate false positives. False SPIN SPIN Specific test test with with Positive result rules rules IN IN disease positives are test results that suggest presence of disease when disease is not actually present. They are also commonly called alpha or type I errors. We want to aim for high specificity when re are serious adverse consequences for patients incorrectly identified as having disease. For example we want a high degree of specificity in breast tissue biopsy outcomes to confirm cancer before doing a mastectomy or starting chemorapy or radiation. If specificity for a test is low, we need follow-up testing to rule out false positives. An easy acronym to remember specificity is SPIN. Specific test with Positive results rules IN disease. We want screening tests to be both sensitive and specific. 12

13 PSA level Sensitivity Specificity PSA level Sensitivity Specificity Slide 33: Relationship Between Sensitivity and Specificity This table shows relationship between sensitivity and specificity in prostate-specific antigen (PSA) levels. For screening tests whose results fall along a continuum of values, such as PSA, re is a trade-off between specificity and sensitivity, based on where you set cutoff Morgan TO et al; NEJM, 1996 Morgan TO et al; NEJM, 1996 score. In setting cut-off point, we need to balance risks of Type I and Type II errors for a given disease. Slide 34: The 2x2 Table Test Test + + Test Test - - Disease Disease Present Present True True Positive Positive False False Negative Negative Disease Disease Absent Absent False False Positive Positive True True Negative Negative The next few slides will show you how to calculate sensitivity and specificity. The gold squares represent 2x2 table, while blue boxes are labels. To understand results from 2x2 table, we need information from test being evaluated and from gold standard (or reference test) results. Sensitivity DISEASE DISEASE Slide 35: Sensitivity Test Test + + Test Test - - Present Present True True positive False False negative Absent Absent False False positive True True negative To calculate sensitivity, or proportion of people with disease who test positive, fill in squares of 2x2 table with numbers with and without disease who tested positive True or negative. People with disease who test True positives Sensitivity= True True positives + false false negatives positive are true positives (TP). People with disease who test negative are false negatives (FN). People without disease and who test positive are false positives (FP). People without disease and testing negative are true negatives (TN). To calculate sensitivity, divide true positives by total number with disease, i.e. TP / (TP+FN). 13

14 Specificity DISEASE DISEASE Present Present Absent Absent True True positive False False positive Test Test + + False False negative True True negative Test Test - - True True negatives = Specificity True True negatives + false false positives Slide 36: Specificity To calculate specificity, use same 2x2 table, and divide true negatives by total number without disease, i.e. TN / (FP+TN). The specificity tells us likelihood that test will come back negative in someone without disease. Slide 37: Predictive Values Predictive value of a test is a measure of Positive predictive value value Negative predictive value value percentage of times result (wher positive or negative) is correct result. The percentage of all positive tests results that are true positives is positive predictive value. The percentage of all negative test results that are true negatives is negative predictive value. Predictive values are useful tools in clinical decision-making. They allow us to determine probability of patient having a disease based on test results. Slide 38: Positive Predictive Value (PPV) NOT NOT inherent characteristic of of a screening test test Percent of of positive tests tests that that are are truly truly positive If If test test result is is positive, what what is is probability that that patient has has disease? Is Is affected by by several factors Specificity & specificity of of screening test test Prevalence of of disease PPV is not an inherent characteristic of a screening test. PPV is affected by both specificity of test and disease prevalence. 14

15 Predictive Predictive Value Value Module 4: Screening Slide 39: Negative Predictive Value (NPV) NOT NOT inherent characteristic of of a screening test test Percent of of negative tests tests that that are are truly truly negative If If test test result is is negative, what what is is probability that that patient does does not nothave disease? NPV is also not an inherent characteristic of a screening test. NPV is also affected by disease prevalence. The NPV can tell us probability that patient is disease-free based on negative results. Sensitivity and and specificity are are constant for for a particular test Slide 40: Test Characteristics and Population Tested particular test Sensitivity and specificity are constant for a PPV PPV and and NPV NPV vary vary dramatically, depending on on prevalence of of target condition in in population tested particular test, but it s surprising and counterintuitive how dramatically PPV and NPV vary prevalence high high PPV, PPV, low low NPV NPV Low prevalence low low PPV, PPV, high high NPV NPV High depending on prevalence of condition in specific population being tested. In a group with low prevalence, a test will have a low PPV and a high NPV. In a group with high prevalence, a test will have a high PPV and low NPV. Slide 41: Predictive Value and Prevalence 100% 100% 80% 80% PVP PVP 60% PVN 60% PVN 40% 40% 20% 20% 0% 0% 0% 0% 10% 10% 20% 20% 30% 30% 40% 40% 50% 50% 60% 60% 70% 70% 80% 80% 90% 90% 100% 100% Here is a visual example of relationship between predictive values and prevalence. Prevalence 15

16 Slide 42: Predictive Values Now that we have done basic 2x2 tables, let s figure out how to calculate PPV. Slide 43: Positive Predictive Value HIV HIV Status Status Let s try some sample calculations to give you a 1,000 1,000 people people at at HIV-Positive (15) HIV-Negative (985) 1.5% Prevalence Prenatal Prenatal Clinic HIV-Positive (15) HIV-Negative (985) 1.5% Prevalence Clinic feel for how this works, using ELISA test to Positive True Positive (14) False Positive (99) Positive True Positive (14) False Positive (99) screen for HIV. The test s sensitivity and = 12.4% PPV ELISA = 12.4% PPV ELISA Result Result specificity are both over 90%. Let s start with a Negative False Negative (1) True Negative (886) Negative False Negative (1) True Negative (886) = 99.9% NPV = 99.9% NPV population with low prevalence only 1.5% -- for Sensitivity (95%) Specificity (90%) Sensitivity (95%) Specificity (90%) instance, a population of 1,000 patients at a prenatal clinic. Using ELISA in this population gives us a positive predictive value of 12% and a negative predictive value of 99.9%. Even among patients who test positive, probability that y have HIV is low. Put anor way, probability that se patients got false positive results and are not HIV-infected is quite high. Slide 44: Positive Predictive Value HIV HIV Status Status 1,000 1,000 people people at at HIV-Positive (60) HIV-Negative (940) 6% Prevalence STD HIV-Positive (60) HIV-Negative (940) 6% Prevalence STD Clinic Clinic Positive True Positive (57) False Positive (94) Positive True Positive (57) False Positive (94) = 37.75% PPV ELISA = 37.75% PPV ELISA Result Result Negative False Negative (3) True Negative (846) Negative False Negative (3) True Negative (846) = 99.6% NPV = 99.6% NPV Sensitivity (95%) Specificity (90%) Sensitivity (95%) Specificity (90%) Now let s look at a population with a high prevalence of HIV. Here we have a population of 1,000 patients at an STD clinic. The HIV prevalence is 6%. Using same ELISA test, positive predictive value is 37%, much higher than neonatal clinic population. The NPV remains high at 99.6%. 16

17 Slide 45: Positive Predictive Value HIV HIV Status Status 1,000 1,000 people people at at HIV-Positive (240) HIV-Negative (760) 24% Prevalence Clinic HIV-Positive (240) HIV-Negative (760) 24% Prevalence Clinic in in Zambia Zambia Positive True Positive (228) False Positive (76) Positive True Positive (228) False Positive (76) = 75% PPV ELISA = 75% PPV ELISA Result Result Negative False Negative (12) True Negative (684) Negative False Negative (12) True Negative (684) = 98.3% NPV = 98.3% NPV Sensitivity (95%) Specificity (90%) Sensitivity (95%) Specificity (90%) Moving to a setting with a very high HIV prevalence, for example in Zambia, has opposite effect. The PPV rises to 75%, and NPV falls slightly to 98.3%. The probability that se test-positive Zambian patients have HIV is quite high. Slide 46: Multiple Screening Tests Sometimes, it is prudent to use a series of tests Use Use of of different tests tests concurrently to to screen for for same same condition to screen for a condition. There may be no gold Example: Prenatal multiple marker screening for for Down Syndrome standard screening tool available for a condition. Measures Measures levels levels of of 33 biomarkers biomarkers in in mor s mor s blood: blood: AFP: AFP: alpha-fetoprotein, alpha-fetoprotein, protein protein produced produced by by fetus fetus The available tools may be too invasive, costly, hcg: hcg: human human chorionic chorionic gonadotropin, gonadotropin, hormone hormone produced produced by by placenta placenta or orwise impractical to use for an initial Estriol: Estriol: a a hormone hormone produced produced by by both both fetus fetus and and placenta placenta Results Results of of ALL ALL 33 tests tests increases increases sensitivity sensitivity and and specificity specificity screening. We may use multiple screening tests simultaneously or sequentially. Let s take screening for presence of Down syndrome in a fetus. A common approach is to simultaneously test for three key bio-markers in mor s blood. Each of se tests individually has low sensitivity and specificity. Toger, however, y provide a viable alternative to amniocentesis. Amniocentesis is a highly sensitive and specific test. However, it is also invasive and has a small but significant chance of causing a miscarriage. It is better suited as a follow-up diagnostic test than a widespread screening test. Slide 47: Multiple Screening Tests Gestational diabetes testing generally employs a Use Use of of two-stage screening to to target testing efforts Example: Early Early pregnancy gestational diabetes two-stage screening protocol. In first screening First First trimester trimester risk risk assessment identifies women women at at higher higher trimester, a risk assessment is done to identify risk risk of of gestational gestational diabetes diabetes Oral Oral Glucose Glucose Tolerance Tolerance Test Test (OGTT) (OGTT) right right away away for for those those women at risk for condition. The Oral whose whose first first screen screen indicates indicates high high risk risk Glucose Tolerance Test (OGTT) is n done for women identified as at risk. This two-stage protocol minimizes number of women who need Oral Glucose Tolerance Test. The OGTT is a time-consuming test. Women must stay at clinic or lab for several hours. Most women s risk of diabetes is low early in pregnancy. Women who are obese or have a prior history of gestational diabetes need to be screened earlier in pregnancy using OGTT. 17

18 Slide 48: Multiple Screening Tests Anor use for two-stage screening is designed Two-stage screening to to maximize predictive value value Example: HIV HIV screening in in suburban primary care care to maximize predictive values. Let s take an office Risk Risk assessment assessment questionnaire about about sexual sexual and and drug drug use use example of HIV screening in a suburban primary history history HIV HIV blood blood test test for for all all patients patients whose whose questionnaire care office. A risk assessment about sexual and indicates indicates risk risk factors factors for for HIV HIV infection infection drug use history is given to patients. Those with risk factors for HIV infection are n given a blood test. By using written questionnaire as a preliminary screening test and selecting higher risk population to receive blood test, number of false positive results is reduced, and PPV of test is increased. These two examples of sequential screening tests illustrate different benefits of sequential screening. Reducing need for more invasive screenings and increasing PPV are just two. Slide 49: Effectiveness of Screening Programs The next few slides discuss effectiveness of screening programs. Slide 50: Screening Effectiveness Evaluation Sensitivity and specificity alone are not sufficient Test Test characteristics (sensitivity & specificity) alone are are never sufficient for for a sound decision about to make a sound clinical decision about wher wher to to use use a screening test test Or screening considerations or not to use a screening test. We need to weigh Benefits Benefits vs. vs. risks risks Prevalence Prevalence of of target target condition condition or factors in terms of individual patient, Inconvenience Inconvenience healthcare system, and for society. We need Costs/resource Costs/resource expenditures expenditures Patient Patient values values and and cultural cultural norms norms to analyze wher benefits outweigh Guyatt: 2009 Guyatt: 2009 risks. We saw that amniocentesis carries significant risks and should be used only when or tests indicate a need for more intensive screening. Cancer screening in very old adults with a short life expectancy has lower benefit (and perhaps higher risks) than screening healthy younger people. Likewise, evidence is mounting that mammograms for women in year old age range provide limited benefits because prevalence of breast cancer in that age group is so low, and y carry potentially unacceptable risks of excess radiation exposure and false positives leading to stress and unnecessary invasive procedures. The level of inconvenience is anor factor to consider. Is screening conducted in a location that patient can easily get to? If screening protocol necessitates a follow-up, such as for PPD or HIV test, will patient be able to follow through? 18

19 We must also look at overall cost and resource needs to conduct a screening. Will it be covered by insurance? Are healthcare providers available for screening and any follow-up tests? Finally, does test fit with patients values or cultural norms? If a population firmly believes in a non-medicalized pregnancy experience, pregnant women from this group will be unlikely to agree to an amniocentesis, even if one is indicated. Likewise, some traditional Latinas or Muslim women may not follow through with breast or cervical cancer screenings due to cultural or religious beliefs. These are but a few considerations in designing a screening program. Slide 51: Study Design To assess diagnostic or screening tests, studies Assessing a screening/diagnostic test test Properties Properties & accuracy accuracy seek to expose operating properties or Comparison Comparison of of test test to to gold gold standard standard Most Most definitive definitive diagnostic diagnostic procedure procedure or or best best available available laboratory laboratory characteristics of test and assess how close a test test Not Not always always a a gold gold standard standard for for a a procedure procedure match se properties are to a gold standard USPSTF USPSTF recommended diagnostic test. Researchers seek to determine power of tool to differentiate between those with and those without a target condition. To study a screening test, we look at test s performance in a group of patients NOT known to have target condition, and compare it to performance of a gold standard test. Gold standard tests are tests that are considered to be diagnostic standard for target condition. (If re is no current gold standard test, researchers may have to follow patients over time to determine who develops symptomatic disease.) The results of both tests are compared. In evaluating se studies, we want to see one of two things. We want to see a great deal of similarity in results between two tests. Or, we want to see that test in consideration exceeds capacity of gold standard test to discriminate between presence and absence of condition. For example, we might ask about test performance of liquidbased pap for cervical cancer screening. A group of women with no indication of cervical cancer would be offered both liquid-based pap and standard Pap smear. The rate of positive, false positive, negative and false negative results would be compared for both liquid and standard Pap smear screens. Slide 52: US Preventive Services Task Force The US Preventive Services Task Force (USPSTF) produces thoroughly researched screening recommendations. Let s spend a few minutes talking about USPSTF and how it works. 19

20 Slide 53: USPSTF Activities Systematically reviews evidence of of effectiveness and and develops recommendations for for clinical preventive services Recommendations include: Screening Screening tests tests Counseling Counseling Preventive Preventive medications medications Courtesy of Diana Pettiti, USPSTF: 2010 Courtesy of Diana Pettiti, USPSTF: 2010 The USPSTF is charged with systematically reviewing evidence of effectiveness and developing recommendations for clinical preventive services. The recommendations span screening tests, counseling, and preventive medications. Slide 54: USPSTF Methodology The USPSTF uses a rigorous methodology. First Define analytic framework outcomes & questions Define and and retrieve relevant evidence analytic framework is defined, including Evaluate QUALITY of of studies (good, fair, fair, poor) poor) Synsize and and judge judge STRENGTH of of overall evidence desired outcomes and key questions to be asked. (convincing, adequate, inadequate) 5. What constitutes relevant evidence is 5. Determine BALANCE of of benefits and and harms Benefits Harms = Net Net Benefit 6. operationally defined. The evidence is n 6. Link Link recommendation to to judgment about net net benefits: Grades: A, A, B, B, C, C, D, D, I I (inadequate evidence) gared. The overall quality of each of se Courtesy of Diana Pettiti, USPSTF: 2010 Courtesy of Diana Pettiti, USPSTF: 2010 studies is evaluated individually. The evidence is n synsized and judged in aggregate. We want to see convincing or adequate evidence at very least. After this, Task Force determines balance of benefits and harms, based on evidence. From all of this information, a grade is assigned that links recommendations to judgments about net benefits. Grades may range from A to D, or an I may be assigned for inadequate evidence. Slide 55: Critical Appraisal Questions Do Do studies have have appropriate research design to to answer key key questions? Are Are existing studies high high quality? Are Are results of of studies applicable to to general US US primary care care population and and setting? Courtesy of Diana Pettiti, USPSTF: 2010 Courtesy of Diana Pettiti, USPSTF: 2010 In evaluating available evidence, Task Force asks a number of questions. Do studies have appropriate research design to answer key questions? Are existing studies of high quality? Finally, are study results applicable to general primary care population and setting? 20

21 Slide 56: Critical Appraisal Questions How How many many relevant studies have have been been done? How How large large are are studies? How How consistent are are results of of studies? Are Are re re or or factors that that help help us us assess certainty of of evidence? (e.g. (e.g. dose-response effects, biologic plausibility) Courtesy of Diana Pettiti, USPSTF: 2010 Courtesy of Diana Pettiti, USPSTF: 2010 The number of studies that have been done is evaluated, as well as size of studies. The Task Force assesses consistency of studies results. Finally, any or factors that can help assess certainty of evidence are weighed. Slide 57: Interpret Task Force Grading Certainty Certainty of of Net Net Benefit Benefit Magnitude Magnitude of of Net Net Benefit Benefit Substantial Substantial Moderate Moderate Small Small Zero/negative Zero/negative High High A B C D Moderate Moderate B B C D Low Low Insufficient Insufficient (I (I Statement) Statement) This slide shows us how to interpret Task Force grading system based on magnitude of net benefit. The task force uses letter grades for ir recommendations. Courtesy of Diana Pettiti, USPSTF: 2010 Courtesy of Diana Pettiti, USPSTF: 2010 Grade Grade Definition Suggestion for Practice Grade Grade Definition Suggestion for Practice A A USPSTF recommends service. USPSTF recommends service. There is high certainty that net benefit is substantial. Offer or provide this service. There is high certainty that net benefit is substantial. Offer or provide this service. B B The USPSTF recommends service. The USPSTF recommends service. There is high certainty that net benefit is moderate or re There is moderate is high certainty certainty that that net net benefit benefit is moderate is moderate or to re is moderate certainty that net benefit is moderate to Offer or provide this service. Offer or provide this service. substantial. substantial. C C USPSTF recommends against routinely providing USPSTF recommends against routinely providing service. There may be considerations that support providing service. There may be considerations that support providing service in an individual patient. Offer or provide this service only if re Offer are or or provide considerations this service only in if service in an individual patient. support re are of or offering considerations or providing in There is moderate or high certainty that net benefit is There is moderate or high certainty that net benefit is service support in of an individual offering patient. or providing small. small. service in an individual patient. D D USPSTF recommends against service. USPSTF recommends against service. There is moderate or high certainty that service has no net There benefit is moderate or that or harms high certainty outweigh that benefits. service has no net benefit or that harms outweigh benefits. Discourage use of this service. Discourage use of this service. USPSTF concludes that current evidence is Read Clinical Considerations section USPSTF concludes that current evidence is Read Clinical Considerations section I I insufficient to assess balance of benefits and harms of insufficient to assess balance of benefits and harms of service. of USPSTF Recommendation Statement. of USPSTF If Recommendation offered service, Evidence service. is lacking, of poor quality, or conflicting, and Evidence is lacking, of poor quality, or conflicting, and patients Statement. should If offered understand service, uncertainty patients should about understand balance of balance of benefits and harms cannot be determined. balance of benefits and harms cannot be determined. benefits uncertainty and harms. about balance of Courtesy of Diana Pettiti, USPSTF: 2010 benefits and harms. Courtesy of Diana Pettiti, USPSTF: 2010 Slide 58: Communicating USPSTF Recommendations Here we see letter grades defined, along with suggested clinical practices. 21

22 Slide 59: Comparison of Screening Tools Sometimes, re is only one effective test to Newer tests tests vs. vs. gold gold standard Best Best tests tests for for your your population screen for a condition. However, often we have Validity Validity in in your your population population Accessibility Accessibility choices, such as wher to use a newer test or Cost Cost existing gold standard. It s important to Capacity Capacity of of local local health health care care system system Need Need a a system system in in place place to to be be able able to to screen screen AND AND to to assess what are best tests for target deal deal with with positive positive results results population. Tests should be valid for your population. A test should have been normed for population you are working with. Tests should be accessible. This means that tests should be covered under health insurance, and affordable. When we talk about accessibility, we look at or issues, like language barriers and overall literacy. We also look at privacy issues, particularly with tests that elicit sensitive information that could stigmatize a patient if information leaked out. The healthcare system must be able to support use of a screening test or program. If we screen for colorectal cancer using FOBT, we need to ensure re is a colonoscopy provider in area for follow-up testing. If we recommend colonoscopy as primary screen, we need to have many more colonoscopists available. We also need to ensure that re are sufficient resources for follow-up treatment. We ask where patients will get follow-up care, wher y can get re, and if care is affordable to m. In sum, re must be a system in place that can deal with positive results. Slide 60: Testing a Test Cervical This diagram illustrates one way to test a Cervical Cancer Cancer Liquid-based Liquid-based Pap Pap smear smear NO screening tool. We start out with patients NEW NEW test NO test Cervical Cervical Cancer Patients Patients without Cancer without evidence evidence of of without any evidence of a target condition. In cervical cervical cancer cancer Standard Cervical Cervical Cancer Standard Cancer this case, let s use example of cervical Pap Pap smear smear GOLD GOLD STANDARD NO cancer. We will compare a relatively new test; STANDARD NO Cervical Cervical Cancer Cancer liquid based Pap smear, with our gold standard, standard Pap smear. We n calculate specificity and sensitivity of both tests. It s quite important to start with patients showing no evidence of target condition. The sensitivity and specificity characteristics of test could be different in asymptomatic vs. symptomatic patients. Screening tests are meant to be used on asymptomatic patients. 22