Epidemiologic Methods for Evaluating Screening Programs. Rosa M. Crum, MD, MHS Johns Hopkins University

Transcription

1 This work is licensed under a Creative Commons Attribution-NonCommercial-ShareAlike License. Your use of this material constitutes acceptance of that license and the conditions of use of materials on this site. Copyright 2008, The Johns Hopkins University and Rosa M. Crum All rights reserved. Use of these materials permitted only in accordance with license rights granted. Materials provided AS IS ; no representations or warranties provided. User assumes all responsibility for use, and all liability related thereto, and must independently review all materials for accuracy and efficacy. May contain materials owned by others. User is responsible for obtaining permissions for use from third parties as needed.

2 Epidemiologic Methods for Evaluating Screening Programs Rosa M. Crum, MD, MHS Johns Hopkins University

3 Learning Objectives Discuss general principles guiding the introduction of screening programs Describe general principles for evaluating screening programs Define biases that may affect the apparent outcome of screening programs Explain the principles of early detection 3

4 Section A Why Is Screening So Controversial?

5 Why Is Screening So Controversial? The public expects screening to work Screening costs may be high; follow-up may involve risks The evidence for many screening modalities is mixed Expert panels often give differing views 5

6 Early Lung Cancer Action Project Enrolled 1,000 volunteers Symptom free Aged 60 or older At least 10 pack-years of cigarette smoking No previous cancer Purpose: to evaluate baseline and annual low-dose CT screening in people at high risk for lung cancer Source: Henschke et al. (1999). Lancet, 354:

7 Frequency of Detection of NCNs Frequency of Detection of NCNs on Baseline Chest Radiography and Low-dose CT Number of participants (n=1,000) Result on low-dose CT 1 NCN 2 3 NCN 4 6 NCN Negative Total Result on chest cardiograph 1 NCN NCN NCN Negative Total ,000 NCN non-calcified nodules Source: Henschke et al. (1999). Lancet, 354:

8 Distribution of Tumors by Size and Stage Stage NCN 2 5 mm NCN 6 10 mm NCN mm NCN mm Total IA 1 12 (1) 8 (2) 1 (1) 22 (4) IB IIA 0 1 (1) (1) IIB IIIA (2) 2 (2) IIIB Total 1 (0) 14 (2) 8 (2) 4 (3) 27 (7) NCN non-calcified nodules * Size on low-dose CT (numbers in parentheses are those by chest radiography) Source: Henschke et al. (1999). Lancet, 354:

9 CT Screening for Lung Cancer Source: Swensen et al.(2005). Radiology, 235:

10 CT Screening for Lung Cancer False-Positive Rates for Lung Cancer Nodule type Yes No Presence of prevalence cancers False-positive rate (%) Yes Presence of incidence cancers No False-positive rate (%) All nodules Nodules > 4mm Source: Swensen et al.(2005). Radiology, 235:

11 No Support for Lung Cancer Screening The United States Preventive Services Task Force last year concluded that current data do not support screening for lung cancer with any method Source: J. Brody. (August 16, 2005.) What an Extra Eye on Cancer Can Do for You. The New York Times. 11

12 No Support for Lung Cancer Screening Source: By S. Woloshin, L. Schwartz and H. G. Welch. (August 22, 2005.) Warned, But Worse Off. The New York Times. 12

13 No Support for Lung Cancer Screening Conclusions Screening for lung cancer may not meaningfully reduce the risk of advanced lung cancer or death from lung cancer. Source: Bach et al. (2007). JAMA, 297:

14 When Should a Screening Program Be Introduced? Test characteristics adequate Validity Reliability Predictive value Costs sufficiently low Evidence for benefit Mechanisms available for testing and follow-up 14

15 Screening Programs and Operational Measures Assessing the effectiveness of screening programs using operational measures 1. Number of people screened 2. Proportion of target population screened and number of times screened 3. Detected prevalence of preclinical disease 4. Total costs of the program 5. Costs per case found 6. Costs per previously unknown case found 7. Proportion of positive screenees brought to final diagnosis and treatment 8. Predictive value of a positive test in population screened Source: adapted from Hulka. (1998). 15

16 Screening Programs and Outcome Measures Assessing the effectiveness of screening programs using outcome measures 1. Reduction of mortality in the population screened 2. Reduction of case-fatality in screened individuals 3. Increase in percent of cases detected at earlier stages 4. Reduction in complications 5. Prevention of/reduction in recurrences or metastases 6. Improvement of quality of life in screened individuals 16

17 Problem of Establishing Sensitivity and Specificity Problem of establishing sensitivity and specificity because of limited follow-up of those who test negative Disease + Test + a c b d Further testing 17

18 Problem of Establishing Sensitivity and Specificity Problem of establishing sensitivity and specificity because of limited follow-up of those who test negative Disease + Test + a c b d Further testing Sensitivity a a + c = Specificity = d b + d 18

19 Problem of Establishing Sensitivity and Specificity Problem of establishing sensitivity and specificity because of limited follow-up of those who test negative Disease + Test + a c b d Further testing Positive predictive value a Negative = predictive = a + b value d c + d 19

20 Problem of Establishing Sensitivity and Specificity Problem of establishing sensitivity and specificity because of limited follow-up of those who test negative HIV + ELISA + a c b d Western blot Example: ELISA and HIV 20

21 Problem of Establishing Sensitivity and Specificity Problem of establishing sensitivity and specificity because of limited follow-up of those who test negative Prostate cancer + + a b TRU PSA c d Example: PSA and prostate cancer 21

22 Section B Natural History of Disease

23 Natural History of Disease 23

24 Natural History of Disease 24

25 Natural History of Disease 25

26 Natural History of Disease 26

27 Natural History of Disease 27

28 Critical Point A critical point is a point in the natural history of the disease before which therapy may be less difficult and/or more effective; or a lesser therapy may be needed If the disease is potentially curable, cure may be possible before this point but not after Or, there may not be a critical point 28

29 Multiple Critical Points in the Natural History of Disease 29

30 Possible Outcomes of a Screening Program 30

31 Trading off: Benefits versus Risks Benefits need to be assessed in context of potential risks of screening Risks include: False-negatives Potential harm of delayed diagnosis False-positives Risks and costs of work-up Potential harm of false-positive diagnosis 31

32 Section C Problems that Complicate Assessing Survival Improvement as a Result of Early Detection

33 Problems that Complicate Assessing Survival Improvement 1. Selection bias a. Referral (volunteer) bias 33

34 BCDDP Screening Age-Specific Incidence Rates and Relative Risks for White Females in Years 2 and 3 Following the Initial BCDDP Screening Age in Years BCDDP Rate SEER Rate Relative Risk

35 Problems that Complicate Assessing Survival Improvement 1. Selection bias a. Referral (volunteer) bias b. Length-biased sampling 35

36 Natural History of Disease 36

37 Natural History of Disease 37

38 Natural History of Disease 38

39 Natural History of Disease 39

40 Natural History of Disease 40

41 Natural History of Disease 41

42 Natural History of Disease 42

43 Natural History of Disease 43

44 Lead Time Lead time is the interval by which the time of diagnosis can be advanced by the screening procedure, as compared with the normal methods for detection and diagnosis 44

45 Problems that Complicate Assessing Survival Improvement 1. Selection bias a. Referral (volunteer) bias b. Length-biased sampling 2. Lead-time bias 45

46 Five-Year Survival and Lead-Time Bias 46

47 Five-Year Survival and Lead-Time Bias 47

48 Five-Year Survival and Lead-Time Bias 48

49 Five-Year Survival: Diagnosis Made Without Screening 49

50 Shift of Five-Year Period by Screening and Early Detection 50

51 Lead-Time Bias 51

52 Problems that Complicate Assessing Survival Improvement 1. Selection bias a. Referral (volunteer) bias b. Length-biased sampling 2. Lead-time bias 3. Over-diagnosis bias 52

53 Relationship of Improved Outcome to Early Detection Two assumptions underlying a relationship of improved outcome to early detection of disease 1. All or most clinical cases of a disease first go through a detectable preclinical phase 2. In the absence of intervention, all or most cases in a preclinical phase progress to a clinical phase 53

54 Natural History of Cervical Cancer 54

55 Natural History of Cervical Cancer 55

56 Section D Studies Designs to Evaluate Screening

57 How Do We Evaluate Screening? Randomized clinical trials Randomization to new screening modality or to usual care Observational approaches Cohort study design Case-control studies Stage shift Trend in mortality 57

58 Design of a Non-Randomized Comparison (Cohort) Study The Benefits of Screening Screened Not screened Die from disease Don t die from disease Die from disease Don t die from disease 58

59 Design of a Randomized Trial The Benefits of Screening Eligible population R a n d o m i z e d Screened Not screened Die from disease Don t die from disease Die from disease Don t die from disease 59

60 HIP Study of Screening First breast cancer screening trial Women aged enrolled in Health Insurance Plan (HIP) of Greater New York Started 1963 Followed participants for 18 years Sam Shapiro, professor emeritus, Health Policy and Management 60

61 HIP RCT on Periodic Screening for Breast Cancer The Benefits of Screening 62,000 women aged years and enrolled in HIP R a n d o m i z e d 31,000 screened 31,000 not screened 39 die from breast cancer 30,961 don t die from breast cancer 63 die from breast cancer 30,937 don t die from breast cancer 61

62 HIP RCT on Periodic Screening for Breast Cancer Health Insurance Plan Data: Groups Sizes (Rounded), Deaths in Five Years of Follow-Up, and Death Rates per 1,000 Women Randomized Study Group size Breast cancer All other Number Rate Number Rate Screened 20, Refused 10, Total 31, Control 31,

63 HIP RCT on Periodic Screening for Breast Cancer Die from breast cancer + Screened for breast cancer , ,937 63

64 Deaths Due to Breast Cancer Deaths Due to Breast Cancer Five Years of Follow-Up after Entry to Study 64

65 Five-Year Case Fatality Rates Five-Year Case Fatality Rates among Breast Cancer Patients 65

66 Mortality Rates from all Causes Except Breast Cancer Mortality Rates from all Causes Except Breast Cancer per 10,000 Person-Years, HIP 66

67 Controversies in Breast Cancer Screening Does it work? The HIP and other screening trials? Whom to screen? Women under 50 years of age? The NIH Consensus Conference and beyond Does it work all over again? The Nordic systematic review So, what do we know? 67

68 Outrage 68

69 Mammography Mammography: RCTs for Women Under 50 as of 1996 (Science, 1997) Study Effect 95% C.I. HIP Malmo Two County Edinburgh Canada Stockholm Gottenburg

70 2000 Nordic Study: Meta-Analysis of Cochrane Library Results: screening for breast cancer with mammography is unjustifiable Data show that for every 1,000 women screened biennially throughout 12 years: One breast cancer death is avoided Total number of deaths increases by six Source: P. Gøtzsche and O. Olsen. (2000). Lancet, 355:

71 Dutch Study: Mammography and Breast Cancer Mortality 2003: assess effect of mammography screening program on breast cancer mortality Results: routine mammography screening can reduce breast cancer mortality rates in women aged years Source: Otto et al. (2003). Lancet, 361:

72 Dutch Study: Mortality Rates 72

73 2003 Study: Two Swedish Counties 2003: assess long-term effect of screening on death from breast cancer in two Swedish counties Results: mammography screening is contributing to substantial reductions in breast cancer mortality in these two counties Source: Tabar et al. (2003). Lancet, 361:

74 Swedish Study: Cumulative Mortality Cumulative 20-Year Mortality from Incident Tumors Diagnosed in Women Aged Years 74

75 Size of Tumors at Diagnosis Mean and Median Size of Invasive Breast Cancer Tumors from for Age Groups in Rhode Island 75

76 SEER Breast Cancer Incidence, SEER Breast Cancer Incidence,

77 SEER Breast Cancer Mortality, SEER Breast Cancer Incidence,

78 How Do We Evaluate Screening? Randomized clinical trials Randomization to new screening modality or to usual care Observational approaches Cohort study design Case-control studies Stage shift Trend in mortality 78

79 Case-Control Study: Screening Sigmoidoscopy Screening sigmoidoscopy and mortality from colorectal cancer Purpose: estimate the efficacy of screening sigmoidoscopy Cases 261 Kaiser Permanente members who died of colorectal cancer from Controls 868 subjects matched for age and sex Compared use of screening during 10 years before diagnosis for cases with use of screening in controls Source: (March 5, 1992). N Engl J Med, 326(10):

80 Design of a Case-Control Study of the Benefits of Screening Screened in the past Never screened Screened in the past Never screened Advanced disease Controls Cases "Controls" 80

81 Case-Control Study Case-control study for estimating efficacy of screening sigmoidoscopy 91.2% 8.8% had had not undergone undergone screening screening 24.2% had undergone screening 85.8% had not undergone screening 261 dead from colorectal cancer: Cases 868 matched controls "Controls" 81

82 Case-Control Study Case-control study for estimating efficacy of screening sigmoidoscopy Cases Controls Underwent screening OR = (8.8)(75.8) =.30 (91.2)(24.2) Adjusted for confounders OR =.41 Source: (March 5, 1992). N Engl J Med, 326(10):

83 Ethical Issues in Screening for Disease Ethical issues in screening for disease 83