Chapter 10. Screening for Disease

Transcription

1 Chapter 10 Screening for Disease 1

2 Terminology Reliability agreement of ratings/diagnoses, reproducibility Inter-rater reliability agreement between two independent raters Intra-rater reliability agreement of the same rater with him/herself Validity ability to discriminate without error Accuracy a combination of reliability and validity 2

3 Inter-Rater Reliability Two independent raters Cross-tabulate Observed proportion in agreement NOT adequate because a certain amount of agreement is due to chance Rater B Rater A + Total + a b g 1 c d g 2 Total f 1 f 2 N 3

4 Kappa (κ) Rater B Rater A + Total + a b g 1 c d g 2 Total f 1 f 2 N p p o a d = [Observed agreement; not chance corrected] N f g f g = [Expected agreement due to chance] N e 2 p o 1 p p e e [Agreement corrected for chance] 4

5 κ Benchmarks 5

6 Example 1: Flip two coins To what extent are results reproducible? Toss B Toss A Heads Tails Total Heads Tails Total p p o a d = =.5 [Overall agreement is 50%] N 100 f1g1 f2g N 100 po pe [no extra agreement above chance] 1 p 1.5 e 2 2 e 6

7 exp Example 2 To what extent are these diagnoses reproducible? Rater B Rater A + Total Total p p obs a d = =.9100 N 100 f g f g N exp 2 2 pobs pexp p substantial agreement 7

8 10.3 Validity Compare screening test results to a gold standard ( definitive diagnosis ) Each patient is classified as either true positive (TP), true negative (TN), false positive (FP), or false negative (FN) Test D+ D Total T+ TP FP TP+FP T FN TN FN+TN Total TP+FN FP+TN N 8

9 Sensitivity Test D+ D Total T+ TP FP TP+FP T FN TN FN+TN Total TP+FN FP+TN N SEN proportion of cases that test positive SEN TP those w/disease TP TP FN 9

10 Specificity Test D+ D Total T+ TP FP TP+FP T FN TN FN+TN Total TP+FN FP+TN N SPEC proportion of noncases that test negative SPEC TN those w/out disease TN TN FP 10

11 Predictive Value Positive Test D+ D Total T+ TP FP TP+FP T FN TN FN+TN Total TP+FN FP+TN N PVP proportion of positive tests that are true cases PVP TP those who test positive TP TP FP 11

12 Predictive Value Negative Test D+ D Total T+ TP FP TP+FP T FN TN FN+TN Total TP+FN FP+TN N PVN proportion of negative tests that are true non-cases PVN TN those who test negative TN TN FN 12

13 Prevalence Test D+ D Total T+ TP FP TP+FP T FN TN FN+TN Total TP+FN FP+TN N [True] prevalence = (TP + FN) / N Apparent prevalence = (TP + FP) / N 13

14 Conditional Probability Notation Pr(A B) the probability of A given B For example Pr(T+ D+) probability test positive given disease positive = SENsitivity SPEC Pr(T D ) PVP = Pr(D+ T+) PVN= Pr(D T ) 14

15 Example Low Prevalence Population Conditions: N = 1,000,000; Prevalence =.001 T+ T D+ D Total Total ,000,000 Prevalence = (those with disease) / N Therefore: (Those with disease) = Prevalence N =.001 1,000,000 =

16 Example: Low Prevalence Population Number of non-cases, i.e., TN + FP D+ D Total T+ T Total ,000 1,000,000 1,000,000 1,000 = 999,000 16

17 Example: Low Prevalence Population Assume test SENsitivity =.99, i.e., Test will pick up 99% of those with disease T+ 990 T Total 1000 D+ D Total TP = SEN (those with disease) = =

18 Example: Low Prevalence Population It follows that: D+ D Total T+ 990 T 10 Total 1000 FN = = 10 18

19 Example: Low Prevalence Population Suppose test SPECificity =.99 i.e., it will correctly identify 99% of the noncases T+ D+ D Total T 989,010 Total 999,000 TN = SPEC (those without disease) = ,000 = 989,010 19

20 Example: Low Prevalence Population It follows that: D+ D Total T+ 9,990 T 989,010 Total 999,000 FPs = 999, ,010 = 9,900 20

21 Example: Low Prevalence Population It follows that the Predictive Value Positive is : D+ D Total T ,990 10,980 T , ,020 Total ,000 1,000,000 PVPT = TP / (TP + FP) = 990 / 10,980 = Strikingly low PVP! 21

22 Example: Low Prevalence Population It follows that the Predictive Value Negative is: D+ D Total T ,990 10,980 T , ,020 Total ,000 1,000,000 PVNT= TN / (all those who test negative) = / =

23 Example: High prevalence population Same test parameters but used in population with true prevalence of.10 D+ D Total T+ 99,000 9, ,000 T 1, , ,000 Total 100, ,000 1,000,000 Prev = / 1,000,000 = 0.10 SEN = / 100,000 = 0.99 SPEC = 891,000 / 900,000 =

24 Example: High prevalence population An HIV screening test is used in one million people. Prevalence in population is now 10%. SEN and SPEC are again 99%. D+ D Total T+ 99,000 9, ,000 T 1, , ,000 Total 100, ,000 1,000,000 Prevalence = / 1,000,000 = 0.10 PVP = 99,000 / 108,000 = 0.92 PVN = 891,000 / 892,000 =

25 PVPT and Prevalence As PREValence goes down, PVPT is affected Figure shows relation between PVP, PREV, & SPEC (test SEN = constant.99) 25

26 Screening Strategy First stage high SENS (don t want to miss cases) Second stage high SPEC (sort out false positives from true positives) 26

27 Selecting a Cutoff Point There is often an overlap in test results for diseased and non-diseased population Sensitivity and specificity are influenced by the chosen cutoff point used to determine positive results Example: Immunofluorescence test for HIV based on optical density ratio (next slide) 27

28 Low Cutoff High sensitivity and low specificity 28

29 High Cutoff Low sensitivity and high specificity 29

30 Intermediate Cutoff moderate sensitivity & moderate specificity 30