Clinical Decision Analysis

Size: px

Start display at page:

Download "Clinical Decision Analysis"

Ashlee Walters
5 years ago
Views:

1 Clinical Decision Analysis

2 Terminology Sensitivity (Hit True Positive) Specificity (Correct rejection True Negative) Positive predictive value Negative predictive value The fraction of those with the disease correctly identified as positive by the test. The fraction of those without the disease correctly identified as negative by the test. The fraction of people with positive tests who actually have the condition. The fraction of people with negative tests who actually don't have the condition. Likelihood ratio If you have a positive test, how many times more likely are you to have the disease? If the likelihood ratio equals 6.0, then someone with a positive test is six times more likely to have the disease than someone with a negative test. The likelihood ratio equals sensitivity/(1.0-specificity). The sensitivity, specificity and likelihood ratios are properties of the test. The positive and negative predictive values are properties of both the test and the population you test. If you use a test in two populations with different disease prevalence, the predictive values will be different. A test that is very useful in a clinical setting (high predictive values) may be almost worthless as a screening test. In a screening test, the prevalence of the disease is much lower so the predictive value of a positive test will also be lower

3 Decision Matrix Disease Present Disease Absent Test Positive True Positive (TP) Hit Sensitivity False Positive (FP) False Alarm Test Results Test Negative False Negative (FN) Miss True Negative (TN) Specificity Correct Rejection TP X 100 = Sensitivity (%) TP + FN TN X 100 = Specificity (%) FP + TN

4 Probability Distribution

5 Effect of d

6 Effect of d

7 Receiver Operating Characteristic (ROC) 100 F45 - SP- 80 Sensitivity (>630) False Positive

8 ROC & d

9 ROC & d

10 ROC & C

11 Gaussion With Equal Variance (GEV) Probability distribution curves of normal and diseased are not always having the same variance as in most audiological tests d varies as a function of criterion in these cases

12 ROC & Criterion

13 A' A' is a way of measuring the test performance A' is calculated from hit rate (HT) and false alarm rate (FA) To achieve a high A' score, a test must have both a high hit rate and a low false alarm A' varies from 0.5 for a useless test to 1.0 for a perfect test A' = (HT - FA) 4HT (1 + HT (1 - FA) - FA)

14 List of Abbreviations HT/FA (Pr[D/+]) (Pr[D/-]) EF d A PD Hit rate/false alarm rate Posterior probability of being correct with a positive test result Posterior probability of being correct with a negative test result Efficiency: percentage of correct test results Measure of test performance; larger d means better performance Measure of test performance; larger A means better performance Disease prevalence; percentage of test population with disease

15 Posterior Probability Also called Predictive value. There are two forms of posterior probability: Positive predictive value (Pr[D/+]): The percentage of people with a positive test result who actually have the disease. (Pr[D/ + ]) = 1+ 1 (FA) (1-PD) (HT) (PD) The percentage of people with a negative test who do NOT have the disease 1 (Pr[D/ ]) = (1-HT) (PD) 1+ (1-FA) (1-PD)

16 Efficiency (EF) Efficiency is the percentage of total test results that are correct EF = HT X PD + (1-FA) X (1- PD) The decimal form should be used for HT, FA, and PD It is a function of disease prevalence When disease prevalence is small, false alarms rate drives efficiency more than hit rate

17 Pr & EF for Different Tests

18 Pr & Prevalence Pr is highly dependent on the prevalence. One way to increase the prevalence is to use a screening test. E.g., Population of 1000 with a PD of 2% How many would have the disease? 20 Use a screening test with HT = 90% & FA = 10%; How many with the disease would fail? 18 (20 X 0.9) How may without disease would fail? 98 (980 X 0.1) How many total failure? 116 (98+18) What would be the new prevalence? 16% (18 of the 116 would have the disease)

19 Application To Audiological Tests Cochlear vs. retrocochlear

20 Comparisons of ROC Curves Hz 710 Hz 226 Hz Sensitivity False Positive

21 Test Protocols Protocol for Combining tests that have been developed to identify a same disease. This can improve the test performance There are two different approaches to combine the tests Parallel Series

22 Parallel T1 T2 Criterion Criterion defines how many individual tests should be positive for the protocol to be positive Strict: all tests should be positive Loose: only one test must be positive

23 In Series T1 Positive - series + + T2 - - Treatment In T1 + - Out Out Negative - series Treatment T2 + - Out

24 Performance Measure for a Parallel Test

25 Real example Patient SC TW Fr.Scrn. Fr.SF+ Fr.SF- Fr.SP+ Fr.SP- F45 -SF F45 -SP Total (+) SC Static compliance TW: Tympanometric width Fr.: Resonant frequency Scrn.: Automatic screening F45 : Frequency corresponding to 45 SF: Sweep frequency recording SP: Sweep pressure recording +: Positive tail compensation -: Negative tail compensation

26 Parallel dp =14 np =68 F45 HT = 79 FA = 29 TW HT =43 FA = 44 Using a loose criterion (i.e., Otosclerosis = positive on either TW or F45 -SF) resulted in a HT of 100% with A' of The test performance for this combination of measures exceeds the performance of any single measure.

27 Test Correlation Helps to determine the protocol performance Test correlation the tendency of individual tests in a protocol to identify the same patients as positive or negative Three type of correlation Maximum-positive (max-pos): identifying he same patients Maximum-negative (max-neg): identifying different patient Mid-point

28 Limits on Protocol Performance

29 Cost-Benefit Analysis Costs are assigned to errors Miss False alarms Benefits are assigned to correct decisions Hit Correct rejection

30 Cost-Benefit Analysis: Simple Analysis Cost Ratios: requires Hit rate False alarm Prevalence of disease Cost analysis Hit rate False alarm Prevalence of disease Financial cost of test administration

31 Cost-Benefit Analysis for Different Test Protocols MRI_G is a Gold standard P7 is series positive P8 is series negative P2-P6 are individual tests PD was considered 5% for a hypothetical group of 1000 individuals

32 Ranking using Cost Ratio Patients with + on the screening will receive the definitive test (hit & false alarm) Patients with - on the screening will not receive the definitive test (will be lost from the system-miss & correct rejection) Dobie method A definitive test is used We decide on the relative costs of (how many false alarms equal one miss)

33 Ranking using Cost Ratio Lets say we decide to accept up to 5 false alarms for every miss Relative cost difference is computed CST(X-Y)=(fsX-fsY)- R*(hsX-hsY) If CST is positive then the cost of X is greater than Y and therefore, Y is better Protocol A1 A2 A3 Cost CST(A1- A2) CST (A1- A3) Ht/FA 95/11 82/6 97/20 R= hs/fs 48/105 41/57 48/190 R= CST(A2- A3)

METHODS FOR DETECTING CERVICAL CANCER

Chapter III METHODS FOR DETECTING CERVICAL CANCER 3.1 INTRODUCTION The successful detection of cervical cancer in a variety of tissues has been reported by many researchers and baseline figures for the