Evidence Based Medicine: Articles of Diagnosis Duke University School of Medicine EBM Course March 5, 2013 Jamie Fox, MD David Ming, MD Departments of Internal Medicine and Pediatrics
Diagnosis
Diagnostic Tests Limit Uncertainty
Objectives To introduce key concepts for understanding diagnosis articles To briefly review the validity criteria for the appraisal of diagnosis articles To understand the concepts behind core statistical measures seen in diagnosis articles To practice calculation of core statistical measures through review of real world examples
DIAGNOSIS ARTICLES: CORE CONCEPTS
Pre test Probability Probability Estimates The probability of the target condition being present before the results of a diagnostic test are known. Post test Probability The probability of the target condition being present after the results of a diagnostic test are known.
Pre test Probability Estimate made prior to testing of how likely it is a patient has a disease Where does this come from? Clinical judgment after H&P (and other tests) Prevalence of disorder in your population Epidemiologic data Clinical manifestations of disease articles Differential diagnosis articles
Diagnostic Tests Spur Action Zone of Action Zone of Uncertainty Zone of Action
Test Threshold Aka, Observation Threshold Aka, Do Nothing Threshold How certain you d like to be that a patient does not have a diagnosis before you are comfortable with holding treatment and/or doing no further testing Depends on: Risk of not treating Risk/cost/burden/scarcity of therapy Patient preferences
Treatment Threshold How certain you d like to be of a diagnosis before proceeding with treatment Depends on: Risk of not treating Risk/cost/burden/scarcity of therapy Patient preferences
DIAGNOSIS: CASE DISCUSSION
THE PATIENT The 5 A s ASSESS APPLY ASK Evidence based Medicine Cycle APPRAISE ACQUIRE
Case Scenario 25 yo F with h/o panic anxiety disorder p/w acute onset pleuritic chest pain, headache, and dyspnea No meds except for OCP (started 2 years ago) and PRN xanax for panic attacks Recent weekend trip to the beach (~5 hr drive R/T) Exam: HR 104, RR 22, and anterior chest wall TTP
What should we do next? 1. Nothing 2. D Dimer 3. Imaging 4. Anticoagulation 0% 0% 0% 0% 1 2 3 4
0% 0% 0% 0% 0% 0% 0% 0% What is your best estimate (probability) that she has a PE? 0% 1. 1 10% 2. 11 20% 3. 21 30% 4. 31 40% 5. 41 50% 6. 51 60% 7. 61 70% 8. 71 80% 9. >80%
Case Scenario #2 68 yo M with h/o prostate CA currently receiving chemotherapy/xrt p/w CP, headache, and dyspnea Discharged 3 weeks ago following radical prostatectomy Has a h/o panic anxiety and under significant stress since yesterday due to wife being hospitalized for hip fracture Also has a h/o severe GERD for which he s on PPI BID Exam: anxious appearing; HR 106; BP 140/85; RR 22; 100% RA; minimal anterior chest wall TTP; remainder nl
0% 0% 0% 0% 0% 0% 0% 0% What is your best estimate (probability) that he has a PE? 0% 1. 1 10% 2. 11 20% 3. 21 30% 4. 31 40% 5. 41 50% 6. 51 60% 7. 61 70% 8. 71 80% 9. >80%
What should we do next? 1. Nothing 2. D Dimer 3. Imaging 4. Anticoagulation 0% 0% 0% 0% 1 2 3 4
Pulmonary Embolism (PE): Why it Matters PE is common 1 2 cases per 1000 people per year 200 300K hospitalizations annually These are conservative estimates of incidence PE is deadly Mortality rate of ~30% 50 100K people die annually from PE Arch Intern Med 2003;163:1711 JAMA. 2003;290:2849 2858
Pulmonary Embolism: Why it Matters Timely treatment saves lives PE mortality reduced to 2.5% with treatment Timely diagnosis essential to initiate treatment NEJM 1992;326:1240 1245
PE: A Diagnostic Dilemma Why diagnosis of PE is problematic: Clinical signs and symptoms are non specific Many patients suspected of PE don t have it No single, simple, reliable diagnostic lab test exists Over and under diagnosis associated with harms Underdiagnosis M&M Overdiagnosis unnecessary and costly diagnostic testing w/potential for substantial harms/side effects Ann Intern Med. 2011;155:448 460
PE: A Diagnostic Dilemma Accurate diagnosis of PE relies upon appropriate combined use of: Clinical assessment Clinical decision rules Laboratory testing D Dimer Imaging VQ scan Chest CTA Goal In patients suspected of PE, how can we safely and efficiently rule out the diagnosis? Ann Intern Med. 2011;155:448 460
PE: Thresholds for Action What is your threshold for action? Test (Observation) Threshold Treatment Threshold
Who certain do you have to be that your patient does not have a PE before you let them go home without any further testing or treatment? 0% 1. 1 5% 0% 0% 0% 0% 2. 5 10% 3. 10 20% 4. 20 30% 5. >30%
How certain do you have to be that your patient has a PE before you start treatment (anticoagulants) without additional testing? 0% 1. <20% 0% 0% 0% 0% 0% 0% 2. 20 30% 3. 30 50% 4. 50 75% 5. 75 90% 6. 90 95% 7. >95%
To the Literature Shall we PICO(T)? P I C O T Adult with clinically suspected PE Clinical decision rule and/or D dimer Imaging or Angiography Test characteristics Diagnosis article
Study Architect Validity criteria for a Diagnostic Test Article 1. Physicians faced diagnostic uncertainty 2. Every patient underwent reference (gold) standard http://www.crutcherstudio.com/architect_services.htm 3. The test being evaluated didn t influence the decision to perform reference (gold) standard
Searching.......
JAMA. 2006;295:172 179
Christopher Study Overview Design Prospective cohort 12 centers in The Netherlands Patient Population Adult patients in ED or inpatient setting with clinically suspected PE (n=3306) Clinical suspicion for PE defined as Sudden onset dyspnea Sudden onset pleuritic CP without another apparent cause Sudden worsening of existing dyspnea
Christopher Study Overview Intervention: Diagnostic multi modal algorithm 2. D Dimer 1. Simplified Well s Score 3. CTA Chest Primary outcome: Symptomatic VTE rate at 3 months (fatal PE, nonfatal PE, DVT)
Background Wells Criteria Validated clinical decision rule that aids in diagnosis of PE Helps establish pre test probability for PE using clinical s/sx alone PE risk categories by Wells score: Low probability <2 pts Intermediate probability 2 6 pts High probability >6 pts
Back to Our Case: Assign a Wells Score Case #1 Wells score = 1.5 (tachycardia) LOW Probability Case #2 Well s score = 4 (tachycardia, recent surgery, active cancer) MODERATE probability
Clinical Questions What is the diagnostic accuracy of the following tests when evaluating patients suspected of PE? Clinical prediction rule (e.g., Well s score) D dimer Imaging (CTA, VQ scan, pulmonary angiogram)
DIAGNOSIS MATH: SENSITIVITY AND SPECIFICITY
A Few Ground Rules No jargon Define stats using real English No formula memorization Focus on the principles behind the calculations
Ready your calculators Sensitivity Specificity Likelihood Ratios Positive (+LR) Negative ( LR)
The 2x2 Table!!
The Truth Allah, Buddha, God Reference Standard Test + + True Positive False Positive Test False Negative True Negative
Sensitivity Of all patients with disease, the proportion with a positive test True Positive True Positive + False Negative Disease + Test + True Positive False Positive Positive in Disease PID False Negative True Negative
Sensitivity: Of all patients with disease, proportion with a positive test True Positive True Positive + False Negative + PE + 266 883 D Dimer 5 1023 266 271 = 98.2%
Specificity Of patients without disease, the proportion with a negative test True Negative True Negative + False Positive Disease + Test + True Positive False Positive Negative in Health NIH False Negative True Negative
Specificity: Of patients without disease, proportion with a negative test. True Negative True Negative + False Positive + PE + 266 883 D Dimer 5 1023 1023 1906 = 53.7%
Limitations of Studies Reporting Sensitivity and Specificity Patients have already received the gold standard test in addition to the diagnostic test in question Not the case in real world usually only one test ordered SN and SP describe a patient population Not necessarily applicable to each individual patient Final diagnosis already known in a study when data is analyzed Does not mimic real world where diagnosis still in question at the time of ordering a test
Limitations of Studies Reporting Sensitivity and Specificity Our question isn t really: If my patient has a PE, what is her chance of having a (+)Ddimer? Really what we want to know is: If my patient has a (+) D dimer what is the chance she has a PE? OR If my patient has a ( ) D dimer what is the chance she has a PE?
LIKELIHOOD RATIOS
Likelihood Ratios better than Sensitivity and Specificity Combine components of SN and SP Portable calculation Applicable for individual patients Useful at the bedside Help to move us into zones of action
The 2 nd Most Important Slide in this Talk Are you ready? Are you sure you re ready? Here it comes wait for it.
The Most Important Slide in this Talk 1. A Ratio is a Ratio is a Ratio 2. Ratios use DIVISION 3. A Ratio can only equal one of 3 things: > 1 < 1 = 1
Likelihood Ratio Concept How likely is it that a patient with disease will have a given test result, compared to how likely is it that a patient without disease will have the SAME test result? Both positive and negative likelihood ratios DISEASE compare the likelihood of the SAME test result in the presenceno of disease DISEASE versus in the absence of disease
Likelihood Ratio Concept Likelihood Ratio L1 Test Result Disease (+) (LR) = = L2 Test Result Disease ( ) LR > 1 Test result more likely present in Dz (+) LR < 1 Test result more likely present in Dz ( ) LR = 1 Test results equally likely in Dz (+) and ( )
Positive vs Negative LR Concept Positive Likelihood Ratio Proportion of patients with disease with positive test Proportion of patients without disease with positive test Negative Likelihood Ratio Proportion of patient s with disease with negative test Proportion of patient s without disease with negative test
Positive vs Negative LR Concept Positive Likelihood Ratio Proportion of patients who test positive WITH disease Proportion of patients who test positive WITHOUT disease Negative Likelihood Ratio Proportion of patients who test negativewith disease Proportion of patients who test negative WITHOUT disease
Positive vs Negative LR Practice Positive Likelihood Ratio Proportion of patients with Strep throat with positive rapid strep Proportion of patients without Strep throat with positive rapid strep Negative Likelihood Ratio Proportion of patient s with strep throat with negative rapid strep Proportion of patient s without strep throat with negative rapid strep
Positive Likelihood Ratio Proportion of patients with positive rapid strep among all patients WITH strep throat Proportion of patients with positive rapid strep among all patients WITHOUT strep throat Negative Likelihood Ratio Proportion of patients with negative rapid strep among all patients WITH strep throat Proportion of patients with negative rapid strep among all patients WITHOUT strep throat
Likelihood Ratio continuous variables Compares a given test result in the presence of disease versus in the absence of disease LR for WBC > 15 in diagnosing appendicitis Proportion of patient s withappendicitis with WBC > 15 Proportion of patient s withoutappendicitis with WBC > 15
Test = rapid flu Disease = influenza LR+ Proportion of patients with Influenza with positive rapid flu Proportion of patients without Influenza with positive rapid flu LR Proportion of patient s with influenza with negative rapid flu Proportion of patient s without influenza with negative rapid flu
Test = high prob VQ Disease = PE LR high prob VQ Proportion of patients with PE with high prob VQ Proportion of patients without PE with high prob VQ
Definitions Likelihood ratio of a positive test: Proportion of patients with disease who have a positive test compared to the proportion of patients without disease who have a positive test. TP TP + FN FP FP + TN Sensitivity 1 specificity Test + Disease + TP FN FP TN
Quick Understanding Check Do you want the LR+ to be? As high as possible As close to 1 as possible As low as possible
Definitions Likelihood ratio of a negative test: Proportion of patients with disease who have a negative test compared to the proportion of patients without disease who have a negative test. FN TP + FN TN FP + TN 1 sensitivity Specificity Test + Disease + TP FN FP TN
Quick Understanding Check Do you want the LR to be? As high as possible As close to 1 as possible As low as possible
LR = 0.01 Less Likely LR = 0.1 Less Likely LR Impact LR = 0.2 Less Likely Increasing impact increasing impact 0 LR = 1 LR = 5 More Likely No impact on likelihood of disease LR = 10 More Likely LR = 100 More Likely
LR Impact One, Five, Ten Rule 1 Not useful (no effect) 5 10 0.2 (1/5) 0.1(1/10) Moderately useful >10 <0.1 Very useful
Likelihood Ratio Practice Our Study LR(+): Proportion of low risk patients ( PE unlikely ) with PE who have a positive D dimer compared to the proportion of low risk patients without PE who have a positive D dimer. PE + + 266 883 D Dimer 5 1023 266 271 883 1906 = 2.1
Likelihood Ratio Practice Our Study LR( ): Proportion of low risk patients ( PE unlikely ) with PE who have a negative D dimer compared to the proportion of low risk patients without PE who have a negative D dimer. PE + + 266 883 D Dimer 5 1023 5 271 1023 1906 = 0.03
Likelihood Ratio Advantages Review 1. Can apply to individual patients 2. Incorporates test and treatment thresholds 3. Can calculate for different cut offs of test result
LR Nomogram
Back to Case #1 Low pre test probability for PE Tests negative for D Dimer Questions: What is our post test probability for PE? What should we do next?
Let s use the LRs http://www.cebm.net/index.aspx?o=1161 http://araw.mede.uic.edu/cgi bin/testcalc.pl
Back to Case #1 Again Low pre test probability for PE Tests positive for D Dimer Questions: What is our post test probability for PE? What should we do next?
PE Imaging and Associated LRs VQ Normal LR = 0.1 Low probability LR = 0.39 Intermediate probability LR = 1.1 High probability LR = 17 CTA Positive LR = 19.7 Negative LR = 0.18 JAMA 1990;263:2753 NEJM 2006;354:2317
Group Practice 6 groups by imaging result Pre imaging probability Case #1 (+D dimer) = Case #2 = Calculate new post test probability for your group s imaging result for cases #1 and #2 using the provided likelihood ratios Likelihood Ratios Normal VQ = 0.1 Low prob VQ = 0.39 Intermed prob VQ = 1.1 High prob VQ = 17 Positive CTA = 19.7 Negative CTA = 0.18
Questions?
Summary To introduce key concepts critical for understanding diagnosis articles Diagnostic tests move us from a zone of uncertainty to zones of action (crossing the test or treatment threshold) Always assess the pre test probability To summarize the validity criteria for the appraisal of diagnosis articles Did every patient receive the diagnostic test under study and an appropriate reference (gold) standard? To understand concepts behind key statistical measures seen in diagnosis articles Truth lies in the heavens above always set up the 2x2 table the same way A Ratio is A Ratio is A Ratio, ratios use division, and all ratios are >1/<1/=1 Remember the plain English definitions for sensitivity, specificity, and LR High +LR and Low LR are best To practice calculating core statistical measures in real world examples D dimer testing to rule out PE VQ or CTA to diagnose PE
References Van Belle A, et al, Effectiveness of Managing Suspected Pulmonary Embolism using an Algorithm Combining Clinical Probability, D Dimer Testing, and Computed Tomography, JAMA 2006;295:172 179. Chunilal SD, et al, Does this Patient have Pulmonary Embolism? JAMA 2003;290:2849 2858. Chunilal S. Evidence summary and review 1: pulmonary embolus. In: Simel DL, Rennie D, eds. The Rational Clinical Examination: Evidence Based Clinical Diagnosis. New York, NY: McGraw Hill; 2009. http://www.jamaevidence.com/content/3486556. Accessed 2/16/2013 Leclercq MGL, et al, Ruling Out Clinically Suspected Pulmonary Embolism by Assessment of Clinical Probability and D Dimer Levels: A Management Study, Thromb Haemost 2003;89:97 103. Lucassen W, et al, Clinical Decision Rules for Excluding Pulmonary Embolism: A Meta Analysis, Ann Intern Med 2011;155:448 460. Saltzman HA, et al The PIOPED Investigators, Value of the Ventilation/Perfusion Scan in Acute Pulmonary Embolism, JAMA 1990;263;2753 2759. Stein PD, et al, Multidetector Computed Tomography for Acute Pulmonary Embolism, NEJM 2006;354:2317 2327.