The cross sectional study design. and pretest. Probability (participants). Index test. Target condition. Reference Standard Mirella Fraquelli U.O. Gastroenterologia 2 Fondazione IRCCS Cà Granda Ospedale Maggiore Policlinico, Milano
Oulines Definition and aim of diagnostic accuracy test study Key components: Study design Index test Reference standard
Diagnostic test accuracy is the ability of a test to distinguish between patients and subjects without the target condition Target condition: a particular disease, disease stage or health status that should prompt clinical action (such as the initiation modification or termination of treatment)
Analytical diagnostic reasoning Initial level of probability of a diagnostic hypothesis (Epidemiology, medical history, physical examination, laboratory etc.) PRETEST PROBABILITY INDEX TEST RESULT Variation of the level of probability POSTTEST PROBABILITY Level of certainty necessary for a therapeutic decision
Objective of a diagnostic test accuracy study To assess how the index test change the probability of the disease
Posrtest probability 10 20 40 60 80 90 100 LR 12 Test Information Ultrasound scan in the detection of hepatocellular carcinoma Colli et al. Am J Gastroenterol 2007 LR 0.6 10 20 40 60 80 100 Pretest probability
Analytical diagnostic reasoning The acceptable level of UNCERTAINTY depends on the penalty for being wrong. For my single patient it will be better to be treated if false positive or not treated if false negative?
How can I transfer the results obtained from diagnostic studies to my single patient?
Diagnostic test accuracy studies Internal validity Correct study design Internal validity Ideal experimental conditions Data homogeneity Reduced heterogeneity External validity External validity Clinically relevant context No center selection No patients selection Comorbidity Data precision and repeatability Data transferability
Factors affecting the transferability of data derived from diagnostic studies to a single patient 1) Critical appraisal of available diagnostic literature by assessing: methodological quality (INTERNAL VALIDITY) transferability (EXTERNAL VALIDITY) reporting (correct expression of diagnostic estimates) (STARD INITIATIVE) 2) Match the characteristic of my patient to that of those reported in the literature as measures of accuracy may vary across patient groups
Diagnostic test validity and generalizability must be evaluated in a clinically relevant population An optimal study includes a broad spectrum of persons who would undergo the test in a clinical setting However, test performances can varies across population subgroups according to patient characteristics
Study design Basic design of diagnostic accuracy studies: Prospective, blinded cross classification of test and reference standard in a clinical relevant setting Liver biopsy TP Relevant spectrum of patients TE Liver biopsy FP FN TN
SPECTRUM BIAS Spectrum effects: evaluation of two very different populations Diagnostic patient control study % Healthy volunteers Test threshold Very sick individuals the healthiest the sickest Test parameter
Spectrum effects: evaluation of representative populations Diagnostic cross sectional study % Patients without disease Test threshold Patients with disease Test parameter SPECTRUM VARIATION
Severity of the disease comorbidities Severity of of the the disease comorbidities
Index test Validity. A test is valid when it measures what it s supposed to Reliability: A test is reliable when it yields consistent result. Repeatability and reproducibility ( Inter and intraobserver variability) Possible sources of danger and costs for unnecessary procedures (e.g. direct harm of radiation exposure : 2% of cancers may be attributed to radiation exposure during CT scanning). Lauer NEJM 2009; 84143 Brenner 2007; 357: 227784
Remember the purpose of your test! Triage Add on Replacement EACH SITUATION MAY REQUIRE DIFFERENT TEST FEATURES!
Existing situation Replacement Triage Addon Initial test Initial test New test Initial test Existing test New test Existing test Existing test New test Bossuyt et al. BMJ 2006;332:10891092
Existing situation Replacement Triage Addon Initial test Initial test New test Initial test Existing test New test Existing test Existing test New test Bossuyt et al. BMJ 2006;332:10891092
Existing situation Replacement Triage Addon Initial test Initial test New test Initial test Existing test New test Existing test Existing test New test Bossuyt et al. BMJ 2006;332:10891092
Existing situation Replacement Triage Addon Initial test Initial test New test Initial test Existing test New test Existing test Existing test New test Bossuyt et al. BMJ 2006;332:10891092
The Reference Standard The procedure (or test) that is used to define the true state of the patient The Reference Standard is generally more invasive and expensive than the index test The Reference standard to establish the final diagnosis should be applied for all included patients
Index Test Reference Standard TP FP FN TN If Reference Standard Disease
From the GOLD STANDARD to the REFERENCE STANDARD The reference standard should accurately reflect the true state of the patient, but is usually imperfect. Its sensitivity and specificity are < 100% Reference standard Target Disease
INDEX TEST Disagreements between Reference Standard and Index Test REFERENCE STANDARD Pos Neg Pos A agreement B Cases detected only by the Index Test Index Test more sensitive Neg C D Cases detected only by the RS agreement Index Test more specific Glasziou Ann Intern Med 2008; 149:816
Trade off Sensitivity/Specificity Broad histopathologic criteria to diagnose colon dysplasia Liver biopsy in diagnosing hepatic fibrosis Imperfect reference standard vs true disease status Sensitivity Specificity Index test Underestimates sensitivity Underestimates specificity
REFERENCE STANDARD: SENSITIVITY < 100 % INDEX TEST SPECIFICITY CAN BE UNDERESTIMATED REFERENCE STANDARD: SPECIFICITY < 100 % INDEX TEST SENSITIVITY CAN BE UNDERESTIMATED