Critically Appraising Geriatric ED Screening Instruments Opening Pandora s Box to Futility or Identifying Novel Opportunities? Christopher R. Carpenter, MD, MSc, FACEP, AGSF June 2, 2015
Disclosure of Relationships Editor Academic Emergency Medicine, ACP Journal Club, and Journal of the American Geriatrics Society Past Chair, ACEP Geriatric Section Chair, SAEM Evidence Based Healthcare & Implementation Interest Group No commercial relationships, including no Advisory boards/consulting Officer or Board Member Shareholder Speaker s Bureau Intellectual property/patents pending Other relationships
Case 1: A broken hip 70 yo female comes to the ED after tripping over a stone in the garden. Lip laceration. PMHx: HTN, depression, fall 4 months ago Discharged Returns two weeks later with another fall and a hip fracture Was the patient s risk for future falls identified during the first ED visit? Was anything else missed on the first visit?
Objectives 1. Rationalize evidence based screening for older ED patients 2. Review screening for 2 geriatric syndromes A. Falls B. Dementia 3. Demonstrate Bayesian reasoning in geriatric assessments
Why ED Screening for Older Adults? Lots of older adults in ED (20% of visits in Canada >65), especially vulnerable elders Older adults hide their problems In theory, for a low cost we can identify problems, initiate interventions, and improve health outcomes ED screening for older adults for many stakeholders: Quality Indicator Guidelines Competency Standard for Resident Training References: Terrell 2009; Hogan 2010; Carpenter 2014
ED Screening for Older Adults Falls Dementia Short term adverse outcomes Delirium Alcoholism Depression Suicidality Risk of a car accident Elder Abuse Malnutrition Caregiver burden
Who gets screened and how? 11 screening tests > 1 hour of screening Just vulnerable older adults? Age 75? Is there is meta screen to quickly and objectively identify vulnerable older adults? Can nursing colleagues perform screens? Can we use tablet computers to screen? Should we ask patients if they want to be screened?
Diagnosis vs. Prognosis Diagnosis Identifying the nature or cause of some phenomenon Cross sectional In medicine often defined by a more invasive gold standard (biopsy, autopsy) Ex. Dementia Prognosis A prediction about how something will develop Longitudinal Usually defined by interval testing (labs, imaging, biopsy), selfreport, or medical record review Ex. Falls 8
Quantitative Attributes of Diagnostic and Prognostic Tests Accuracy Sensitivity/specificity Likelihood ratios Reliability Kappa Calibration How well estimated probability matches observed incidence Discrimination How well test distinguishes between patients more or less likely to have an outcome
Dementia: Scope of the Problem Dementia related medical care costs exceed $150 million annually in the U.S. 30% 40% of geriatric ED patients demonstrate non delirium cognitive dysfunction ED nurses miss >80% of these patients, EM physicians miss >70%, and inpatient physicians miss >60% Reference: Carpenter 2011
Assessing for Dementia During an ED Visit 6 ED validated instruments exist, only assessed in the U.S. Short Blessed Test Brief Alzheimer s Screen Mini cog Six Item Screener Caregiver AD8 Ottawa 3DY
Dementia: Sources of Data
Example Ottawa 3DY
Example Short Blessed Test
Short Blessed Test Multiple names across studies Quick Confusion Scale Brief Mental Status Exam Orientation Memory Concentration Test Different studies used different criterion standards Based upon one study that used STARD criteria and MMSE <24 LR + 2.7 (95% CI 2.2 3.0) LR 0.08 (95% CI 0.03 0.2)
Table Dementia Screening Instrument LRs Test # of studies LR+ (95% CI) SBT 4 2.7 (2.2 3.0) BAS 1 2.0 (1.7 2.2) Mini cog 1 4.9 (2.4 8.3) O3DY 1 2.0 (1.6 2.1) SIS 3 3.3 (2.1 5.2) cad8 2 2.2 (1.6 2.8) LR (95% CI) 0.08 (0.03 0.2) 0.10 (0.03 0.3) 0.30 (0.10 0.62) 0.1 (0.03 0.3) 0.39 (0.26 0.58) 0.27 (0.1 0.5) Reference: Wilber 2005, Wilber 2008, Carpenter 2011
Likelihood Ratio = [Probability of Test Result in Patient With Disease or Outcome]/[Probability of Test Result in Patient Without Disease or Outcome] Positive Likelihood Ratio = Sensitivity/[1 Specificity] Negative Likelihood Ratio = [1 Sensitivity]/Specificity
Dementia (Non Delirium, Cognitive Dysfunction) Screening Instrument Synopsis Multiple instruments (SBT, BAS, O3DY) significantly reduce the probability for dementia (negative LR 0.10) but none significantly increase the probability (highest positive LR is 4.9) Various criterion standards used, none including DMS IV neuro psychiatric testing, CSF sampling, or imaging Few trials used STARD criteria
Where Can I Get These Instruments? http://www.jorem.org/index.php/jorem
Acad Emerg Med 2011; 18: 782 796 Acad Emerg Med 2014; 21: 1069 1082 Acad Emerg Med 2014; 21: 102 121
Falls: Scope of the Problem Most common mechanism of injury in older adults 10% of ED visits by older adults 6% of urgent hospitalizations $30 billion in direct medical cost per year Reference: CDC website Cost of Falls among Older Adults
Assessing for Post Care Fall Risk During an Two instruments and multiple individual risk factors have been assessed in the ED ED Visit
Falls: Sources of Data PUBMED search identified 185 articles EMBASE search identified 267 articles CINAHL search identified 41 articles CENTRAL search identified 15 articles DARE search identified 9 articles Hand search of scientific assemblies identified 7 abstracts 601 manuscripts and abstracts remaining after duplicates removed Cochrane search identified 4 articles Clinicaltrials.gov search identified 73 trials 596 excluded after reviewing titles/abstracts 5 full manuscripts and 7 abstracts reviewed 9 manuscripts and abstracts excluded No assessment of fall risk (2) Inability to reconstruct 2x2 tables (6) Non-geriatric focus (1) 3 primary studies included in this systematic review
Fall Risk Stratification Caveats These studies assess risk of falls following an ED evaluation among community dwelling older adults. These study do NOT predict: Risk for falls that occur in the ED or hospital Risk for falls among nursing home patients
Fall Risk Instruments Evaluated in ED Carpenter 2009 (N=261) Fall in last year Non healing foot sore Cannot cut own toenails Self reported depression Tiedemann 2013 (N=158) 2 falls in last year (2) 6 meds (1) References: Carpenter 2009; Tiedemann 2013
Performance of ED Falls Screens Tool Carpenter Score >1 Carpenter Score >2 Tiedemann Score >0 Tidemann Score >1 Tidemann Score >2 Positive LR (95% CI) 2.40 (1.95 2.80) 1.30 (1.2 1.3) 1.48 (1.28 1.72) 2.00 (1.61 2.50) 3.76 (2.45 5.78) Negative LR (95% CI) 0.11 (0.06 0.20) 0 (0 0.14) 0.44 (0.30 0.64) 0.40 (0.28 0.57) 0.46 (0.34 0.64) No study has acceptable positive LR
Most Accurate Single Predictors Positive likelihood ratio Depression (2.4 6.6) Foot sore (3.2) Fall in last 6 months (2.3) Uses cane or walker (2.3) Fall indoors (2.2) Injurious fall (2.1) Negative likelihood ratio Cuts toenails (0.57) No imbalance (0.61) No fall in last year (0.62) Does not wear glasses (0.65) No cane or walker (0.68) Lives with somebody (0.69)
ED Based Fall Research Issues Different definitions of falls used Self report used to define risk factors like dementia, depression, and baseline function Fall risk assessed at a single point in time (during the ED visit), but risk may increase with progression of an illness References: Carpenter 2011
Fall Screening Risk Synopsis Existing instruments demonstrate potential to distinguish low risk subset, but do not accurately identify a group at high risk for falls More accurate instruments are needed Objective tests of function seems like an obvious solution but were highly predictive of future falls in the 2 studies
Case Resolution 70 yo female returned to ED with recurrent fall and hip fracture Was the patient s risk for future falls identified during the first ED visit? No Was anything else missed on the first visit? No
Summary: What screening tests should I use for these four problems? Dementia: Ottawa 3DY or Short Blessed Falls: Huge problem, may be able to identify low risk pts, need better predictors of high risk
Summary, continued Geriatric EM risk stratification decision instrument investigators must adhere to derivation/validation research & reporting guidelines Future iterations of the ACEP/AGS/SAEM (?CAEP?) Geriatric ED Guidelines should quantify the prognostic accuracy of screening instruments for older adults
Collaborate with AGEM and the ACEP Geriatric Section http://www.acep.org/geriatricsection/ http://tinyurl.com/saem AGEM
Questions? Contact Information E mail: carpenterc@wusm.wustl.edu http://emed.wustl.edu/carpenter_christopher/projects.aspx Journal Club website: http://tinyurl.com/washu EMJournalClub Twitter1 @GeriEDNews Twitter2 @SAEMEBM
EXTRA SLIDES
Types of Bias Threats to Diagnostic Accuracy Name Context bias Verification bias Double gold standard Imperfect gold standard bias Incorporation bias Interval bias Description Outcome assessors swayed by disease prevalence Variable criterion standard applied depending upon index test result Different criterion standards employed for subsets Imperfect criterion standard Outcome assessor with access to criterion standard while interpreting index test or vice versa Time between index test & criterion standard alters disease status
Types of Bias Sources of Variability Name Subtypes Description Spectrum bias Case mix or subgroup bias Test performance influenced by disease severity & unique to subsets of patients Cutoff bias Temporal bias Thresholds for continuous data defined arbitrarily or after data analysis Technology improvements, operator expertise, or both improve over time limiting the contemporary accuracy of earlier research
Revised QUADAS Domains 1) Applicability do patients and setting match the review question? 2) Index test interpretation of this test without knowledge of gold standard? If threshold used, was it pre specified? 3) Reference standard does this criterion standard correctly classify the disease in question? Was the reference standard interpreted without knowledge of index test?
Revised QUADAS Domains, cont 4) Flow and timing was there an appropriate interval between the index test and reference standard? Did all patients receive the same reference standard? Were all patients included in the analysis?