Development and Applica0on of Real- Time Clinical Predic0ve Models Ruben Amarasingham, MD, MBA Associate Professor, UT Southwestern Medical Center AHRQ- funded R24 UT Southwestern Center for Pa?ent- Centered Outcomes Research CEO, Parkland Center for Clinical Innova?on Date: Monday, December 1, 2015 Time: 12:00 1:00pm
Objec?ves Background: Predic?on and the Prac?ce of Medicine A Taxonomy for Predic?on and Decision Support Applica?on to Readmissions Further Considera?ons Q&A 2
What We Do in Medicine: Predic?on 1. What does this pa?ent have? 2. What will this pa?ent develop? 3. What will be the effect of a given therapy? 3
Predic?on in the Context of Modern Medicine Staggering increase in medical informa?on Increasing volume of decisions at mul?ple levels High fragmenta?on of care Increasing capacity for error 4
Glimmers of a New Age in Medicine Massive data capture Real-?me precision monitoring Vast computa?onal power Natural Language Processing Machine Learning 5
A Modified Taxonomy Term Predic?on Rule Decision Rule Decision Analysis Decision Support Prac?ce Guidelines e- Health Predic?ve Analysis Defini0on Provides diagnos?c or prognos?c probabili?es; does not recommend decision. Predic?on rule sufficiently validated to recommend decisions Quan?fies value of specified outcomes from published literature; health policy Designed to prevent errors when implemen?ng decisions already made Address several issues for pa?ents with a par?cular syndrome; ocen consensus based Technologies or socware systems that can autonomously employ and some?mes reengineer, modify, or update clinical risk predic?on models Reilly BM et al. Annals of Internal Medicine 2006;144:201-209 Amarasingham R et al. Health Affairs 2014 6
Clinical Applica?on of Predic?ve Analy?cs for Different Condi?ons and?meframe Hours 30 days 90 days Years Cardio- Pulmonary Arrest Sepsis Pa?ent Safety Event Surgical Complica?on Readmissions All- Cause AMI CHF PNA HIV Cirrhosis 7 Short- Term Diabe?c Complica?ons Chronic Kidney Disease Preventable DM Admissions
Predic?ng 30- Day Readmission in Heart Failure in Real- Time Heart failure readmissions are common, costly, and many are poten?ally preventable EMR- based readmission models are promising: Can tap many domains of risk Produce ac?onable intelligence usable in real-?me New domains of interest include: Socio- demographics, mental health/substance abuse, social disadvantage 8
Conceptual Model and Program Theory 9
Modeling HF Severity of Illness Model: Age, Albumin, T bili, CK, Crea?nine, Na, BUN, pco2, WBC, TN- I, Glucose, INR, BNP, ph, Temp, Pulse, Diastolic BP, Systolic BP (deriva?on=273, 034; valida?on=629,400) Tabak et al, Medical Care, 2007 10
Odds Ra?os for 30 Day Readmission for Final Model Variables Variable 11 Multivariate OR (95% CI) Tabak Mortality Score (composite) 1.52 (1.31-1.76) Single 1.44 (1.08-1.21) Male 1.37 (1.02-1.84) Number of Home Address Changes 1.13 (1.07-2.17) Medicare 1.59 (1.17-2.17) Residence in low SES census tract 1.38 (0.98-1.74) History of documented cocaine use 1.78 (1.17-2.72) History of missed clinic visit 1.35 (0.99-1.83) Used a health system pharmacy 0.72 (0.52-1.02) Number of prior inpatient admissions 1.17 (1.05-1.27) Presentation time (day or night) 1.38 (1.05-1.81) History of Depression or Anxiety 1.44 (1.00-2.07)
Performance of Electronic Heart Failure Model (N=1372, C- Stat 0.72) 70 60 Derivation Samples Validation Samples 51.65 50 45.68 40 30 26.93 26.0 20 10 8.77 12.22 14.27 16.08 17.94 19.98 0 Very Low Low Intermediate High Very High Predicted Readmission Risk Category 12 Amarasingham et al, Medical Care, 2010
From Predic?on to Ac?on: HF Interven?on Challenges & Solu?ons Iden?fy HF pa?ents in the first 24 hrs of admission Ocen no defini?ve admission diagnosis (SOB, edema, cough) Natural language processing to flag cases Too many pa?ents & too expensive to try to do everything on everyone Iden?fy those at highest risk to get enhanced suite of inpa?ent & outpa?ent interven?ons Provide electronic surveillance, coordina?on, & tracking of interven?on processes of care & outcomes 13
Readmission Preven?on Interven?on Workflow Admission Discharge 30 Days 90 Days 24 hours 7 days ID Risk List Orders 1 2 3 4 Inpatient Intervention 5 5 Outpatient Intervention Evaluation & Improvement 6 Electronic Medical Record Real- Time HL7 based interface EMR applica0on 14
Interven?on Results: 26% Rela?ve Reduc?on in Odds of Readmission Results Demonstrated: Concentrated care management efforts on ¼ of the pa?ents 26% rela?ve reduc?on in odds of readmission Absolute reduc?on of 5 readmissions per 100 index admissions Saved hospital $1 million 15
Compara?ve Effec?veness of Different Models for Predic?ng Readmissions 1. EMR- based v. administra?ve claims models 2. Develop & validate an all- cause, mul?- condi?on model 3. First- day v. full- stay models 4. Mul?- condi?on generic v. disease- specific models 5. Customizing models to individual hospitals, data systems, pa?ent popula?ons 16
Develop and Validate an All- Cause, Real- Time, Mul?- condi?on model 40 Derivation cohort Validation cohort 30 30.6 30.0 20 19.2 17.4 13.4 12.5 10 6.0 5.5 9.3 9.0 0 group 1 group 2 group 3 group 4 group 5 Predicted Risk Category Amarasingham et al. Under Review, 2014 17
EMR All Cause model versus CMS and LACE models 18
All- Cause 30- Day Readmission Model Full Hospital Stay versus First Day (n=16430) Predicted Risk, % C- sta?s?c (95% CI) Lowest Highest All- cause full- stay Deriva0on cohort 0.715 (0.704 0.727) 4 37 Valida0on cohort 0.689 (0.677 0.701) 4 37 All- cause 24- hour 0.667 (0.655 0.680) 6 32 Other models: LACE* 0.647 (0.635 0.660) 6 28 HOSPITAL model 0.636 (0.623 0.649) 7 27 CMS HWR 0.665 (0.652 0.678) 5 30 19
Other Considera?ons 1. Interven?ons for highest risk pa?ents 2. Considering clinical vs. social risk 3. Explana?on vs. Predic?on 4. Non- health care data sources 5. Changing EMR data models 6. Changing clinical interven?ons 7. Changing popula?ons 20
Larger Complexi?es of Predic?ve Modeling in Healthcare 21
Clinical Applica?on of Predic?ve Analy?cs for Different Condi?ons and?meframe Hours 30 days 90 days Years Cardio- Pulmonary Arrest Sepsis Pa?ent Safety Event Surgical Complica?on Readmissions All- Cause AMI CHF PNA HIV Cirrhosis 22 Short- Term Diabe?c Complica?ons Chronic Kidney Disease Preventable DM Admissions
Conclusions Predic?ve analy?cs are a promising way to help improve?meliness, safety and quality in health care; CHF model and interven?on are an example of successful implementa?on of predic?ve analy?cs in real- world setng; All Cause, Mul?- Condi?on Readmission Model has poten?al to have more significance for applica?on; Many challenges remain in methodology, applica?on and policy for electronic predic?ve analysis to thrive. 23
Ques?ons Contact Informa0on: Ruben Amarasingham ruben.amarasingham@phhs.org www.pccipieces.org 24