Leveraging Analytics and Artificial Intelligence to Predict Health Risk A case study on using machine learning to boost atrial fibrillation identification and anticoagulant drug use Arthur L. Forni, MD, FACP CEO, WESTMED Practice Partners In collaboration with Optum Enterprise Analytics November 28, 2018
Disclosures Presenter has the following interest to disclose: Practicing physician, WESTMED Medical Group Full-time Optum employee (CEO, WPP) ACE/PESG and AMSUS staff have no interest to disclose. This continuing education activity is managed and accredited by AffinityCE/ Professional Education Services Group in cooperation with AMSUS. ACE/PESG, AMSUS, and all accrediting organizations do not support or endorse any product or service mentioned in this activity. 2
Learning objectives At the conclusion of this activity, the participant will be able to: Understand clinical and economic consequences of undetected atrial fibrillation and its relationship to acute ischemic stroke. Understand basics of the use of artificial intelligence in predictive/imputational modeling. Understand current techniques to combat black-box perceptions of AI/ML. 3
Atrial fibrillation can be difficult to discern Diagnosis can take time Intermittency A-fib can be fleeting and intermittent; symptoms can be vague (e.g., palpitations, dizziness, fatigue, mild shortness of breath). Many patients don t seek care. means a difficult diagnosis. ECG can be misinterpreted, events can go uncaught or symptoms can be attributed to other illnesses (e.g., panic attack). A-Fib must be confirmed with ECG tracing. If ECG doesn t catch the episode, an ambulatory Holter monitor is worn for 24 48 hours. If events are infrequent, a cardiac event monitor can be worn over a prolonged period. 4
Why is atrial fibrillation so important? Risk grows with age, diagnosis is often missed, and stroke is a common complication. Prevalence in All people 0.5% 1% Ages 50+ 5% Ages 60+ 10% 20% 30% of cases undiagnosed according to medical literature Risk of stroke in untreated A-Fib for: Male, 60, no risk factor <1.0% Male, 75, no risk factor 2.2% Male, 75, 1 risk factor 3.2% Male, 75, prior stroke 4.8% Female, 75, 3 risk factors 9.7% 5
Anticoagulation therapy helpful, but costly 70 75% reduced risk of stroke 25% reduced risk of death 8 average quality life years gained* Lifetime cost: $70,000 $80,000* *Average based on 70-year-old man with non-valvular atrial fibrillation, CHADS score 1, no contraindications to anticoagulation. 6
Westmed: Economic calculation Economic calculation 250,000 Total primary care patients 50+ 75,000 Patients older than 50 3,750 Patients expected to have A-Fib Based on a 30% undiagnosed rate ~1,125 Patients are not currently treated for A-Fib Based on a 3-5% risk of stroke ~35-55 Potential strokes could be prevented If left untreated, $4.2 million cost of treating strokes ($75K/stroke) If treated with anticoagulation, $4.5 million cost to treat 1,125 patients ($4K/patient/year) 7
Artificial intelligence/machine learning For whom? Why?...at-risk for time-bomb conditions. AI/machine learning can identify or predict A-fib members An unmet need Claims-based categorical models and traditional stats: inadequate. Major opportunity AI scalably outperforms traditional models and is more easily enhanced with novel data sources like genomics, voice signal, etc. 8
Key advantages of AI A U T O M A T I C V A R I A B L E S E L E C T I O N Scalable More efficient use of medical SMEs and more exhaustive feature sets. N O F I X E D R U L E S Learning AI models learn and improve as data accumulates, and with the addition of new data sources. I N T E R A C T I O N S Interactions AI outperforms in part by identifying variables interactions. 9
Machine learning approach Tree-based models are like a game of 20 questions Yes or no: Do you have one of the machine-identified features? Are you elderly? Do you have hypertension? Do you have other heart rhythm problems? Has a doctor been checking you with echocardiogram? Are you taking an angina drug? When trained, the model recognizes patterns associated with positive and negative classes. + + + Blue = mostly have A-Fib 10
A-Fib prediction Recurrent neural networks (deep learning) 10% 10% 12% 12% 30% 70% 95% LSTM LSTM LSTM LSTM LSTM LSTM LSTM Morphine, ibuprofen Routine visit, coughing Routine visit, HTN Routine visit Morphine, fatigue Digoxin, heart failure Digoxin, warfarin TIME 11
WESTMED Medical Group pilot 350,000 lives Model trained on 2.5 million lives in a de-identified Optum dataset Model tested against 1,000 WESTMED members without A-Fib/flutter and 100 with A-Fib/flutter Structured fields Medical and Rx claims AI/ML model Patient list sent to physicians. Absent diagnosis code for atrial fibrillation despite probability of >50%/>80%. Physician orders A-Fib testing, signifies agreement (or not) with disease engine. Codes, cardioverts and anti-coagulates as appropriate. 12
Initial pilot Impute (detect) a-fib in 1,096 subjects Tree-based AI (gradient boosted trees) at 50% threshold Member without A-Fib code ( ) Member with A-Fib code (+) AI predicted negative ( ) 980 32 AI predicted positive (+) 10 74 Performance on the 106 known to have A-Fib: Sensitivity 69.8% Specificity 99.0% Positive predictive value 88.1% Negative predictive value 96.8% Accuracy 96.2% Based on results, the Westmed clinical team requested repeat with 60,000 additional members and several more cardiovascular conditions. 13
In 61,000 additional members, likelihood of additional cardio conditions (imputation) Undiagnosed individuals, identified at given risk level Disease 50% threshold 60% threshold 70% threshold 80% threshold Hyperlipidemia 5,639 3,485 2,004 959 CAD 1,057 579 324 156 CHF 591 372 226 112 Cases detected: no physician panels were involved. Why aren t these being checked or treated? A-Fib and Flutter 343 225 142 89 14
Historical prospective study (prediction) Capitalizing on Westmed long-term EHR dataset Member criteria: age 50+ no prior A-Fib diagnosed with A-Fib between 2012 2017 Lookback to confirm no prior A Fib ID2 5 YE A R S ID3 5 YE A R S ID4 5 YE A R S ID5 5 YE A R S 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 uniform lookback window of 5 years diagnosis window Key: member without A-Fib member with A-Fib 15
P R E D I C T E D P R E D I C T E D P R E D I C T E D 2x2 truth table at 15% precision, in training data Results at 15% positive predictive value Tree-Based Machine Learning A C T U A L Neural Net A C T U A L N E G POS N E G POS N E G 11,332 433 Sensitivity 41.3% N E G 10,508 274 Sensitivity 62.9% POS 1,708 305 Specificity 86.9% POS 2,548 464 Specificity 80.5% Reference A C T U A L NEG POS NEG P O S True negative Correctly predicted for A-Fib False positive Incorrectly predicted as positive for A-Fib False negative Incorrectly predicted negative for A-Fib True positive Correctly predicted positive for A-Fib Sensitivity P (predicted + true +) Specificity P (predicted true ) Positive predictive value P (true + predicted +) 16
Model performance comparison: predicting A-Fib Both models trained on UHC claims Tested on UHC claims (not seen by model) Tested on Westmed EMR (not seen by model) ROC or C Stat measures the ability to maximize true positives and minimizes false positives. 17
Important variables across the population Tree-based model for imputation (detection) 18
Clinical Risk Dashboard
Prospective study Natural history of the predicted cohort Claims-stream at 6, 12, 18, and 24 months in future: The fraction positive for A-fib? Improved lead-time/stroke prevention? 20
Leveraging AI for better health A clinical opportunity for early detection of chronic conditions Better health outcomes (e.g., atrial fibrillation) Improved NPS results for members and providers Decreased medical spend Publication (e.g., The New England Journal of Medicine) Product opportunities with OptumInsight 21
Next actions 27 conditions in flight # Condition Imputation Prediction 1 Diabetes X X 2 Asthma X 3 Chronic obstructive pulmonary disease X 4 Ischemic cerebrovascular disease X 5 Acute ischemic stroke X 6 Hypertension X 7 Chronic renal failure X 8 Depression X 9 HIV positive without ICD criteria for AIDS X 10 AIDS X 11 Adult rheumatoid arthritis X 12 Multiple sclerosis X 13 Inflammatory bowel disease X 14 Atrial fibrillation and flutter X X 15 Coronary artery disease X 16 Congestive heart failure X X 17 Hyperlipidemia, other X 18 Opioid use disorder (OUD) X X 19 Chronic kidney disease X 20 Sepsis/Septicemia 21 Back conditions 22 Degenerative joint disease, hip 23 Degenerative joint disease, knee 24 Chronic sinusitis 25 Tonsillitis, adenoiditis, or pharyngitis 26 Kidney stones 27 Mood disorder, bipolar
Next actions Leveraging decades of data for Predictive Modeling and Population Health The research is completed or in flight for the following areas: Conditions Researched for Imputation Asthma, chronic obstructive pulmonary disease, ischemic cerebrovascular disease, hypertension, chronic renal failure, depression, HIV positive without ICD criteria for AIDS, AIDS, adult rheumatoid arthritis, multiple sclerosis, inflammatory bowel disease, coronary artery disease, hyperlipidemia Conditions Researched for Both Imputation and Prediction Diabetes, atrial fibrillation and flutter, Congestive heart failure, opioid use disorder Conditions Researched for Prediction (Only) Acute ischemic stroke, chronic kidney disease Other conditions to be researched in the future Sepsis/septicemia, back conditions, degenerative joint disease hip, degenerative joint disease knee, chronic sinusitis, tonsillitis, adenoiditis, pharyngitis, kidney stones, mood disorder, bipolar Discussion: How to use these tools to improve population health for Service Members and Veterans
CE/CME credit If you would like to receive continuing education credit for this activity, please visit: http://amsus.cds.pesgce.com Hurry, CE Certificates will only be available for 30 DAYS after this event! 24
Thank you. Arthur L. Forni, MD, FACP CEO, Westmed Practice Partners