The Data Science of Physiologic Signals. Una-May O Reilly ALFA, CSAIL, MIT

Transcription

1 The Data Science of Physiologic Signals Una-May O Reilly ALFA, CSAIL, MIT

2 An Intersection and Inflection Point

3

4

5 2009 PhysioNet Challenge 10 hours 1 hour AHE Event λ=60mmhg = 30 minutes Prediction Problem: AHE or not Lag= 10 hours Lead=1 hour Event Parameters Threshold: λ < 60 MMHG Others PhysioNet Challenge, 2009.

6 2009 PhysioNet Challenge Provided: 60 training cases, 50 test cases (split A:10, B:40) Figure 1 from G. Moody and L. Lehman, Predicting acute hypotensive episodes: The 10th annual physionet/computers in cardiology challenge, in Computers in Cardiology, vol. 36, 2009, pp

7 Agenda Pinpointing the predictive power of large scale waveform data GIGABEATS project 1. Predictability mining Hyper-parameters of physio time series modeling Sensitivity analyses» beatdb : A Large Scale Waveform Feature Repository, Franck Dernoncourt, Kalyan Veeramachaneni and Una-May O'Reilly, MLCDA@NIPS 2013 : Machine Learning for Clinical Data Analysis and Healthcare.» BeatDB: An end-to-end approach to unveil saliencies from massive signal data sets. Franck Dernoncourt, S.M, thesis, MIT Dept of EECS, February Predictability optimization Gaussian Process hyper-parameter optimization» Gaussian Process-based Feature Selection for Wavelet Parameters: Predicting Acute Hypotensive Episodes from Physiological Signals, Franck Dernoncourt, Kalyan Veeramachaneni and Una-May O'Reilly. IEEE 28th International Symposium on Computer- Based Medical Systems. IEEE Computer Society, 2015.

8 Data preparation for a Modeling competition Competition defines the problem by lead lag event threshold Modeler chooses features Physiological Data: Strips per patient Feature Time Series Predictability Mining

9 Data preparation for a competition Segmentation Labeling Predictability Mining

10 Hyper-Parameters of Prediction Modeling LEAD EVENT LAG Predictability Mining

11 GIGABEATS Predictability Mining Systematic, investigation of modeling hyper-parameters AT LARGE DATA SCALE Wrapped around a standard time series modeling methodology Investigates combinations of hyperparameter Lead, lag, threshold ranges of problem definition features Has cloud-scaled, distributed Beat feature extraction Data set assembly (lead, lag, event-threshold)» Segmenting» Labeling» Feature engineering Machine learning Delivers multiple models and their performance on testdata systematically Each model addresses the investigative problem with a different hyper-parameter combination Facilitates sensitivity analysis around the data s predictive power

12 Parameters of Sensitivity Analysis I Data Parameters 5000 patients with the most ABP data 125 Hz and there are a total of 240,000 hours of ABP data 1.2 billion beats (0.9 billion being valid) AHE prediction with ABP Varied Hyper-Parms! Lead (6)! Lag (6)! AHE Threshold (5) Fixed:! Features (14/ window)

13 Sensitivity Analysis I Case exemplars Data Imbalance AHE thresholds change exemplars Predictability Mining

14 Sensitivity Analysis I Features 1. Mean of MAP 2. Root-mean-square level (RMS) of MAP 3. Standard deviation of MAP 4. Kurtosis of MAP 5. Skewness of MAP 6. Systolic blood pressure (Max ABP) 7. Diastolic blood pressure (Min ABP) 8. Pulse pressure 9. Duration of each beat 10. Duration of systole 11. Duration of diastole 12. Pressure area during systole 13. Crest factor (Peak-to-average ratio) 14. Mean arterial pressure (MAP) Lead and feature set not optimized Lag and Event Threshold Influence AUC at lead=10 min

15 Hyper-Parameters of Prediction Modeling LEAD EVENT LAG Feature selection

16 Parameters of Sensitivity Analysis II Varied Hyper-Parms! Lead (6)! Lag (6)! Wavelet Features (190) Total combinations: 3680 per Mother Haar, Gaussian-2, Symlet-2 Fixed:! Threshold (60 mmhg) AHE prediction with ABP What Features to Select Data Parameters same!

17 Wavelets Symlet-2 Mother s=0.25, τ = 0 Symlet-2 Mother S=1, τ = 0 Symlet-2 Mother S=1, τ = 1

18 Resulting Wavelet Parameter Scan We convolved each beat(!) Per Beat Feature Matrix 10 X 19 = 190 features/beat We precomputed all wavelet features Pre-computing 3 CWT with 10 scales and 19 different time shifts takes ~6 hours using core nodes (Intel Xeon L5640 processors), and storing the results of each CWT requires 300 GB.

19 Gaussian-2 AUC for lead, lag = (10,10) Each cell contains one AUC

20 CWT Mother Comparison Bio-2 Gaussian-2 Haar Symlet-2

21 AUC Summary

22 Influence of Lead Gaussian-2 CWT, take best AUC for each lead and lag

23 Sensitivity to Sample Size

24 Observations Drivers of AUC performance Hyper-parameters» Lead, lag, threshold: problem definition» Feature selection is a huge subset of hyper-parameter space Sample size matters! We are predictability mining to yield sensitivity analyses Effective at pinpointing predictive power but it won t scale! Costly even now» Pre-computations, distributed system save time but not cost We need to predictability optimize for AUC performance Cut the cost down

25 Predictability Optimization Objective: find the modeling hyper-parms that maximize AUC Efficiently Methods Random search in hyper-parm space 200 samples (<5% of 3680) Becomes our 2 nd baseline Gaussian Processes Randomized by initial sample Parameterized by initial sample size and kernel 200 samples

26 Gaussian Process Optimization We assume each AUC is a random variable Gaussian distributed x i = (s,τ,lead,lag) y i = (AUC of AHE prediction)

27

28 Choosing a Kernel

29

30

31 Summary of Predictability Optimization GP reduces cost to 1/3 of random search We reduce the # of AUCs computed of entire sample to <5% GP usage needs more investigation Systematize selecting its parameters: kernel, initial sample sz» A different time series, eg ECG» Different hyper-parameters: features, problem definition " Impact on kernel selection» More hyper-parameters " Impact on initial sample size» Robust conclusions on distribution of algorithm But we ve identified Pinpointing Prediction Power as the issue Predictability mining and optimization for large data Time to refine the use case

32 Predictability Optimization as a Service Data Server ML Server Optimi zation Server Results BeatDB Worker

33 Acknowledgments ALFA principal Kalyan Veeramachaneni, PhD Franck Dernoncourt PhD candidate Also: Alex Waldin, UROPs of ALFA

34 Questions?