Machine Learning Models for Blood Glucose Level Prediction

Transcription

1 Machine Learning Models for Blood Glucose Level Prediction Sadegh M. Hui Shen Cindy Marling Matt Wiley Franck Schwarz Nigel Struble Jay Shubrook Lijie Xia Razvan Bunescu School of EECS & Diabetes Institute, Ohio University Talk at University of Akron, Oct 20, 2017 Kevin Plis

2 Management of Type I Diabetes Approximately 20 million people have Type 1 Diabetes: In type 1 diabetes, the pancreas produces no insulin. Patients depend upon external supplies of insulin, via injections or insulin pumps. Diabetes can not be cured, but it can be treated and managed: To delay or prevent long-term complications, patients try to keep Blood Glucose Levels (BGL) as close to normal as possible. Good blood glucose control requires: 1. Monitoring BG levels. 2. Corrective steps to avoid hyper- and hypo-glycemia. 2

3 Chronic Complications vs. Blood Glucose Control RISK Microalbuminuria Mild Retinopathy Mild Neuropathy Albuminuria Macular Edema Proliferative Retinopathy Periodontal Disease Impotence Gastroparesis Depression Foot Ulcers Angina Heart Attack Coronary Bypass Surgery Stroke Kidney Transplant Dialysis Blindness Amputation Good CONTROL Poor 3

4 Management of Type I Diabetes Blood glucose levels are monitored using: Glucometers (fingerstick measurements). Continuous Glucose Measurement Systems (CGMS): CGM sensor. Insulin pump. 4

5 Continuous Glucose Monitoring (CGM) in Insulin Pump Therapy Systems CGM Sensor: interstitial BGL. every 5 minutes. Insulin Pump delivers insulin through boluses and basal rate: 5

6 Achieving Good Blood Glucose Control Patients must continually monitor their blood glucose levels and adjust insulin doses, striving to keep blood glucose levels as close to normal as possible: Requires significant effort from patients and doctors. Try to avoid especially: Hypoglycemia Hyperglycemia Excessive Glycemice Variability forecasting of blood glucose levels detection of glycemic variability 6

7 Outline 1. BGL prediction using manually engineered physiological models: Physiological Equations + Kalman filtering + SVR. 2. BGL prediction using trained physiological models. RNNs with LSTM units. 3. Attention models for time series prediction. 4. From physiological models to automated BGL control. 7

8 Automatic BGL Prediction Develop time series forecasting models that predict BGL 30 or 60 minutes into the future: Accurate predictions up to 60m in advance would allow plenty of time to take preventive action, to avoid hypo- or hyper-glycemia. Inputs for the prediction model: Previous blood glucose measurements taken at 5-minute intervals through a CGM system. Daily event data: Insulin dosages, recorded in the CGM device. Life events, collected through a smartphone interface. 8

9 Input: Life Events Developed a smartphone interface to collect relevant life events: Meals (carb amounts, glycemic index). Sleep (start, end). Work (start, end). Exercise (intensity, start, duration). Hypoglycemic event. Health events (stress, depression,...). Other events. Designed to encourage entering events immediately before/after they happen: to minimize incorrect/incomplete data. 9

10 10

11 First Evaluation Dataset Total of 1,400 days worth of clinical patient data: CGMS + insulin + life events. Human performance on the task of BGL prediction: Asked 3 diabetes experts to manually label an evaluation dataset with their 30/60 min predictions: 200 timestamps, coming from 5 patients with T1D. 40 points per patient. Manually selected to reflect a diverse set of situations. Built a GUI to facilitate navigating the data and labeling. 11

12 12

13 13

14 Physician Performance Compared the 3 physicians against 2 baselines: t 0 predicts that future BGL is the same as current BGL. Auto-Regressive Integrated Moving Averages (ARIMA), trained on past BGL data. Evaluation measures: Root Mean Square Error (RMSE). Total cost of ternary classification: Future BGL is Same (S), Lower (L), Higher (H) as current BGL. Same means within 5 (10) mg/dl for 30 (60) min prediction. cost(l, S) = cost (H, S) = 1; cost(l, H) = 2. 14

15 Physician Performance Physicians, who use daily event data, outperform ARIMA. Physicians regularly refer to daily events: Timing of meal events and boluses, carb amouns, bolus types. 15

16 Physician Performance Physicians, who use daily event data, outperform ARIMA. Physicians regularly refer to daily events: Timing of meal events and boluses, carb amouns, bolus types. Use daily events to extract features for automatic BGL prediction. 16

17 Physiological Modeling of BG Dynamics Use equations from literature [6, 7, 8, 9] to model dynamics of variables that are relevant to BG behavior: Almost identical equations (based on the same data). Characterize the overall dynamics into 3 compartments: Meal absorption dynamics. Insulin dynamics. Glucose dynamics. Changed some of the equations and their parameters to better match published data and feedback from our doctors. 17

18 A Physiological Model of BG Dynamics A continuous dynamical model that is described by: 1) The input variables U. 2) The state variables X. 3) The state transition function f that computes the next state given the current state and input i.e. X t+1 = f(x t,u t ). 1) The vector of input variables U contains: U C (t), the carbohydrate intake measured in grams (g). U I (t), the amount of insulin measured in insulin units (U): Computed from bolus events and basal rate data. 18

19 A Physiological Model of BG Dynamics 1) The state variables H are organized according to the 3 compartments: 1) Meal Absorption Dynamics: C g1 (t) = carbohydrate consumption (g). C g2 (t) = carbohydrate digestion (g). 2) Insulin Dynamics: I S (t) = subcutaneous insulin (µu). I m (t) = insulin mass (µu). I(t) = level of active plasma insulin (µu/ml). 3) Glucose Dynamics: G m (t) = blood glucose mass (mg). G(t) = blood glucose concentration (mg). 19

20 A Physiological Model of BG Dynamics 2) The state transition function f captures dependencies among variables in H and U at consecutive time steps: 20

21 A Physiological Model of BG Dynamics 2) The state transition function f captures dependencies among variables in H and U at consecutive time steps: 21

22 A Physiological Model of BG Dynamics 2) The state transition function f captures dependencies among variables in H and U at consecutive time steps: 22

23 Glucose Dynamics: Insulin Dependent Utilization 23

24 Glucose Dynamics: Insulin Dependent Utilization G m (t+1) = G m (t) Δ dep Δ ind Δ clr Δ dep + Δ abs + Δ egp Δ dep = α 1 I(t) [G(t) + α 2 ] 24

25 Physiological Model Equations 25

26 A Physiological Model of BG Dynamics The state transition equations were used in an Extended Kalman Filter (EKF) model: Run a state prediction step every 1 minute. Run a correction step every 5 minutes. The EKF model itself can be used to make 30 or 60 minute predictions: Performance is lower even than the t 0 baseline. Could improve by tunning the α parameters for each patient: Time consuming, unfeasible due to large number of params. Difficult to incorporate other types of life events in the model. 26

27 A Support Vector Regression (SVR) Model with Physiological Features The state vector H(t) computed by the physiological model is H(t) = [C g1 (t), C g2 (t), I S (t), I m (t), I(t), G m (t), G(t)]: Run the EKF model up to time t 0, with a correction step every 5 minutes => H(t 0 ). Run the EKF model in prediction mode for 60 more minutes => H(t ) and H(t ). Create the following features for the SVR model: All predicted state variables in H(t ) and H(t ). The difference vectors H(t 0 ) H(t ) and H(t 0 ) H(t ). 12 features delta i = BG(t 0 ) BG(t 0 5i). Optionally, train ARIMA on 4 days before t 0, and use the 12 predictions in the one hour after t 0 as features

28 SVR Evaluation Train SVR on the week of data preceding each test point t 0 : Use a Gaussian kernel: Tune parameters γ, ε, and C using grid search on the week preceding the training week. If not enough tunning examples, use generic parameters tuned on another patient. Compare the best doctor performance with: ARIMA and the t 0 baselines. SVR model using physiological features, with (SVRφ+A) and without (SVRφ) ARIMA features. A previous SVR system (SVRπ+A) that uses CGM data, ARIMA, and an ad-hoc implementation of daily event features. 28

29 Experimental Results on BGL Prediction 29

30 Experimental Results on BGL Prediction Both SVRφ systems outperform the 3 diabetes experts! 30

31 Outline 1. BGL prediction using manually engineered physiological models: Physiological Equations + Kalman filtering + SVR. 2. BGL prediction using trained physiological models. RNNs with LSTM units. 3. Attention models for time series prediction. 4. From physiological models to automated BGL control. 31

32 BGL Prediction: What next? We used as input only the history of BG levels, insulin and carbohydrates: Manually engineered physiological model equations. There are many other factors that can impact BG levels: Exercise, Stress, Depression, Medication, How do we measure the relevant factors? 2. How do we use more factors in the physiological model? 32

33 1. How to Measure? Exercise / effort has a significant impact on BGL behavior. Patients have been telling us when and how much they exercise: Unfeasible in practice. Very coarse-grained data. Better use sensors to acquire measurements of relevant physiological parameters: Skin conductivity. Heart rate. 3D acceleration. Skin and Air Temperature. 33

34 2. How to Model? Manually re-engineer the physiological model equations to incorporate exercise: Cognitively demanding, because complex problem => a complex model is required. Time consuming: the equations have to be re-engineered every time we add another physiological parameter. Automatically learn physiological models of BG behavior: Deep Learning mantra(s): The learning algorithm is simple: Fits on 1 slide (a bit more complicated in practice ) The learned behavior is complex: The complexity comes from the training data! 34

35 Manually Engineered State Transition Model G G State transition equations H t+1 = f(h t,u t+1 ) are recursive: Use RNNs to learn them! Neural networks are Universal Approximators 35

36 Equivalent Recurrent Neural Network t U t+1 H t G t H t+1 t+1 G t+1 G t+30, G t+60 36

37 Recurrent Neural Networks (RNN) U t = [BGL t-1, Insulin t, Carbs t, ] G t = BGL t G 1 G 2 G 2 G t G t+1 f(h 0,U 1 ) f(h 1,U 2 ) f(h t-1,u t ) f(h t,u t+1 ) H 1 H 2 H 2 H t H t+1 U 1 U 2 U 2 U t U t+1

38 RNN Training Procedure Use long short-term memory (LSTM) units: Experiments showed better performance than vanilla RNNs. The BG levels are scaled by 0.01, whereas all other input values are normalized in [0, 1]. Target is relative difference BG(t 0 +60) BG(t 0 ). Missing BG levels are linearly interpolated. [Mirshekarian et al., EMBC 17] Interpolated BG levels are never used as prediction targets during training. Backpropagation through time (BPTT) is done for 12 hours. This is also how long in the past the LSTM network will be unrolled at test time. The MSE objective is minimized using RMSProp with a batch of

39 Dealing with Cold Start Examples The number of training examples varies from only a few days for the first test points to more than two months for the last test points. To address the insufficient number of training examples for the early test points, training is done in two steps: Pre-training. Fine-tuning. 39

40 Pre-training An LSTM model is pretrained on 3 development patients + the other 4 test patients: Learning rate of Early stopping on 2 other development patients. The weights are initialized using the Glorot uniform scheme. The pretrained weights are used to initialize the parameters only for the first test example in each patient. For each subsequent test example, the weights are initialized with those learned for the previous test example. 40

41 Fine-tuning After pre-training, a separate model is fine-tuned on the BG level history for each test point. Use annealing for the learning rate. The number of epochs increases linearly with the number of training examples available in the BGL history for each test point: For test examples early in the sequence, the initial pre-trained weights will not be changed much. For a test example late in the sequence of BG levels, the initial weights could be changed more substantially, based on a much larger number of patient specific training examples. 41

42 Experimental Evaluation Setting The original dataset of 200 points contains 67 points that have what-if events, i.e. meals or carbs in the prediction horizon [t 0, t ]: Both the physiological model and the doctors had access to these events. Idea was to evaluate performance for what-if predictions. The RNN cannot use the what-if events... unless it is used to initialize a second RNN running from t 0 to t that does not see BG levels (future work!). We evaluated both the SVR+φ and the RNN on the 200 points, but ignoring the what-if events. 42

43 Trained Physiological Models vs. Manually Engineered Physiological Models RMSE on the 200 test points, ignore what-if events: TPM = Trained Physiological Model (RNN). EPM = Enginered Physiological Model (SVR+ARIMA+PM+Kalman). Horizon t 0 ARIMA EPM TPM 30 min min when trained on only previous 2 weeks, decreased to 37.4 RMSE on the 129 points without what-if events: Horizon t 0 ARIMA EPM TPM 60 min

44 Experimental Results 1. The trained Recurrent Neural Networks (LSTMs) are competitive with the manually engineered Physiological Models. Both are competitive with and often outperform expert physicians. 2. RNN model is more flexible, it can incorporate arbitrary physiological measurements: Do they help in practice? 44

45 45

46 Preliminary results Experimented with data from one patient, wearing a Basis band: LSTM: the original RNN system, trained on BGL, insulin and carbs. Tuned layer size 20, dropout rate 0.1, early stopping. Used only points with no what-if events in prediction horizon. LSTM+band: trained on BGL, insulin, carbs + heart rate, GSR, temperature. Ran 12 rounds, from different random initializations: LSTM: mean (std 0.35) LSTM+band: mean (std 0.22) 46

47 Outline 1. BGL prediction using manually engineered physiological models: Physiological Equations + Kalman filtering + SVR. 2. BGL prediction using trained physiological models. RNNs with LSTM units. 3. Attention models for time series prediction. 4. From physiological models to automated BGL control. 47

48 Memory-Augmented BGL Prediction History repeats itself: A patient may go through the same situation multiple times: Currently BGL around 100, ate 100 carbs 30 min ago, bolused 2mg 35 min ago. Similar situations expected to lead to similar future BGLs. Find the most similar situation at time t in the past, use its BGL(t + 60) as an additional feature: RNNs can theoretically learn long-term dependencies. In practice, hard to train to remember => use explicit memory. 48

49 Attention Model for Time Series Prediction G 1 G 2 G 2 G t f(h 0,U 1 ) f(h 1,U 2 ) f(h t-1,u t ) H 1 H 2 H 2 H t U 1 U 2 U 2 U t 49

50 Attention Model for Time Series Prediction u = argmax a H +, H ) )*+ a(h t,h 1 ) a(h t,h 2 ) Use a H +, H ) and G u+60 as features. f(h 0,U 1 ) f(h 1,U 2 ) a(h t,h 3 ) f(h t-1,u t ) H 1 H 2 H 2 H t U 1 U 2 U 2 U t 50

51 Attention Model for Time Series Prediction Use a two-hidden layer fully connected network to compute attention: a H +, H ) =σ(w 2 relu W 7 H + ; H ) + b2 + b1) Use a H +, H ) and G u+60 as input features for the linear output layer, together with H t, to predict G t or G t

52 Evaluation Datasets Diabetes simulators: AIDA, from Generated 20 individual patient profiles: Each patient has 600 independent days. BGLs come every 15 minutes. University of Viginia (Uva) from tegvirginia.com: Generated 5 patient profiles: Each patient has 90 consecutive days. BGLs come every 1 minute (use every 5 minute). Real patient data: 5 patients, each with 70 consecutive days. BGLs measured every 5 minutes. 52

53 AIDA Sample 53

54 Uva Sample 54

55 Experimental Results for 60 Minute Horizon Points with what-if events are kept: System AIDA UVa Real LSTM LSTM+Mem Marginal, but statistically significant (over 12 runs). Points with what-if events are removed: System AIDA UVa Real LSTM LSTM+Mem

56 Outline 1. BGL prediction using manually engineered physiological models: Physiological Equations + Kalman filtering + SVR. 2. BGL prediction using trained physiological models. RNNs with LSTM units. 3. Attention models for time series prediction. 4. From physiological models to automated BGL control. 56

57 Future Work: Automated BGL Control Pseudo-Inverse Physiological Model - Patient = Real Physiological Model η is desired BGL. η is actual BGL. μ is actual insulin delivered. Based on Figure from and discussion with Dr. Jim Zhu. μ is what-if insulin computed based on the physiological model. μb is insulin correction computed by the stabilizing controller, designed based on the physio model. 57