Development and Validation of a Tool to Identify Patients With Type 2 Diabetes at High Risk of Hypoglycemia-Related Emergency Department or Hospital Use Andrew J. Karter, PhD; E. Margaret Warton, MPH; Kasia J. Lipska, MD, MHS; James D. Ralston, MD, MPH; Howard H. Moffet, MPH; Geoffrey G. Jackson, MHA; Elbert S. Huang, MD, MPH; Donald R. Miller, ScD JAMA Intern Med. doi:10.1001/jamainternmed.2017.3844 Published online August 21, 2017. 1 Wan-Ting, Lin 2017/11/21
Introduction One in 4 emergency hospitalizations for adverse drug events is related to hypoglycemia, and these rates are higher in older patients. Severe hypoglycemia has been associated with falls and automobile accidents, cardiovascular autonomic dysfunction and ventricular arrhythmia, dementia, and death. The risk of hypoglycemia varies widely in patients with T2D. Whereas interventions to prevent hypoglycemia exist, there are no validated methods to target these interventions efficiently. Accordingly, we developed and validated a hypoglycemia risk stratification tool to categorize 12-month risk of hypoglycemia-related emergency department (ED) or hospital use among patients with T2D. 2
Aim Question Can electronic medical records be used to reliably categorize risk of future hypoglycemia-related emergency department or hospital use in patients with type 2 diabetes? 3
Study design We used a prospective cohort study design to develop a risk tool to categorize the 12-month risk of hypoglycemia-related ED or hospital use. Selection of pre-baseline candidate predictors was based on a literature review of clinical risk factors associated with hypoglycemia and limited to data typically available in electronic medical records (EMRs). 4
Study design We derived and internally validated this tool in a split sample (4:1) of 206,435 adult patients with T2D in an integrated health care delivery system(kaiser Permanente Northern California [KPNC]) using clinical and demographic data from EMRs. We used recursive partitioning in the derivation sample to create a risk classification tree. 5
Study design The classification tree leaf nodes were further categorized into high-, intermediate-, or low-risk groups on the basis of predicted risk and then transcribed into the hypoglycemia risk stratification tool. After testing the tool in the internal sample, we conducted external validation in 2 completely independent samples of patients with T2D from the Veterans Administration Diabetes Epidemiology Cohort (DEpiC) (VA sample, n = 1,335,966) and from Group Health Cooperative (GH sample, n = 14,972). 6
Study population Using EMR data from KPNC( 21 years as of the baseline date of January 1, 2014), N=233,330 Patients with unknown diabetes type, N=24,719 Patients with probable type 1 diabetes, N=3,615 Eligible patients with T2D, N=206,435 Randomly 80% 20% Derivation sample, N=165,148 Internal validation sample, N=41,287 Similar eligibility criteria were applied in the creation of the 2 external validation samples. 7
Outcome Our outcome was the occurrence of any hypoglycemia related ED or hospital use during 12 months post-baseline. (ICD-9), codes: 251.0, 251.1, 251.2, 962.3, or 250.8,without concurrent 259.8, 272.7, 681.XX, 682.XX, 686.9X, 707.1-707.9, 709.3, 730.0-730.2, or 731.8 codes) Secondary discharge diagnoses for hypoglycemia were not used because they are often attributable to events that occurred during the ED or hospital encounter (eg, inpatient insulin management, sepsis, acute renal failure). 8
Exposures On the basis of a literature review of clinical risk factors associated with hypoglycemia in T2D, we selected 156 (122 categorical and 34 continuous) candidate clinical, demographic, and behavioral predictor variables for model development (Supplement). we excluded variables that were expensive or impractical to collect, had ambiguous meanings, or were not typically available in an EMR. Medication exposures were based on pharmacy dispensings during 6 months pre-baseline; laboratory values were based on the last test results with in 2 years pre-baseline; and prior medical events (eg, history of hypoglycemia-related utilization using the same outcome definition) were based on all available pre-baseline records (maximum, 18 years of medical history; mean [SD], 16.5 [3.4] years). 9
Statistical analysis We first regressed the outcome(any hypoglycemia-related utilization) on each of the 156 candidate predictors using univariate logistic regression models to generate odds ratios. selected candidate variables that had a resulting P <.10 We then used recursive partitioning (using SAS JMP, version 12) on the selected candidates to construct a binary classification tree to predict the occurrence of at least 1 hypoglycemia related utilization episode 12 months post-baseline. This method uses a machine-learning, nonlinear, and nonparametric approach to split ( partition ) events into pairs of subgroups based on continuous or categorical predictors, and has the unique advantage that it identifies complex nested interactions, unlike linear modeling methods. 10
Validation Studies Discrimination, the ability of a model to accurately distinguish between subjects who do vs do not develop the outcome, is based on the area under the receiver-operator curve (Cstatistic),with greater than 0.7 classified as good discrimination. We also visually assessed calibration (the extent to which the predicted risks over or underestimate the observed risks) using calibration plots. 11
Validation Studies We further evaluated components of generalizability (ie, reproducibility in patients not used for the derivation of the model) and transportability (practical application in different settings) in the 2 external primary care T2D populations (VA and GH samples). 12
Sensitivity Analyses We conducted further analyses to evaluate whether our tool was sensitive to restrictions in available length of enrollment (pre-baseline period transportability). 短期收案時間 We also evaluated temporal sensitivity (historical transportability)of the tool by applying it to KPNC data in the subsequent year (using a baseline date of January 1, 2015). 時間敏感度 We also evaluated tool performance when patients with type 1 diabetes were included (ie, sensitivity to misclassification). 13
Sensitivity Analyses Finally, as a measure of ecological validity of this tool, we evaluated the association between the predicted level of risk of hypoglycemia-related utilization and actual, self-reported severe hypoglycemia events based on a 2005 survey of 15,231 patients with T2D. 生態效度 14
15
16
17
18
OR: 34.6 (24.2-49.3) OR: 20.7 (8.6-45.0) OR: 23.3 (21.9-23.7) 19
C statistic: 0.83 C statistic: 0.79 C statistic: 0.81 高估 高估 20
Result In the sensitivity analyses for length of available medical history, good discrimination was confirmed despite shorter medical history (C statistic = 0.82, 0.83, 0.84 for 2, 5, and 10 years, respectively). 短期收案時間 We then evaluated temporal sensitivity (historical transportability33). There was good discrimination (C statistic = 0.83) even after KPNC experienced a significant 21% increase in the rate of hypoglycemia-related utilization (0.48% vs 0.59% for 2014 and 2015, respectively). 時間敏感度 21
Result We also evaluated sensitivity to misclassification of diabetes type by including all diabetes patients in our sample and found good discrimination (C statistic = 0.84). Patients categorized as high risk by the tool were 5 times more likely (49.7% vs 9.2%; P <.001) to self-report a severe hypoglycemic episode relative to those categorized as low risk. 生態效度 22
Limitations We excluded secondary discharge diagnoses for hypoglycemia because these events may occur during the ED or hospital encounter. Although we did not include those events in our model development, we estimate that this would include less than 2% of ED encounters (data not shown). Thus, while the tool successfully stratifies the population into 3 levels of risk, it should not be used to estimate the probability of hypoglycemic-related utilization for an individual patient. 23
Limitations Geographic, methodologic, and spectrum transportability were also demonstrated because these external validation populations were from different geographical locations, used differing methods of caring for and identifying patients with T2D, and had different distributions of disease severity and case mix. Some of the inconsistent findings are attributable to sparse data in the GH validation sample; only 45 events were observed during follow-up. The tool was designed to predict hypoglycemia-related utilization and thus did not take into account severe hypoglycemia occurring outside the health care system. 24
Conclusion This hypoglycemia risk stratification tool categorizes the 12- month risk of hypoglycemia-related utilization in patients with T2D using only 6 inputs. This tool could facilitate targeted population management interventions, potentially reducing hypoglycemia risk and improving patient safety and quality of life. 25
Thanks for your listening. 26