Using Random Forest Models to Identify Correlates of a Diabetic Peripheral Neuropathy Diagnosis from Electronic Health Record Data

Transcription

1 Pain Medicine 2017; 18: doi: /pm/pnw096 Original Research Article Using Random Forest Models to Identify Correlates of a Diabetic Peripheral Neuropathy Diagnosis from Electronic Health Record Data Sarah DuBrava, MS,* Jack Mardekian, PhD, Alesia Sadosky, PhD, E. Jay Bienen, PhD, Bruce Parsons, MD, Markay Hopps, MPH, and John Markman, MD *Pfizer, Inc, Groton, Connecticut; Pfizer, Inc, New York, New York; Outcomes Research Consultant, New York, New York; Department of Neurosurgery, University of Rochester Medical Center, Rochester, New York, USA Correspondence to: Sarah DuBrava, MS, Clinical Statistics, Pfizer - Primary Care Business Unit, 445 Eastern Point Rd, MS , Groton, CT 06340, USA. Tel: ; Fax: ; sarah.j.dubrava@pfizer.com. Funding source: This study was funded by Pfizer, Inc. Disclosure: Alesia Sadosky, Sarah DuBrava, Jack Mardekian, Bruce Parsons, and Markay Hopps are employees and shareholders of Pfizer, the sponsor of this study; E. Jay Bienen is an independent scientific consultant who was funded by Pfizer in connection with manuscript development; John Markman collaborated with Pfizer on the project but was not financially compensated for his involvement, including for manuscript development. Abstract Objective. To identify variables correlated with a diagnosis of diabetic peripheral neuropathy (DPN) using random forest modeling applied to electronic health records. Design. Retrospective analysis. Setting. Humedica de-identified electronic health records database. Subjects. Subjects 18 years old with type 2 diabetes from January 1, 2008 September 30, 2013 having continuous data for 1 year pre- and postindex with DPN (n 5 35,050) and without DPN (n 5 288,328) were identified. Methods. Demographic, clinical, and health care resource utilization variables (e.g., inpatient and outpatient encounters, medications, and procedures) were input into a random forest model to identify the most important correlates of a DPN diagnosis. Random forest modeling is a computationally extensive, robust data mining technique that accommodates large sets of variables to identify associated factors using an ensemble of classifications trees. Accuracy of the model was evaluated using receiver operating characteristic curves (ROC). Results. The final random forest model consisted of the following variables (importance) associated with a DPN diagnosis: Charlson Comorbidity Index score (100%), age (37.1%), number of pre-index procedures and services (29.7%), number of pre-index outpatient prescriptions (24.2%), number of preindex outpatient visits (18.3%), number of pre-index laboratory visits (16.9%), number of pre-index outpatient office visits (12.1%), number of inpatient prescriptions (5.9%), and number of pain-related medication prescriptions (4.4%). ROC analysis confirmed model performance, with an area under the curve of and accuracy of 89.6% (95% confidence interval 89.4%, 89.8%). Conclusions. Random forest modeling can determine likelihood of a DPN diagnosis. Further validation of the random forest model may help facilitate earlier diagnosis and enhance management strategies. Key Words. Diabetes; Diabetic Peripheral Neuropathy; Random Forest Model; Electronic Health Records; Health Care Resource Utilization VC 2016 American Academy of Pain Medicine. All rights reserved. For permissions, please journals.permissions@oup.com 107

2 DuBrava et al. Introduction With an estimated prevalence in the United States of 9.3% [1], diabetes is a major health care issue. The prevalence is likely to increase as a result of the aging of the population and changes in lifestyle factors, the latter of which has especially been noted to contribute to a higher rate of type 2 diabetes among children and adolescents [2]. Diabetic peripheral neuropathy (DPN) is one of the most common neurologic complications of diabetes, resulting from disease processes that include decreased microvascular blood flow due to poor glycemic control [3 5]. DPN represents a spectrum from being asymptomatic to that accompanied by painful symptoms that may range from mild to severe, in which case it is known as painful diabetic peripheral neuropathy (pdpn). In contrast to diabetes, which can potentially be reversed in some patients, the underlying source of DPN, damage to the peripheral nerves, is irreversible, and there is no cure. Thus, the need to appropriately manage DPN becomes a long-term endeavor once it develops. The presence of DPN, with or without pain, is associated with significantly greater health care resource utilization and direct medical costs relative to patients with diabetes, with the highest utilization and costs among pdpn patients reporting severe pain [6]. Additionally, resource utilization and costs have been reported to substantially increase, by almost 50%, after a diagnosis of DPN [7]. Thus, reducing DPN may not only be of benefit to patients, but may also be advantageous to the health care system. A first step toward this goal is determining the likelihood of DPN among individuals with type 2 diabetes by identifying specific factors associated with a diagnosis. A limited number of studies have evaluated a variety of demographic and disease factors that may be associated with developing DPN [8 12]. In one study, DPN significantly correlated with diabetes duration, quality of metabolic control, cigarette smoking, and the presence of cardiovascular disease, and DPN severity correlated with hypertension, dyslipidemia, microalbuminuria, alcohol consumption, and body mass index [11]. However, the cross-sectional nature of this study limited inferences about whether these risk factors are independent contributors to DPN or are concomitant comorbidities associated with poor glycemic control. While additional studies have consistently reported that glycemic control is a key predictor of DPN and have provided data suggestive of other associated factors [9,10,12], there remains a paucity of studies evaluating a complete range of variables. One approach is to use real-world data to identify variables that correlate with a DPN diagnosis. This approach can inform health care providers of patients who may need specific evaluation, and identify patients who are at risk, including undiagnosed patients, which can facilitate effective management strategies. The greater availability of electronic health records (EHRs), which are increasingly being implemented in health care systems to manage patient care and improve quality outcomes, represent real-world, patientlevel data and present an opportunity to evaluate a variety of variables for the value of their correlation. EHRs capture integral components of care that may not be readily available in claims databases, including information from all providers involved in a patient s care across multiple health care organizations. Use of such data can be of potential importance to managed care stakeholders to enable identification of discrete populations and for making treatment decisions. New ways of evaluating such data can expand on its application to the clinical setting. Therefore, the objective of this analysis was to identify variables that correlate with a DPN diagnosis by applying a modeling technique, random forest, to EHR data. Random forest modeling is a computationally extensive, robust data mining technique that can accommodate large sets of proposed variables as input to identify factors associated with the outcome of interest using an ensemble of regression or classification trees [13]. Both categorical and numerical outcomes can be included as variables in the model, and it has good predictive accuracy properties, having built-in cross-validation that enables ranking of the importance of the variables associated with the outcome. Random forest has recently been used to identify predictors of a fibromyalgia diagnosis based on EHR data [14], and another study suggested that this technique may be of potential value for the diagnosis and evaluation of progression of diabetic retinopathy [15]. Methods Data Source The data source for this retrospective analysis is structured, de-identified EHR data from the Humedica database ( which has broad geographic representation encompassing approximately 30 million patients and contains information on demographics, diagnoses, and health care resource utilization including inpatient and outpatient encounters, medications, and procedures. Humedica extracts data from each health care provider s EHR system and normalizes the data from among the disparate systems regardless of which EHR is used; many of the more than 20 provider groups run EHR systems from multiple vendors over disparate sites of care. Subjects A total of 323,378 adult patients ( 18 years of age) with type 2 diabetes were identified from the database 108

3 based on ICD-9-CM diagnosis codes from January 1, 2008 September 30, 2013 and having continuous data for 1 year pre- and postindex. From among these patients, a DPN cohort (n ¼ 35,050) was identified based on an ICD-9 code for DPN (357.2 or 250.6). For patients identified with DPN, the date of DPN diagnosis was the index date, and the date of the diabetes diagnosis was the index date for subjects without DPN. Random Forest Modeling From the total population, a training dataset (n ¼ 216,279) was used for model development, and a test dataset (n ¼ 107,099) was reserved to confirm model performance. The random forest model was developed to identify variables that correlate with a DPN diagnosis among those with diabetes based on input into the model of 70 variables (Supplementary Data Table S1). Variables used in the model were derived from the 1 year pre-index data. These variables included demographics; clinical characteristics such as the Charlson Comorbidity Index (CCI), which was included because it is a useful measure that provides an inventory of clinical comorbidities to enhance understanding of the medical history of patients; health care resource characteristics captured from the EHR database that were previously identified as being significantly different between the DPN and diabetes cohorts [6]; and variables identified from the literature as being potentially correlated [11]. However, as not all databases are alike, not all risk factors can be identified. All available data were used in defining variables with no imputation for missing data; routinely reported data that were missing were left out of the analysis. All available ICD-9 codes were used in defining variables, hence the absence of an ICD-9 code was not considered missing data. The variables were entered into the random forest model (repeated 5 cross-validations of 10-fold each with 1,500 bootstraps) with internal downsampling to account for cohort imbalance, as DPN was present in 10.8% of the diabetes population. The top 10 most important correlated variables were identified among the 70 variables input into the model and ranked based on normalization to 100% for the variable with the largest loss in performance by its omission from the model. The model was then rerun with these 10 variables to further characterize their importance in the absence of the influence of the other variables. Accuracy (sensitivity and specificity) of the model for identifying correlates was evaluated using receiver operating characteristic curves (ROC). In order to enhance the interpretability of the variables identified in the model, a set of rules was developed using the C5.0 technique [16], which generates sets of criteria that can be used to identify subsets of subjects who have a high probability of DPN or a high probability of no DPN; 0.70 was considered reasonable as a high Diabetic Peripheral Neuropathy Predictive Modeling cutoff value for classifying a patient as having DPN, and 0.20 was a low DPN probability. The minimum, maximum, and 20th, 40th, 60th, and 80th percentiles of the 10 factors are first computed based on the training data set. Each combination of the 10 factors across the six percentiles is then used to create 6 10 possible combinations. One example combination used in a simulated data set consists of all 10 factors at their minimum value in the training set. Duplicate values of combinations and combinations resulting in probabilities of DPN between 0.20 and 0.70 are eliminated, resulting in 148,070 combinations and their probabilities in the simulated data set to derive the C5.0 rules. While any of these rules could be used for estimating the probability of a DPN or no DPN diagnosis, each component of the rule must be satisfied in order to make the determination. Results Subjects Table 1 presents the demographic characteristics as previously described [6], and shows that relative to diabetes, the DPN cohort was significantly older (64.8 vs 61.4 years; P < ) and had a higher proportion of males (49.0% vs 46.8%; P < ) and smokers (34.2% vs 28.9%; P < ). While both cohorts were more than three-quarters white, the overall racial distribution was significantly different (P < ), as was geographic distribution (P < ). Both cohorts were characterized by similar body mass indices that indicated obesity ( 30). Random Forest Model From among all the variables evaluated during the 1- year pre-index period that were used as input into the random forest model, the top 10 variables associated with a DPN diagnosis are shown in Figure 1, along with their relative importance for correlating with a DPN diagnosis, normalized to 100% for the variable showing the greatest importance. Except for the pre-index CCI score, which was the most important correlated variable; age; and hypertension, the other identified variables were associated with health care resource utilization. After the CCI score, the variables with the highest importance were number of procedures and services (30.8%), followed by number of outpatient prescriptions (29.2%) and age (24.3%). The ROC curve confirmed the accuracy of the model variables (Figure 2), with an area under the curve (cstatistic) of and sensitivity and specificity of and 0.846, respectively. The ROC curve also showed that at a probability of 0.500, sensitivity was and specificity was When the model was rerun using only these 10 variables in the training data set (Figure 3), the CCI score maintained its relative importance (100%) and hypertension dropped out, as its importance was 0%. The rest 109

4 DuBrava et al. Table 1 Demographic characteristics of the cohorts [6] Variable Diabetes* DPN (n ¼ 288,328) (n ¼ 35,050) P Age, mean (SD) 61.4 (13.4) 64.8 (12.2) < Gender, n (%) < Female 153,463 (53.2) 17,871 (51.0) Male 134,761 (46.8) 17,166 (49.0) Race, n (%) < African American 42,476 (14.7) 5,328 (15.2) Asian 3,209 (1.1) 204 (0.6) Caucasian 219,016 (76.0) 27,460 (78.3) Other/Unknown 23,627 (8.2) 2,058 (5.9) Smoker, n (%) 83,220 (28.9) 11,980 (34.2) < Region < Midwest 140,213 (48.6) 17,889 (51.0) Northeast 12,515 (4.3) 1,926 (5.5) South 123,907 (43.0) 13,736 (39.2) West 6,537 (2.3) 809 (2.4) Unknown 5,156 (1.8) 690 (2.0) Average household income, mean (SD) $39,740 (9,918) $39,579 (9,690) Body mass index, kg/m 2, mean (SD) 33.6 (7.4) 33.7 (7.5) *Does not include patients with neuropathy codes. Charlson Comorbiditiy Index Number of procedures/services Number of outpatient prescriptions Age Number of laboratory visits Number of outpatient visits Number of outpatient office visits Number of inpatient prescriptions Hypertension Number or pain-related medication prescriptions of the variables displayed a change in importance from the run of the full model, with age moving up to the second ranking (37.1%) followed by number of procedures and services (29.7%) and the other health care resource utilization variables. The sensitivity of this model version was with a specificity of and a c- statistic of % 29.2% 24.3% 23.0% 19.3% 16.2% 12.7% 12.3% 11.6% Importance (normalized to 100% for the most predictive variable) Figure 1 Top 10 variables associated with a diabetic peripheral neuropathy (DPN) diagnosis identified using the training data set from the full random forest model with Charlson Comorbidities for the prior year up to but not including the index date. the class severity. The accuracy of this model was 89.6% (95% confidence interval 89.5%, 89.8%), with a c-statistic of The predictive cutoff value of resulted in sensitivity and specificity of and 0.894, respectively. Enhancing Model Interpretability 100% As shown in Figure 4, the results obtained with the training data were confirmed by applying the top 10 variables to the test data set with no adjustments on The rule-based approach resulted in 21 sets of rules (Supplementary Data Table S2), 14 that correlated with DPN and 7 that correlated with no DPN; any one 110

5 Diabetic Peripheral Neuropathy Predictive Modeling Figure 2 Receiver operating characteristic curve (ROC) evaluating the sensitivity and specificity of the random forest predictive model based on the training data set. Charlson Comorbiditiy Index Age Number of procedures/services Number of outpatient prescriptions Number of outpatient visits Number of laboratory visits Number of outpatient office visits Number of inpatient prescriptions Number or pain-related medication prescriptions 0 5.9% 4.4% 18.3% 16.9% 12.1% 24.2% 29.7% 37.1% Importance (normalized to 100% for the most predictive variable) Figure 3 Importance and ranking of variables associated with a diabetic peripheral neuropathy (DPN) diagnosis using random forest modeling on only the top 10 identified variables. 100% of these rules could be used to determine whether a subject is likely to be diagnosed with DPN provided that each component of the rule is satisfied. As examples from the total set of rules, Table 2 presents the four rules (#5, #6, #9, and #13) that resulted in the largest number of test subjects identified with DPN, and the two rules (#16 and #19) that resulted in the largest number of test subjects identified with no DPN. Among the 148,070 subjects in the simulated dataset, 32,161 cases satisfied the conditions of Rule #5, and 100% of these cases were correctly identified as belonging to the DPN-associated class (Table 2). When applied to the test data set of 107,099 subjects, 19.1% were identified as DPN, with a sensitivity of 88.7% and specificity of 111

6 DuBrava et al. Figure 4 Receiver operating characteristic curve (ROC) for the test data set. 66.7%, implying that a subject with characteristics satisfied by this rule has a high potential for a DPN diagnosis. Similarly, Rule #16 identified 99.9% of the no-dpn predictive class from the simulated data set, and in the test data set, 28.8% of subjects were identified as no DPN, with a sensitivity of 98.4% and a specificity of 3.7%. Discussion While determining the likelihood of a DPN diagnosis from data elements that are captured in routine clinical practice can be of potential value to health care providers, identifying such correlates from the wide array of data elements requires using techniques capable of processing large sets of variables. A recent study showed significant differences between patients across the spectrum of DPN (DPN, painful DPN, and severe painful DPN) relative to diabetes for a range of demographic, clinical, and health care resource variables [6]. The current analysis used random forest modeling as a novel approach to expand on that study by evaluating those variables for their importance as correlates. The results show that of the 10 identified variables, 7 were related to the differences in health care resource utilization reported in that study, with 1 demographic variable, age, and 2 clinical variables, CCI, and hypertension. When the model was rerun to more fully characterize these variables, hypertension was shown not to be correlated, and the model retained all the health care resource use variables. The high importance of CCI is likely a result of the presence of diabetes-related comorbidities in this assessment measure, although the CCI did not include the index date. Similarly, age may be expected to be a correlated variable, as type 2 diabetes and DPN are agerelated conditions. The presence of variables related to health care resource use was not surprising given the higher rate of health care resource use that has been reported in patients with DPN relative to those with diabetes without DPN [6,17 19]. However, the predominance of these variables, comprising 7 of the 9 variables in the final model, was unexpected. Interestingly, nearly all of the health care resource variables that were identified as associated with a DPN diagnosis were related to outpatient resources. The only inpatient resource was Number of inpatient prescriptions. In particular, the most important correlated health care resource variable was number of procedures and services, which catalogues the wide variety of episodes of care used to record health maintenance and can be considered a metric of visit-care intensity; each procedure is accompanied by an ICD-9, Healthcare Common Procedure Coding System (HCPCS), or Current Procedural Terminology (CPT) code. Health care resource utilization variables as diagnostic correlates have also been reported for fibromyalgia using similar methodology [14], suggesting that the potential importance of these variables as a metric for identifying patient subsets is unrecognized. Further evaluation may help determine the appropriateness of applying this metric, especially with regard to specific resource categories, to EHR data. The ROC curve analysis supported the validity of the model, with c-statistics for the training and test sets consistently above the threshold of 0.80 that represents good accuracy. For enhancing interpretability of these 112

7 Diabetic Peripheral Neuropathy Predictive Modeling Table 2 The top four rules to identify diabetic peripheral neuropathy (DPN) and two rules to identify no DPN that resulted in the largest number of test subjects based on the results of the model using a technique known as C5.0 rules [16]. The full set of rules is provided in Supplementary Data Table S2 Rule number Predictive class Rule (all components must be met) data, development of sets of rules can be used to differentially evaluate the likelihood of DPN and no DPN based on the availability of data among specific variables identified by the random forest model. As shown by the results reported here, these rules can further elucidate the factors that drive individual correlates of a diagnosis, and which can thus be used to identify individuals who may require additional screening to determine the potential presence or risk of DPN. However, it should also be noted that specificity of these rules was low, although this may be expected based on the imbalance between DPN and no DPN; subjects with DPN represented 10.8% of the evaluated cohort. Number of subjects predicted in simulated data set (N ¼ 148,070) to belong to predictive class Percent of subjects correctly identified in predictive class Number (%) of subjects from test data set (N ¼ 107,099) identified by rule [sensitivity, specificity] 5 DPN Outpatient prescriptions written > 10 32, ,411 (19.1) [88.7, 66.7] 6 DPN Charlson Comorbidity Index Score > 0 61, ,959 (8.4) Age > 18 [84.8, 42.8] Outpatient office visits 2 9 DPN Charlson Comorbidity Index Score > 0 31, ,208 (6.7) 5 < Procedures/services 16 [83.3, 47.4] 13 DPN Charlson Comorbidity Index Score > 1 121, ,693 (7.2) [98.3, 15.1] 16 No-DPN Charlson Comorbidity Index Score ¼ 0 8, ,892 (28.8) Outpatient prescriptions written 10 [98.4, 3.7] Outpatient office visits > 0 19 No-DPN Charlson Comorbidity Index Score ¼0 10, ,827 (55.9) Outpatient prescriptions written 10 [97.9, 2.8] Laboratory visits 4 Outpatient office visits 7 Although a strength of this study is its external validity resulting from use of real-world EHR data from multiple sites, some limitations should be noted, including the fact that derivation of the variables was restricted to the 1-year pre-index period and limited to those variables that could be identified in EHR claims data. Among these variables, the number of procedures and services was defined at a high level and did not include a level of detail that captured education, nutrition consultations, and drug refill services. Additionally, as with all such database analyses that rely on ICD-9 coding, there exists the potential for errors in coding or recordkeeping during data collection. In this regard, another limitation is that there may be nonuniform application of the DPN diagnostic codes depending on the physician s specialty, which was not available for all diagnoses. Future studies using this methodology should require specification of provider specialty to understand the value of this variable as a correlate with a DPN diagnosis. Furthermore, only patients with type 2 diabetes were included in this analysis in order to evaluate a more homogeneous population; type 1 patients develop diabetes at a younger age and show a lower prevalence of DPN pain (22%) than those with type 2 diabetes (38%) [20]. Thus, another limitation is the generalizability of these results, as it should be considered that a different set of variables could be identified as correlates among patients with type 1 diabetes or among an evaluated diabetes population unrestricted by type. However, the results reported here may be considered relevant from a managed care perspective, given that approximately 90% of all patients with diabetes are type 2 and that medical records generally identify diabetes by type. The model could potentially also be criticized for not including poor glycemic control and diabetes duration as variables. Nevertheless, the identified variables, while not specific for the purpose of making a DPN diagnosis, can be used to identify a subset of 113

8 DuBrava et al. individuals who may require more comprehensive evaluation for DPN, which has an insidious onset and may be initially asymptomatic in a proportion of patients [21,22]. Conclusions In summary, this analysis showed that random forest modeling can be applied to EHR data to determine likelihood of a DPN diagnosis. While the CCI appeared to be the most important correlate of a DPN diagnosis, suggesting a substantial contribution of comorbid conditions, the random forest methodology also identified several health care resource utilization variables as being among the important variables, with Number of procedures and services the most important among the health care resource variables, indicating a high intensity of care during visits. Additional validation of random forest analysis for determining the likelihood of a DPN diagnosis may help facilitate identification of patients who may benefit from more comprehensive screening and/or earlier therapeutic intervention. These results also suggest that in the landscape of today s evolving health care system with the increasing availability and importance of real-world data such as those captured by EHR, random forest modeling may be a useful analytic approach for identifying and characterizing a wider array of variables associated with health outcomes. In particular, as highlighted by these results, further exploration of differences in resource utilization as a factor that is associated with a DPN diagnosis may be warranted, especially for use in epidemiologic research and for potentially identifying patients with presymptomatic or asymptomatic DPN. Supplementary Data Supplementary Data may be found online at Acknowledgments We would like to thank Birol Emir for his expert insight into study design and execution and assistance with manuscript development. References 1 Centers for Disease Control and Prevention. National Diabetes Statistics Report: Estimates of diabetes and its burden in the United States, Atlanta, GA: U.S. Department of Health and Human Services Available at: betes/pubs/statsreport14.htm (accessed 2014 June 20). 2 Mohamadi A, Cooke DW. Type 2 diabetes mellitus in children and adolescents. Adolesc Med State Art Rev 2010;21(1): Young MJ, Bennett JL, Liderth SA, et al. Rheological and microvascular parameters in diabetic peripheral neuropathy. Clin Sci 1996;90(3): Quattrini C, Harris ND, Malik RA, Tesfaye S. Impaired skin microvascular reactivity in painful diabetic neuropathy. Diabetes Care 2007;30(3): Fowler MJ. Microvascular and macrovascular complications of diabetes. Clin Diabetes 2008;26(2): Sadosky A, Mardekian J, Parsons B, et al. Healthcare utilization and costs in diabetes relative to the clinical spectrum of painful diabetic peripheral neuropathy. J Diabetes Complications 2015;29(2): Mehra M, Merchant S, Gupta S, Potluri RC. Diabetic peripheral neuropathy: Resource utilization and burden of illness. J Med Econ 2014;17(9): Tesfaye S, Stevens LK, Stephenson JM, et al. Prevalence of diabetic peripheral neuropathy and its relation to glycaemic control and potential risks factors: the EURODIAB IDDM complications study. Diabetologia 1996;39: Adler AI, Boyko EJ, Ahroni JH, et al. Risk factors for diabetic peripheral sensory neuropathy. Results of the Seattle Prospective Diabetic Foot Study. Diabetes Care 1997;20(7): Adler A. Risk factors for diabetic neuropathy and foot ulceration. Curr Diab Rep 2001;1(3): Tesfaye S, Chaturvedi N, Eaton SE, et al. Vascular risk factors and diabetic neuropathy. N Engl J Med 2005;352(4): Cardoso CR, Salles GF. Predictors of development and progression of microvascular complications in a cohort of Brazilian type 2 diabetic patients. J Diabetes Complications 2008;22(3): Breiman L. Random forests. Mach Learn 2001;45 (1): Emir B, Mardekian J, Masters ET, et al. Predictive modeling of a fibromyalgia diagnosis: Increasing the accuracy using real world data [abstract]. Arthritis Rheumatol 2014;66 Suppl(11):S Casanova R, Saldana S, Chew EY, et al. Application of random forests methods to diabetic retinopathy classification analyses. PLoS One 2014;9(6):e Kuhn M, Johnson K. Applied Predictive Modeling. New York: Springer;

9 Diabetic Peripheral Neuropathy Predictive Modeling 17 Ritzwoller DP, Ellis JL, Korner EJ, Hartsfield CL, Sadosky A. Comorbidities, healthcare service utilization and costs for patients identified with painful DPN in a managed-care setting. Curr Med Res Opin 2009;25(6): dacosta DiBonaventura M, Cappelleri JC, Joshi AV. A longitudinal assessment of painful diabetic peripheral neuropathy on health status, productivity, and health care utilization and cost. Pain Med 2011;12 (1): Dworkin RH, Panarites CJ, Armstrong EP, Malone DC, Pham SV. Healthcare utilization in people with postherpetic neuralgia and painful diabetic peripheral neuropathy. J Am Geriatr Soc 2011;59(5): Mixcoatl-Zecuatl T, Calcutt N. Biology and pathophysiology of painful diabetic polyneuropathy. In: Lawson E, Backonja M, eds. Painful Diabetic Polyneuropathy: A Comprehensive Guide for Clinicians. New York: Springer; 2013: Dyck PJ, Davies JL, Wilson DM, et al. Risk factors for severity of diabetic polyneuropathy: Intensive longitudinal assessment of the Rochester Diabetic Neuropathy Study cohort. Diabetes Care 1999;22(9): Boulton AJM, Malik RA, Arezzo JC, Sosenko JM. Diabetic somatic neuropathies. Diabetes Care 2004; 27: