Diabetes risk scores and death: predictability and practicability in two different populations

Diabetes risk scores and death: predictability and practicability in two different populations Short Report David Faeh, MD, MPH 1 ; Pedro Marques-Vidal, MD, PhD 2 ; Michael Brändle, MD 3 ; Julia Braun, PhD 1 ; Sabine Rohrmann, PhD 1 1 Unit of Demography and Health Statistics and Division of Cancer Epidemiology and Prevention, Institute of Social and Preventive Medicine, University of Zurich, Zurich, Switzerland 2 Institute of Social and Preventive Medicine (IUMSP), anne University Hospital, anne, Switzerland 3 Division of Endocrinology and Diabetes, Kantonsspital St. Gallen, St. Gallen, Switzerland Correspondence: David Faeh, Institute of Social and Preventive Medicine (ISPM) University of Zurich Hirschengraben 84 8001 Zurich, Switzerland Tel.: 044 634 46 16 Fax.: 044 634 49 86 Mail: david.faeh@uzh.ch 1

Abstract The aim was to examine the capacity of commonly used type two diabetes mellitus (T2DM) risk scores to predict overall mortality. The US-based (n=3138; 982 deaths) and the Swiss-based Co study (n=3946; 191 deaths) were used. The predictive value of eight T2DM risk scores regarding overall mortality was tested. The Griffin score, based on few, self-reported parameters, presented the best ( ) and second best (Co) predictive capacity. Generally, the predictive capacity of scores based on clinical (anthropometrics, lifestyle, history) and biological (blood parameters) data was not better than of scores based solely on clinical, self-reported data. T2DM scores can be validly used to predict mortality risk in general populations without diabetes. Comparison with other scores could further show whether such scores also suit as a screening tool for quick overall health risk assessment. Key words diabetes; risk scores; risk prediction; mortality 2

Introduction Several scores have been proposed to identify persons at risk for type two diabetes mellitus (T2DM) 1-3. Many variables used in T2DM risk scores are common health risk factors, e.g. smoking, hypertension and (central) obesity. Hence, T2DM risk scores could possibly also serve to predict disease and death independently from T2DM. However, to date, only one score has been examined for this ability 4. The advantage of these scores over other risk scores (e.g. cardiovascular risk scores) is that most T2DM risk scores do not require blood sampling to obtain biological parameters (e.g. blood lipids). Thus, T2DM risk scores could be used for a general health risk assessment, potentially leading to lifestyle interventions. A comparison of different T2DM risk scores could show which score best meets this requirement. In this study, we assessed the predictive capacity of eight T2DM risk scores regarding overall mortality. Because there may be substantial variation in the predictive performance of the scores according to the population 3, one European and one US population-based sample was used. Methods Data from two population-based prospective studies was used. The US was conducted in 1988-94 and had mortality follow-up until December 2006 5 ; for this study, only data from non-hispanic white participants with fasting blood glucose <7 mmol/l and without T2DM was used (n=3138; 982 deaths). The Swiss Co study was conducted in 2003-6 and had mortality follow-up until September 2012 6 ; for this study, only data from participants without T2DM was used (n=3946; 191 deaths). Eight T2DM risk scores relying on self-reported information on anthropometrics, lifestyle or medical history (clinical) or on a combination of clinical factors and blood parameters (biological) were used: Balkau, Wilson, Griffin, Kahn (clinical and clinical+biological), FINDRISK, the Swiss and the German Diabetes Risk Score (GDRS) 1,3,4,7. Participants with missing information in any of the scores were excluded. The GDRS could only be calculated using the data, because no information on dietary habits is available in Co. For each score, a continuous (score) and a binary (risk yes/no) variable were computed. The predictive capacity regarding death from all causes was assessed using Harrell s C 8 and Area-under-the-curve (AUC) for the continuous variable. For the binary variable, we used sensitivity, specificity, positive and negative predictive values (PPV and NPV). No adjustment was made for age or sex as these variables were part of some scores. The predictive capacity of three individual biological parameters was also assessed: fasting blood glucose, glycated hemoglobin (HbA 1 c) and insulin resistance (HOMA-IR). Results The populations used to develop the T2DM scores and the variables the scores are composed of are shown in Tables A1 and A2 in the supplementary file. In, the percentage of participants for whom the score was not assessable due to missing information varied from 0% (Griffin) to 17% (Kahn s clinical + biological), whereas no participant from Co had missing information. The prevalence of participants at risk of T2DM varied between scores and, to a smaller extent, between surveys. In, the Griffin score presented the best predictive capacity, as shown by the largest Harrell s C, AUC (continuous variable), PPV and NPV (binary variable). In Co, Kahn s clinical score had the best predictive capacity, but the Griffin score ranked second best. Wilson, Balkau, Findrisc and SDRS had a high specificity but a very poor sensitivity (binary), thus being inappropriate for the prediction of death. The Griffin score offered the best combination of a high specificity and an acceptable sensitivity. In both surveys, the predictive capacity of scores based on clinical and biological data was not better than of scores based solely on clinical data (Table). Among individual biological parameters, HbA 1 c had the best predictive capacity in, while insulin resistance (HOMA-IR) had the lowest predictive capacity in both studies (Table). Discussion We examined whether T2DM risk scores could be envisaged as screening tool for overall health risk assessment and used all-cause mortality as outcome proxy. The capacity of T2DM scores to predict mortality differed considerably within a study population. However, the scores also performed differently in the two studies. The scores also varied regarding the number and the type of required variables, which was reflected in a broad range of the percentage of participants excluded because of missing values. Looking at both studies, the Griffin score generally had the best predictive capacity. Further, as it does not require blood sampling, the Griffin score is less costly and could be used for self-assessment by internet or paper questionnaire. Moreover, the applicability of this score appeared quite universal, since it performed similarly well in a European and in a US population. Still, the number of participants classified as being at risk using the Griffin score was high. This may be problematic because monitoring, coaching and treating a potentially large number of individuals at risk could be expensive and logistically demanding. It would be preferable to have a score with similar sensitivity/specificity values but leading to a lower prevalence of individuals at risk. In all scores and both study population samples, the sensitivity was relatively low. One reason for this may be that these scores were not developed to predict death as outcome and/or that the cutoff point should be adapted. Low sensitivity is problematic because a substantial proportion of persons who will die in the future are 3

missed. Interestingly, and in agreement with the literature 2, scores including biological data were not better than scores based solely on clinical data. This finding suggests that the inclusion of biological variables might not improve the predictive capacity of T2DM risk scores regarding overall mortality. Further studies are needed to better assess whether biological variables could be replaced by more easily obtainable parameters, e.g. also in scores predicting a cardiovascular outcome. HbA 1c had a good predictive capacity and led to a relatively low prevalence of participants at risk. Hence, HbA 1c could be used as a risk factor for overall mortality. In fact, HbA 1c performed significantly better than the lipid parameters traditionally used in models predicting cardiovascular disease mortality 9. HbA 1 c can also be measured and interpreted in the non-fasting state 10, thus facilitating screening procedures. We conclude that T2DM risk scores might be useful to predict the risk of death as a proxy for overall health risk. In both populations, scores based solely on clinical data were as efficient as scores additionally including biological variables. This could open door for the development of simple, self-administrable screening questionnaires allowing to assess and approach lifestyle factors. Acknowledgment This work was supported by the Swiss National Science Foundation (grants 3347CO-108806, 33CS30-134273, 32473B-125710). The Colaus study was supported by grants from the Swiss National Science Foundation [grant no: 33CSCO-122661 and 3CSC0-139468]; GlaxoSmithKline and the Faculty of Biology and Medicine of anne. Conflict of Interest None declared Key Points Numerous scores exist helping to identify persons at risk for type two diabetes mellitus The capacity of these scores to predict all-cause mortality and the usefulness in clinical practice has not been examined previously There were large differences regarding the prevalence of persons defined at risk and concerning the predictive capacity of the scores Scores based on solely clinical data may perform better than scores relying on clinical plus biological data This could open door for the development of simple, self-administrable screening questionnaires allowing to assess and approach lifestyle factors. References 1. Schmid R, Vollenweider P, Bastardot F, Waeber G, Marques-Vidal P. Validation of 7 type 2 diabetes mellitus risk scores in a population-based cohort: Co study. Arch Intern Med 2012;172:188-9. 2. Noble D, Mathur R, Dent T, Meads C, Greenhalgh T. Risk models and scores for type 2 diabetes: systematic review. Bmj 2011;343:d7163. 3. Schmid R, Vollenweider P, Waeber G, Marques-Vidal P. Estimating the risk of developing type 2 diabetes: a comparison of several risk scores: the Cohorte annoise study. Diabetes Care 2011;34:1863-8. 4. Heidemann C, Boeing H, Pischon T, Nothlings U, Joost HG, Schulze MB. Association of a diabetes risk score with risk of myocardial infarction, stroke, specific types of cancer, and mortality: a prospective study in the European Prospective Investigation into Cancer and Nutrition (EPIC)-Potsdam cohort. Eur J Epidemiol 2009;24:281-8. 5. National Center for Health Statistics. Plan and operation of the Third National Health and Nutrition Examination Survey, 1988-94. Series 1: programs and collection procedures. Vital Health Stat 1 1994:1-407. 6. Firmann M, Mayor V, Vidal PM, et al. The Co study: a population-based study to investigate the epidemiology and genetic determinants of cardiovascular risk factors and metabolic syndrome. BMC Cardiovasc Disord 2008;8:6. 7. Schulze MB, Hoffmann K, Boeing H, et al. An accurate risk score based on anthropometric, dietary, and lifestyle factors to predict the development of type 2 diabetes. Diabetes Care 2007;30:510-5. 8. Pencina MJ, D'Agostino RB. Overall C as a measure of discrimination in survival analysis: model specific population value and confidence interval estimation. Stat Med 2004;23:2109-23. 9. Faeh D, Rohrmann S, Braun J. Better risk assessment with glycated hemoglobin instead of cholesterol in CVD risk prediction charts. Eur J Epidemiol 2013;28:551-5. 10. Selvin E, Steffes MW, Zhu H, et al. Glycated hemoglobin, diabetes, and cardiovascular risk in nondiabetic adults. N Engl J Med 2010;362:800-11. 4

Table. Characteristics of selected type 2 diabetes risk scores and single biological markers predicting all-cause mortality in two different surveys Assessment Continuous variable Binary variable Prevalence (%)* Missing (%)** Harrell's C AUC Sensitivity (%) Specificity (%) PPV (%) NPV (%) Predictor Type Items (n) Risk scores Wilson CB 6 4 3 11 0.605 0.505 63 51 5 5 97 97 41 5 70 97 Balkau C 4 8 4 6 0.635 0.516 67 52 11 6 94 97 43 6 72 97 Kahn 1 CB 10 28 25 17 0.637 0.573 67 57 41 33 78 80 44 5 76 98 Kahn 2 C 13 37 33 8 0.628 0.602 66 60 51 49 68 71 39 5 77 98 SDRS C 8 1 3 6 0.665 0.520 70 52 2 6 99 99 57 10 72 97 GDRS** C 10 12-6 0.718-76 - 23-92 - 54-75 - Findrisc C 7 9 17 6 0.708 0.551 75 56 17 25 95 87 58 6 74 97 Griffin C 7 23 18 0 0.763 0.592 82 59 46 33 87 86 62 7 78 98 Single biological markers HOMA-IR 2.5 B 1 35 23 10 0.561 0.538 57 54 42 33 69 76 37 4 73 97 FBG 6.1 mmol/l B 1 10 13 10 0.626 0.547 66 54 18 16 93 93 54 6 72 97 HbA 1 c 5.9%** B 1 10-4 0.679-70 - 22-90 - 38-81 - -: not assessed in the Co Study. *of persons at risk of type 2 diabetes; **only available for ; The scores were calculated based on a sample excluding participants with missing information for any of the scores, i.e. 17% AUC; area under the receiver operating curve; PPV, positive predictive value; NP, negative predictive value; Co, Cohorte annoise;, third National Health and Nutrition Examination Survey; C, clinical; CB; clinical + biological; B, biological; SDRS, Swiss Diabetes Risk Score from the Swiss Diabetes Association; GDRS, German Diabetes Risk Score from the EPIC-Potsdam cohort; HOMA-IR, Homeostasis Model Assessment of insulin resistance, defined as fasting insulin (mu/l) x fasting blood glucose (mmol/l) / 22,5; FBG, fasting blood glucose; HbA 1 c, glycated hemoglobin. 5

Supplementary File Table A1 Variables used for the generation of the diabetes risk scores Wilson Balkau (C) Kahn (C) SDRS GDRS Kahn (CB) FINDRISC Griffin Age Sex Genetics Familial history * Personal history Black race Lifestyle Smoking Alcohol Diet Physical activity Anthropometry BMI Weight Height Waist circumference Cardiovascular risk factors Resting pulse Hypertension HDL-Cholesterol Triglycerides Other Fasting glucose Prescribed Steroids Uric acid Number of variables 5 4 9 8 11 11 7 7 * women only men only C : clinical, CB : clinical and biological (only specified in case of various equations provided by the authors); SDRS, Swiss Diabetes Risk Score from the Swiss Diabetes Association; GDRS, German Diabetes Risk Score 6

Table A2 Populations used and characteristics of the diabetes risk scores Wilson Balkau (C) Kahn (C) SDRS GDRS Kahn (CB) FINDRISC Griffin Study Framingham DESIR ARIC FINRISK EPIC ARIC FINRISK Ely and Wessex Study Country USA France USA Finland Germany USA Finland England Number of patients 3,140 3,817 12,729 4,435 27,548 12,729 4,435 1,077 Patients age (years) * 54.0 (9.8) 30-65 45-64 25-64 35-65 45-64 25-64 40-64 Follow-up (years) 8 9 10 5 10 7 10 5-10 0 Threshold to define high risk 33% 30% 33% 15 points 500 points 46% 12 points 37% AROC 0.85 0.713-0.827 0.71 0.86 0.84 0.79 0.86 0.8 * mean (SD) or range cross sectional study men women AROC : Area Under the ROC Curve; C : clinical, CB : clinical and biological (only specified in case of various equations provided by the authors); SDRS, Swiss Diabetes Risk Score from the Swiss Diabetes Association; GDRS, German Diabetes Risk Score 7