Prediction models in nutritional epidemiology

Transcription

1 Science & Research Original Contribution Author's copy! Any use beyond the limits of copyright law without the consent of the publisher is prohibited and punishable. This applies in particular to duplications, translations, microfilming as well as storage and processing in electronic systems. Peer-reviewed Manuscript received: October 3, 212 Revision accepted: April 29, 212 Prediction models in nutritional epidemiology With the German diabetes risk score (GDRS) as an example Kristin Mühlenbruch, Nuthetal Summary Research in nutritional epidemiology includes the identification of risk factors for specific diseases but also makes it possible to predict individual disease risks based on prediction models. The objectives or outcomes of such prediction models are mainly lifestyle related diseases such as cardiovascular diseases or type 2 diabetes mellitus; these diseases are particularly suitable for the initiation of preventive interventions. In this review article, the aims and methodological aspects of developing such a prediction model will be described as well as the evaluation of a prediction model based on the criteria sensitivity, specificity or area under the receiver-operating-characteristic (ROC) curve. For a better understanding, the development of the German diabetes risk score (GDRS) will be demonstrated with regard to the aforementioned criteria. For the illustration of clinical application of the GDRS, the online tool as well as the simplified paper version will be presented. Keywords: nutritional epidemiology, risk prediction, prediction model, discrimination, calibration, German diabetes risk score Background The aim of studies in the field of nutritional epidemiology is the identification of risk factors for specific diseases. The European Prospective Investigation into Cancer and Nutrition (EPIC)-Potsdam study is a good example of such a study, with cancer and chronic diseases as the main objectives [1]. The findings of such Citation: Mühlenbruch K (213) Prediction models in nutritional epidemiology. With the German diabetes risk score (GDRS) as an example. Ernaehrungs Umschau international 6(8): This article is available online: DOI /eu studies can particularly be transferred to disease prevention. Type 2 diabetes is a good example due to its relation to lifestyle and its increasing rate of newly diagnosed patients (incidence) [2]. The most important risk factors can also be used as the basis for a preferably accurate risk prediction; with the help of such risk predictions, people at particularly high disease risk can be identified. This was the rationale for the development of risk prediction models which became popular mainly in the area of cardiovascular diseases; examples are the Framingham risk score or the PROCAM score [3, 4]. For type 2 diabetes, more than 2 single prediction models were published since 1999 [5, 6]. One example of such a diabetes prediction model is the German diabetes risk score (GDRS) [7]. With the use of (statistical) regression analysis, prediction models allow the calculation of individual risks based on a specific risk profile. In general, these risks are related to a time horizon of 5, 8 or 1 years in the future. In the following basic methodological requirements and methods will be discussed which are meaningful for the development or derivation of a prediction model as well as for their evaluation. For illustrative purposes, the derivation of the GDRS will be described in more detail and how this prediction model can be evaluated. Methodological basics for derivation and evaluation Study population and study design For the derivation or development of a prediction model, at first, it is important to know in which target population it will be applied. To determine the target population key aspects are the age of onset of the disease of interest and already known high risk groups. For a chronic disease such as type 2 diabetes which occurs in adult age and in both men and women, a universally valid prediction model for a wide range of people in the adult population would be desirable. Therefore, also the development of a score for the application in such a 132 Ernaehrungs Umschau international 8/213

2 population should be based on a study population from the general adult population. For the selection of a study population, registries of residents are often used for a random selection of a specific number of people for the inclusion into the study. Not only the definition of the study population but also the study design is important for the development of a score, since it is of interest what kind of people develop the disease after a specific period of time and what kind of people do not. That means that initially healthy people will be followed-up for a certain timespan. During follow-up assessments, the disease status is repeatedly determined. The described study design is the so called prospective cohort study. Key aspects of this study design are the baseline assessment and the orientation towards the future. The baseline assessment is performed during the recruitment of study participants and includes comprehensive examinations and interviews regarding potential risk factors, while follow-up measurements mainly serve to identify newly diagnosed participants (incident cases) [8] ( Figure 1). Development of a risk prediction model For the derivation of a risk prediction model most notably already identified risk factors for the disease in previous studies are included, but also potential risk factors derived study population Direction of the observation start of the study/baseline assessment Cohort study Fig. 1: Design of a prospective cohort study [8] With the logistic regression model, it is possible to calculate the likelihood that a specific disease occurs. Most frequently, cross-sectional studies and especially case-control studies are analyzed with the logistic regression; but also prospective study data can be analyzed with this model. In this case, only a specific time period can be used in which the disease of interest occurred or another outcome appeared, but not the time each person contributes in the study. To do that, a regression model can be applied which was particularly developed for the analysis of survival times. For this approach, the time until development of a specific disease is of special interest. The Cox-regression model which is based on the proportional hazards (PH) model developed by COX (1972) [9] is the model of choice for such cases. The assumption of this model is that the hazard or risk of exposed and non-exposed persons develops proportionally over a specific period of time. In this context, exposed means that a risk factor such as smoking is present, and non-exposed that this risk factor is not present. from the underlying study data. The potentially relevant risk factors are then included in a regression model (logistic model or Cox model; Box 1) as independent variables and parameter estimates (regression coefficients) are calculated which represent the strength of the association between the risk factor and the disease. Most commonly, risk factors are removed from the regression model if the relation to the disease risk cannot be statistically secured (statistical significance based on the p-value for the parameter estimate). For clinical practice, parameter estimates are often transformed into simpler score points; these are assigned to each risk factor. Finally, the summation of these score points gives the total score points (also called risk score); the total score points can be used for exposed persons non-exposed persons Box 1: Logistic regression model and Cox-regression model diseased persons not diseased persons diseased persons not diseased persons time calculating the disease risk. In general, a score with m risk factors can be calculated as follows: Score = points 1 risk factor 1 + points 2 risk factor points m risk factor m (1) Evaluation of the prediction models accuracy Although prediction models most commonly include statistically significant risk factors, this does not automatically mean that they lead to a good prediction of individual disease risks. To evaluate such accuracy in prediction, two basic questions are of importance: 1. Does the prediction model enable to distinguish between people who develop the disease and those who do not develop the disease? This question refers to the ability to separate two groups of people from each other based on specific information; this is also called discrimination. 2. Does the predicted disease risk correspond with reality? Prediction models can hardly predict the fate of a single person accurately; however, at least the predicted disease risks for a group of similar persons should agree with the actual disease risk for this group of Ernaehrungs Umschau international 8/

3 Science & Research Original Contribution Criterion sensitivity (Se) specificity (Sp) false positive rate (FPR; 1 specificity) false negative rate (FNR; 1 sensitivity) positive predictive value (PPV) negative predictive value (NPV) Meaning Tab. 1: Overview of the criteria for diagnostic tests and their meaning persons. The agreement between predicted and observed disease risks is also called calibration. Probability that the test result is positive, if the person is diseased; true positive Probability that the test result is negative, if the person is not diseased; true negative Probability that the test result is positive, if the person is not diseased Probability that the test result is negative, if theperson is diseased Probability that a person is diseased, if the test result is positive Probability that a person is not diseased, if the test result is negative Key characteristic for the determination of the discriminatory ability are sensitivity, specificity, predictive values and the receiver operating characteristic (ROC) curve ( Table 1). The important characteristics for the calibration are as aforementioned the predicted and observed risks. Details of these characteristics are described in Box 2. Validation of prediction models In addition to the criteria for discrimination and calibration of a prediction model, it is important to know how generalizable this model is. To prove this transportability to other populations or the general applicability, validation studies need to be performed. The basic principle of a validation is the application and calculation of the score in another Box 2: Determination of discrimination and calibration Discrimination With regard to the theory of diagnostic tests, the first step for the evaluation is the choice of thresholds (score points). These thresholds or cut-offs distinguish between the positive and negative test result. This is illustrated in the following table. Threshold (Cut-off) Quantitative measure <Threshold (Cut-off) Test + Test The sensitivity (Se) can be calculated as the fraction of diseased persons who had a positive test result (true positive) and the specificity (Sp) as the fraction of persons not diseased with a negative test result (true negative). These two measures represent the validity criteria for diagnostic tests. A good diagnostic test for example would have a sensitivity of.9 (9 %) and specificity of.75 (75 %). In accordance with these criteria also the fraction of false positive (FPR) or false negative (FNR) test results can be calculated. These four criteria can be displayed in a fourfold table [1]. Based on such a fourfold table, sensitivity and specificity as well as FPR and FNR can be calculated as follows: Se = Sp = FPR = A A + C A B + D C A + C = 1 Sp True disease status (D) Test result (T) positive (D+) negative (D ) positive (T+) A B A + B negatieve (T ) C D C + D (2) (3) (4) B FNR = = 1 Se (5) B + D For the receiver operating characteristic (ROC) curve sensitivity and specificity are calculated for all possible cut-off values. The graphical illustration is based on plotting sensitivity against 1 specificity ( Figure 2). The validity criterion for discrimination is the area under the ROC curve (ROC-AUC). For the diagonal line the value of this area is.5 and for the A + C B + D A + B + C + D = N whole area (orange line) 1.. The ROC-AUC represents the probability that given a randomly selected pair (diseased and healthy), the diseased person would get a higher test result (quantitative measure) compared to the healthy person. A ROC-AUC of.5 means that with a probability of 5 % a diseased person has a higher test result, and is therefore not better than tossing a coin (uninformative model). In contrast, a ROC-AUC of 1. means that with a probability of 1 % diseased persons would have a higher test result than the healthy persons (perfect model) [11]. ROC-AUCs >.7 represent a good, >.8 a very good and >.9 an excellent model. Besides sensitivity and specificity, most notably the prognostic values or predictive values are important for clinical practice. Among these, the 134 Ernaehrungs Umschau international 8/213

4 population than the population used for the derivation of the score. This includes the calculation of criteria for evaluating the performance or accuracy of the score. For internal validation, the study population is divided into a derivation sample which is used for the development of the score, and a validation sample which is used for validating the score (so called split-sample validation). The problem with this and also other internal validation methods is the reliance on the same population which was only randomly split into two samples. To prove the transportability to another population than the study population, external validation studies are more appropriate. These are usually performed in study samples from independent studies and therefore also include regional or ethnic differences. Requirements for external validations as well as already for the development of a prediction model are: large population based studies prospective cohort studies participants at baseline Given these requirements, the main problem often is the differing assessment of the risk factors in different studies. After calculation of the score in the external cohort, the discriminatory ability and the calibration of the score need to be investigated again. This gives an impression of the applicability of this score in an external population, but often only ROC-AUC or other measures of discrimination are calculated and not calibration at least in the field of diabetes prediction models [5]. However, studies showed that especially the calibration of a prediction model could be insufficient although the discriminatory ability was indeed acceptable [14]. Due to this fact, it can be concluded that prediction models should not be adopted without caution/uncritically; the prediction accuracy should rather be tested before implementation into clinical practice. In the end, adaptations of the model based on the observed prediction accuracy are indeed possible (recalibration). positive predictive value (PPV) and the negative predictive value (NPV) can be distinguished; with predictive values, it is possible to determine the probability that the test result of a diagnostic test represents the true disease status. Not only sensitivity and specificity are included in the calculation but also the frequency of a disease (prevalence or incidence) P(D+) as well as the converse probability P(D-) that the disease does not appear [12]. P(D + ) Se PPV = P(D + ) Se + P(D ) FPR P(D ) Sp NPV = (7) P(D ) Sp + P(D + ) FNR These values can also directly be estimated from the fourfold table if P(D+) is approximately. A + C This N would lead to the following relations: PPV = A A + B (6) (8) D NPV = (9) C + D Sensitivity 1,,9,8,7,6,5,4,3,2,1 ROC-AUC,,,1,2,3,4,5,6,7,8,9 1, 1 Specificity Fig. 2: Illustration of a receiver operating characteristic curve and the area under the curve (ROC- AUC). Additional explanations in the text. Calibration Calibration gives an impression of how well the predicted risks agree with the observed risks. A specific test is the Hosmer-Lemeshow (HL) test which is based on the Chi-square distribution [13]. For applying this test, the study population is usually divided into ten equally sized groups (deciles of risk); study participants were ordered by their calculated risk score (or predicted disease risk) before. Using the mean predicted risk of each group, it is possible to determine the expected number of cases in the respective group. The test compares the predicted or expected number of cases with the observed number of cases in each group. A statistically significant deviation would result in a small p- value (<.5) and indicates a poor calibration. To perform this test, it is often not recommended to use deciles of risk but risk categories which are relevant for the implementation of the prediction model into clinical practice. Ernaehrungs Umschau international 8/

5 Science & Research Original Contribution GDRS development as an example The German diabetes risk score (GDRS) was developed based on data from the EPIC-Potsdam study [7, 15]. The EPIC-Potsdam study is a prospective cohort study with participants mainly aged years at baseline assessment. Due to the selection of study participants from the general adult population in Potsdam and surroundings [1], this study is well-suited for the development of a diabetes prediction model. In addition, the periodic follow-up assessments each 2 to 3 years enable a detailed impression of the disease status for the participants in the course of time [16]. For the selection of the set of risk factors, already known diabetes risk factors were chosen, inter alia from results of previous studies dealing with type 2 diabetes risk, but also general socio-economic risk factors Risik factor β HR (95 % CI) Points Waist circumference (cm) ( ) 7.4 Height (cm) ( ) 2.4 Age (years) ( ) 4.3 Hypertension (yes/no) ( ) 46 Intake of red meat (each 15 g/day) ( ) 49 Intake of whole-grain bread (each 5 g/day) ( ) 9 Consumption of coffee (each 15 g/day) ( ) 4 Moderate alcohol consumption ( ) 2 (1-4 g/day) Sports, biking or gardening ( ) 2 (h/week) Former smoker ( ) 24 Current smoker (2 cig./day) ( ) %-CI = 95 %-confidence interval; β = parameter estimate; GDRS = German diabetes risk score; HR = Hazard Ratio Tab. 2: Overview of the risk factors in the GDRS with corresponding parameter estimates, hazard ratios and points in the EPIC-Potsdam study Obser- Mean Expected Predicted ved predic- number risk risk ted of Points (%) N* n** (%) risk cases < 41 < > * Number of study participants which are classified into this category ** Number of incident diabetes which are classified into this category EPIC = European Prospective Investigation into Cancer and Nutrition Tab. 3: Observed and expected risks for developing diabetes during a follow-up time of 5 years in the EPIC-Potsdam study were included. Due to the prospective study design, a Cox-regression was performed. The mean follow-up time for the participants was 7 years. With the risk score a 5-year risk based on the Cox-regression should be calculated. Table 2 shows the selected risk factors for the GDRS as well as the corresponding parameter estimates, hazard ratios (HR) with 95 % confidence intervals, and allocated score points. The strength of the association for the 11 included risk factors with disease risk is expressed with the value of β. Here, a positive algebraic sign indicates a positive relation; that means for example that with each cm increase in waist circumference, the diabetes risk increases by.74, if all other risk factors are held fix. In contrast, a negative algebraic sign indicates an inverse relation to disease risk; that means for the GDRS that with each cm increase in body height, the diabetes risk decreases by.24, if all other risk factors are held fixed. These parameter estimates can be converted into relative risks in this case into hazard ratios (HR). A HR > 1 indicates increased risk while HR < 1 indicates a reduced risk; the reference is assumed with a HR of 1. The larger the distance of the parameter estimate to or of the HR to 1, the stronger is the association with the disease risk. For the allocation of (score) points to the risk factors the parameter estimates were multiplied with 1 and the calculation of the total score points can be performed with equation 1: GDRS = +7.4 waist circumference (cm) 2.4 height (cm) +4.3 age (years) +46 hypertension +49 red meat (15 g/day) 9 whole grain bread (5 g/day) 4 coffee (15 g/day) 2 moderate alcohol consumption (1 4 g/day) 136 Ernaehrungs Umschau international 8/213

6 2 physical activity (h/week) +24 former smoker +64 current smoker (2 units/day) (1) Based on the individual score value (GDRS) the exact individual disease risk can be calculated with equation 11; this equation was also derived from Cox-regression and together with the score points it includes the baseline survival function ( ): P (diabetes) = exp (Scorepoints/1) (11) The result of this equation is the absolute value of the risk for developing diabetes within the next 5 years, and for better understanding it is expressed in percent. For the evaluation of the accuracy/performance of the GDRS the described criteria will be calculated exemplary. First of all, Table 3 shows a classification by score points of the GDRS. The points can directly be converted into a predicted risk. In this table only cases which occurred during a follow-up time of 5 years are presented by risk group (n). The exemplary calculation is based on the choice of the threshold or cut-off of 41 score points from GDRS to classify as either a positive or a negative test result. With that the following classification of the study population emerges: GDRS points 41 points < 41 points test + test The classification in Table 3 leads to the following fourfold table: Test- Incident total result diabetes (D) positive negative (T) (D+) (D ) positive (T+) negative (T ) total Based on this fourfold table sensitivity (Se), specificity (Sp), false positive rate (FPR=1-specificity), false negative rate (FNR) as well as PPV (positive predictive value) and NPV (negative predictive value) can be determined for our example. Hence, the sensitivity of the GDRS for a cut-off of 41 points is 571/589 =.969 (96.9 %), and the specificity is 1653/24578 =.433 (43.3 %). The PPV for this example is 571/1,671 =.39 (3.9 %) and the NPV is 1653/1671 =.998 (99.8 %). That means that with a probability of 3.9 % a person with a positive test result really has the disease, and with a probability of 99.8 % a person with a negative test result is really free of the disease. Figure 3 shows sensitivity and false positive rate (= 1 - specificity) of the GDRS for three selected cut-off values of the predicted risk. The predicted risk can be calculated from the score points of the GDRS and can attain values from to 1. The value of the predicted risk represents the probability of developing diabetes within the next 5 years; for example the value.45 represents a probability of 45 %. The ROC curve for the GDRS ( Figure 4) displays sensitivity and false positive rate for all possible cut-off values of the predicted risk. With regard to calibration, Table 3 directly gives an impression thereof, when comparing the observed risk of a group with the range of the predicted risk for the same group. For the GDRS the observed risk, which can be determined 1, 1,,9 Cut-off of predicted risk,2,9,8 Cut-off of predicted risk,3,8,7,7 Sensitivity,6,5,4 Cut-off of predicted risk,45 Sensitivity,6,5,4,3,3 Cut-off Se 1-Sp Sp, , , 1 1,,1,2,3,4,5,6,7,8,9 1, 1 Specificity,2,1,,,1,2,3,4,5,6,7,8,9 1, 1 Specificity Fig. 3: Sensitivity and 1-Specificity of the GDRS for 3 selected cut-off values GDRS = German diabetes risk score Fig. 4: ROC curve of the GDRS for all possible cut-off values GDRS = German diabetes risk score; ROC = receiver operating characteristic Ernaehrungs Umschau international 8/

7 Science & Research Original Contribution Observed risk,3,25,2,15,1.5,5,1,15,2,25,3 Predicted risk Fig. 5: Calibration Plot for the observed and predicted risks of the GDRS based on five risk groups GDRS = German diabetes risk score a p-value of.16. For determining the transportability of the GDRS to populations other than EPIC-Potsdam the GDRS was validated in an independent cohort study, the EPIC-Heidelberg study, and two cross-sectional studies (TÜF [Tübingen Family study] and MeSyBePo [Metabolic Syndrome Berlin Potsdam]). The ROC-AUC in EPIC-Heidelberg indicated a good discrimination (.82). For the identification of undiagnosed diabetes cases in a cross-sectional study design the GDRS also showed a good discriminatory ability in TÜF (.83) and MeSyBePo (.759) [7]. The results of a Dutch study further indicated that the GDRS is an appropriate model for the prediction of diabetes [14]. Fig. 6: Online-tool of the GDRS GDRS = German diabetes risk score Patient questionnaire By conducting the German Diabetes Risk Score (DRS) you can determine your individual risk to develop type 2 diabetes within the next 5 years. Age How old are you (years)? Ô < 35 points Ô points Ô point Ô points Ô points Ô points Ô points Ô points Physical activity Are you physically active at least 5 hours a week? (e.g., sport, gardening, cycling) Fig. 7: Extraction of the paper questionnaire of the GDRS (self-assessment tool) GDRS = German diabetes risk score by the number of cases in relation to the overall group size for the respective group (n/n x 1), lies within the range of the predicted risks for this group. The following calibration plot ( Figure 5) further illustrates Smoking What is your smoking status? Ô I never smoked GERMAN DIABETES RISK SCORE points Ô I used to smoke on average less than 2 cigarettes a day points Ô I used to smoke on average 2 or more cigarettes a day 3 points Ô I smoke on average less than 2 cigarettes a day points Ô I smoke on average 2 or more cigarettes a day 6 points the agreement of observed and predicted risks for the five risk groups used for the GDRS. For a prediction model with perfect calibration all points would be located on the diagonal line. The HL-test for the GDRS results in Online-tool and paper questionnaire of the GDRS Online-tool Using the online-tool of the GDRS ( Figure 6), it is possible to calculate the exact risk for developing diabetes within the next five years based on exact information regarding age of a person, body height, waist circumference and, most notably, diet using pictures for the portion size and data on frequencies of consumption. The result of the calculation is displayed in percent and colors (flash light) indicating whether it is rather a high or a low risk. In addition, exemplary changes in e.g. smoking, diet or physical activity are used to re-calculate the risk and thereby showing the direct change in absolute risk, or rather how much the risk could be reduced by such changes. Paper questionnaire In addition to the online-tool, a simplified paper questionnaire was developed for the GDRS ( Figure 7) [17]. It enables people without access to a computer or the internet to determine their individual risk. In contrast to the online version the simplified paper version is based on response categories. That means that e.g. age, waist circumference or dietary or smoking behavior is summarized in respective groups or categories; this only needs to be marked with a cross. For each response category points are assigned which in the end have to be summed to calculate the individual risk. In the corresponding answer sheet these points are summarized into five 138 Ernaehrungs Umschau international 8/213

8 categories which represent low, still low, increased, high or very high risk. The questionnaire is a self-assessment tool and is therefore self-explaining; that means that no additional person is needed to help with the completion. Compared to the online version which is based on the exact prediction model, the calculation of risks with the paper version is slightly less accurate due to the categorization of risk factors; yet discrimination of the paper version is comparably good with a ROC-AUC of.83 (ROC GDRS :.84) [17]. Risk communication For the implementation of a validated prediction model such as the GDRS not only the development of an online-tool or paper questionnaire is important, but also the presentation of the result needs special attention. A common approach is to define risk groups which are classified by the score points for an easier communication (that the risk is low, increased or high). This was also applied with the paper version of the GDRS. Often, the classification into such risk groups is based on evidence from clinical studies and is therefore related to specific the - rapeutic interventions [18]. Another approach is the presentation of individual risks. Here, the main goal is to report the personal risk directly to the applicant and as understandable as possible. This shall serve the purpose to motivate persons with a high risk to lower their risk level. But the question at stake is whether a relatively low risk of 3 % or 3 out of 1 really motivates towards lifestyle or behavioral changes. That is also the reason why alternatives (comparison with a mean person of the same age) to the absolute risk are used. Outlook Prediction models are appropriate instruments for a non-invasive identification of high-risk individuals; in the setting of general practitioners, this is also possible in connection with blood parameters. Based on the risk factors included in the calculation of the score, individualized recommendations can be formulated which could result in a risk reduction. Nevertheless, the goal is rather to prevent the onset of the disease or to delay it in the long term by initiating strategies in the context of preventive interventions in connection with general practitioners. To implement this procedure, it is required to perform cost-effectiveness evaluations and an analysis of the number of cases which could be prevented by which kind of interventions. This will be the basis of future studies. Kristin Mühlenbruch Deutsches Institut für Ernährungsforschung (DIfE) Potsdam-Rehbrücke Abteilung Molekulare Epidemiologie Arthur-Scheunert-Allee , Nuthetal Kristin.Muehlenbruch@dife.de Conflict of Interest The authors declare no conflict of interest according to the guidelines of the International Committee of Medical Journal Editors. References 1. Boeing H, Korfmann A, Bergmann MM (1999) Recruitment procedures of EPIC-Germany. European Investigation into Cancer and Nutrition. Ann Nutr Metab 43(4): Kurth BM (212) First results from the Study on Adult Health in Germany (DEGS). Bundesgesundheitsblatt 55(8): Assmann G, Cullen P, Schulte H (22) Simple scoring scheme for calculating the risk of acute coronary events based on the 1-year followup of the prospective cardiovascular Munster (PROCAM) study. Circulation 15(3): D Agostino RB Sr., Vasan RS, Pencina MJ et al. (28) General cardiovascular risk profile for use in primary care: the Framingham Heart Study. Circulation 117(6): Buijsse B, Simmons RK, Griffin SJ et al. (211) Risk assessment tools for identifying individuals at risk of developing type 2 diabetes. Epidemiol Rev 33(1): Diabetes Risk Scores Developed and Subsequently Tested in Populations Primarily Sampled From the General Population. German Institute of Human Nutrition. Online Database. URL: drpm-database/ Zugriff: Schulze MB, Hoffmann K, Boeing H et al. (27) An accurate risk score based on anthropometric, dietary, and lifestyle factors to predict the development of type 2 diabetes. Diabetes Care 3(3): Schulze MB (27) Methoden der epidemiologischen Ernährungsforschung. Ernährungs Umschau 54(5): Rothman KJ, Greenland S, Lash TL. Modern Epidemiology. 3rd ed. Lippincott Williams & Wilkins, Philadelphia (28) 1. Kreienbrock L, Schach S. Epidemiologische Methoden. 4 ed. Elsevier GmbH, München (25) 11. Steyerberg EW. Clinical Prediction Models. A Practical Approach to Development, Validation, and Updating. Springer Science+Business Media, New York (29) 12. Weiß C. Basiswissen Medizinische Statistik. 4 ed. Springer Medizin Verlag, Heidelberg (28) 13. Hosmer DWJ LS. Applied Logistic Regression. 2nd ed. John Wiley & Sons, New York (2) 14. Abbasi A, Peelen LM, Corpeleijn E et al. (212) Prediction models for risk of developing type 2 diabetes: systematic literature search and independent external validation study. BMJ 345: e Schulze MB (27) The German Diabetes Risk Score (DRS). Ernährungs Umschau 54(3): Bergmann MM, Bussas U, Boeing H (1999) Follow-up procedures in EPIC-Germany data quality aspects. European Prospective Investigation into Cancer and Nutrition. Ann Nutr Metab 43(4): Schulze MB, Holmberg C, Hoffmann K et al. (27) Entwicklung eines Kurzfragebogens zur Bestimmung des Diabetesrisikos auf Grundlage des Deutschen Diabetes-Risiko-Scores. Ernäh - rungs Umschau 54(12): Grundy SM, Cleeman JI, Merz CN et al. (24) Implications of recent clinical trials for the National Cholesterol Education Program Adult Treatment Panel III guidelines. Circulation 11(2): DOI: /eu Ernaehrungs Umschau international 8/