CARDIOVASCULAR RISK ASSESSMENT ADDITION OF CHRONIC KIDNEY DISEASE AND RACE TO THE FRAMINGHAM EQUATION by PAUL E. DRAWZ, MD, MHS Submitted in partial fulfillment of the requirements for the degree of Master of Science Clinical Research Scholars Program CASE WESTERN RESERVE UNIVERSITY August, 2009 1
CASE WESTERN RESERVE UNIVERSITY SCHOOL OF GRADUATE STUDIES We hereby approve the thesis/dissertation of candidate for the degree *. (signed) (chair of the committee) (date) *We also certify that written approval has been obtained for any proprietary material contained therein.
Table of Contents List of Tables... 3 List of Figures... 5 Acknowledgements... 6 Abstract... 7 Specific Aims and Hypotheses... 8 Background and Significance... 9 Chronic kidney disease... 9 Risk stratification The Framingham Equation... 9 Race and cardiovascular disease... 11 Chronic kidney disease and cardiovascular risk... 11 Methods... 14 Overview... 14 Model development ALLHAT... 14 Validation samples... 18 Statistical analyses... 20 Results... 24 Baseline characteristics... 24 Model development... 28 Model validation ARIC... 31 Model validation CHS... 35 Discussion... 38 Model performance... 39 Relation to prior studies... 41 Future directions... 42 Conclusions... 45 Appendix... 47 Bibliography... 49 2
List of Tables Table 1. Baseline characteristics of the ALLHAT developmental sample according to JNC-V hypertension categories... 25 Table 2. Baseline characteristics of the ALLHAT developmental cohort... 26 Table 3. Baseline characteristics of the ARIC cohort according to JNC VII blood pressure categories... 27 Table 4. Baseline characteristics of the CHS cohort according to JNC VII blood pressure categories... 27 Table 5. Baseline characteristics of the ARIC cohort... 29 Table 6. Baseline characteristics of the CHS cohort... 29 Table 7. β-coefficients for ALLHAT models without (Model 1) and with CKD and race (Model 2) and the Framingham equation... 30 Table 8. C-statistic for Models 1 and 2 (includes race and CKD) and Framingham by race and gender in ARIC... 32 Table 9. Nagelkerke R 2 for Models 1 and 2 (includes race and CKD) and Framingham by race and gender in ARIC... 32 Table 10. CHD risk classification comparing models with (Model 2) and without race and CKD (Model 1) in all men in ARIC... 33 Table 11. Net reclassification improvement (NRI) from Model 1 to Model 2 (includes race and CKD) by race and gender in ARIC... 33 Table 12. Hosmer-Lemeshow goodness-of-fit test for Models 1 and 2 (includes race and CKD) and Framingham by race and gender in ARIC... 34 Table 13. Reclassification calibration statistic for Model 1 and Model 2 (includes race and CKD) by race and gender in ARIC... 35 Table 14. C-statistic for Models 1 and 2 (includes race and CKD) and Framingham by race and gender in CHS... 36 Table 15. Nagelkerke R 2 for Models 1 and 2 (includes race and CKD) and Framingham by race and gender in CHS... 36 Table 16. Net reclassification improvement (NRI) from Model 1 to Model 2 by race and gender in CHS... 37 3
Table 17. Hosmer-Lemeshow goodness-of-fit test for Models 1 and 2 (includes race and CKD) and Framingham by race and gender in CHS... 38 Table 18. Reclassification calibration statistic for Model 1 and Model 2 (includes race and CKD) by race and gender in CHS... 38 Table 19. Age adjusted CHD rates by cohort... 40 4
List of Figures Figure 1. CONSORT Diagram for ALLHAT samples... 25 Figure 2. CONSORT Diagram for ARIC and CHS validation cohorts... 26 5
Acknowledgements I would like to thank the following individuals for their guidance and mentorship with this project and my career development: Dr Mahboob Rahman, for his mentorship, facilitation of research projects without which I would never have been able to undertake, and for helping me develop an approach to research inquiry, study design, analysis, and presentation. Drs R Tyler Miller and John Sedor, for their commitment to my career development. Dr Denise Babineau, for her statistical advice and continuing support of my research. Dr Barbara Cromer, for her research and grant writing advice and being my CRSP advisor. Finally, immeasurable thanks to my wife, Sarah, for being a great life partner, for constantly listening to my thoughts on clinical research projects, and for helping to raise two incredibly fun, energetic, and smart kids in Maggie and Jack. 6
Cardiovascular Risk Assessment Addition of Chronic Kidney Disease and Race to the Framingham Equation Abstract by PAUL E. DRAWZ, MD, MHS Patients with chronic kidney disease (CKD) are at high risk for coronary heart disease (CHD). The performance of the Framingham equation, used to assess CHD risk, is consistently worse in minority subjects and those with CKD. The purpose of this study was to evaluate the addition of race and CKD to the traditional risk factors in the Framingham equation. Two CHD prediction models were developed in subjects from ALLHAT: the first included only traditional Framingham variables, the second added CKD and stratified by race. The performance of these models was evaluated in subjects from the Atherosclerosis Risk in Communities Study and the Cardiovascular Health Study. In both cohorts, the new models performance was poor and was significantly worse than the traditional Framingham equation. In conclusion, the current results do not support the addition of race and CKD to a model including traditional Framingham risk factors. 7
Specific Aims 1. Develop a model to better predict CHD by including CKD and race with traditional risk factors from the Framingham equation. 2. Validate and compare the new model to a model including only the traditional risk factors in: a) patients from ARIC and CHS and b) an ALLHAT validation cohort. Secondary Aims 1. Validate and compare the new model including CKD and race to a model including only the traditional risk factors in black patients from: a) ARIC and CHS and b) the ALLHAT validation cohort. 2. Validate and compare the new model including CKD and race to a model including only the traditional risk factors in patients with CKD from: a) ARIC and CHS and b) the ALLHAT validation cohort. Primary Hypothesis 1. Incorporation of race and CKD into a prediction model that includes the traditional risk factors from the Framingham equation will improve model performance and discrimination and will result in improved risk classification. Secondary Hypothesis 1. Incorporation of race and CKD into a prediction model that includes the traditional risk factors from the Framingham equation will improve model performance and discrimination and will result in improved risk classification, especially in blacks and patients with CKD. 8
BACKGROUND AND SIGNIFICANCE Chronic kidney disease Chronic kidney disease (CKD) affects more than 25 million Americans (1). The burden of morbidity and mortality from CKD derives largely from the disproportionate risk of coronary heart disease (CHD). Patients with CKD are at very high risk for cardiovascular disease and CKD has been shown to be an independent risk factor for CHD (2-10). Even a glomerular filtration rate (GFR) in the low-normal range has been shown to be associated with increased risk for CHD (2). Risk stratification The Framingham Equation The mainstay of CHD treatment is prevention, which includes identification and treatment of risk factors. Treatment of specific risk factors, in particular cholesterol, is based on patients overall risk for CHD (11). The principal instrument used by clinicians for risk stratification is the Framingham equation which was developed from a population-based cohort study of 5345 subjects 30 to 74 years old (12). Known risk factors including age, cholesterol, blood pressure, cigarette smoking, and diabetes were included in a Cox proportional hazards model and score sheets were developed from the β-coefficients. Individual patient risk for CHD can be predicted using these score sheets and then used to guide therapy and counsel patients. The National Cholesterol Education Program Adult Treatment Panel (ATP III) guidelines recommend LDL goals and LDL levels for initiation of dietary modification and pharmacologic interventions based on past medical history and Framingham risk scores 9
(11). Patients with a history of CHD, atherosclerotic disease, and diabetes are considered to be at high risk and are treated more aggressively (11). Diabetes is considered a CHD risk equivalent because, in a European cohort, the hazard ratio for fatal CHD was not significantly different from 1.0 for subjects with diabetes and no history of CHD compared to subjects without diabetes and a history of CHD (13). For patients without CHD, atherosclerotic disease, and diabetes, ATP III guidelines recommend treatment based on calculated Framingham risk. For instance, the recommendations for a patient with no risk factors for CHD would be a goal LDL of < 160 mg/dl, initiation of lifestyle modifications at LDL levels 160 mg/dl and consideration for pharmacologic intervention at LDL levels 190 mg/dl. The corresponding LDL levels for a patient with a 10-year risk for CHD between 10 and 20% are < 130 mg/dl, 130 mg/dl, and 130 mg/dl, respectively (11). While the Framingham equation has been shown to have good predictive ability, it has recognized weaknesses. It does not account for race or CKD. The equation was developed in a predominantly white population in Massachusetts prior to the widespread recognition of CKD as a risk factor for CHD. While the model has been validated in multiple diverse populations and has performed well, its ability to discriminate between those who did and did not develop CHD has been consistently worse in minority subjects (14, 15). This discrepancy may be related to different distributions of risk factors in whites and blacks and the fact that the relative impact of risk factors on CHD risk differs between whites and blacks. 10
Race and cardiovascular risk Multiple studies have evaluated the interaction between race and various risk factors for CHD. In the Atherosclerosis Risk in Communities (ARIC) study, hypertension was found to be a more powerful predictor of CHD in black women, while diabetes was found to be a greater risk factor in white compared to black women (14). Pooled data from the Charleston Heart Study and the Evans County, Georgia, Heart Study revealed that cigarette smoking may be a greater risk factor for black men compared to white men (16). However, the opposite result was found in analysis of pooled NHANES I and II data (17). Finally, total cholesterol has been shown to be a risk factor in white but not in black men (16). Given these data, it is likely that appropriate inclusion of race in the Framingham equation will improve the model s predictive ability, especially in blacks because they were underrepresented in the original model development. Chronic kidney disease and cardiovascular risk Another weakness of the Framingham model is that it was developed without consideration for kidney function and has been shown to have poor predictive ability in patients with CKD (12, 18). One reason for this was that CKD was not as wellrecognized a risk factor for CHD when the model was developed. Another is the low prevalence of CKD in the Framingham population there were only 416 patients with a GFR less than 60 ml/min per 1.73m 2 in the developmental cohort (19). Multiple prior studies have shown CKD to be a significant risk factor for CHD. Data from NHANES II showed that proteinuria and a GFR less than 70 ml/min per 1.73m 2 11
were significant risk factors for cardiovascular disease and death and that the risk for CHD was greater in blacks than whites (3). In ARIC, a reduced GFR was an independent risk factor for atherosclerotic cardiovascular disease (9). In a pooled cohort from ARIC, Framingham, Framingham Offspring, and the Cardiovascular Health Study (CHS), CKD was found to be an independent risk factor for cardiovascular disease and all-cause mortality (19). While CKD is a well established independent risk factor for CHD, whether it is a risk equivalent similar to diabetes is unresolved. In CHS, CKD was found to be a CHD risk equivalent (20). However, in previous studies in ARIC and Framingham, CKD was not shown to be a CHD risk equivalent (21, 22). Given that CKD is known to be a risk factor for CHD but may or may not be a risk equivalent, a recent study attempted to evaluate the increased predictive ability attained by addition of CKD to the Framingham model. In a population derived from ARIC and CHS, addition of CKD to the Framingham equation did not increase overall predictive ability (23). A limitation of this study was that it only compared the addition of CKD to the Framingham score and the addition of CKD to a fitted Cox regression model. The comparison of interest would have been between the fitted Cox model including CKD and the original Framingham model without CKD. Another limitation was the small number of subjects with CKD; they stratified by race and gender and had only 242 white men, 36 African American men, 401 white women, and 77 African American women with CKD. In addition, analysis was limited to improvement in discrimination, that is, a comparison of the area under the receiver operating characteristic (ROC) curves, or C- statistics (24, 25). 12
Evaluation of change in C-statistics is insensitive to small changes in model improvement for risk stratification models that already have moderate to high levels of discrimination, even with the addition of highly significant biomarkers (24, 25). Other methods for assessing improvement in risk stratification have been developed and include net reclassification improvement (NRI), a measure of improved classification of subjects into clinically meaningful categories (24). An appropriately powered and conducted analysis of the addition of both CKD and race to risk stratification models that include the traditional Framingham risk factors may increase our ability to discriminate between those who do and do not develop CHD. A new predictive model incorporating race and CKD should provide more accurate estimates for individual patient s CHD risk, especially in blacks and patients with CKD. We proposed to develop a predictive instrument for CHD in patients that incorporated race and CKD. We hypothesized that the addition of GFR and race would alter the relative weights for the other factors in the Framingham equation, resulting in a new equation with increased discriminative ability and therefore greater predictive value for CHD. After developing the model, we proposed to evaluate the ability of the new model to predict CHD. We hypothesized that the new model, which incorporates race, GFR, and the new weights for the traditional risk factors, would be more accurate at predicting risk for CHD in all patients and particularly in blacks and patients with CKD. 13
METHODS Overview The goal of the study was to evaluate the addition of race and CKD to risk stratification models that include the traditional Framingham risk factors. The first step was the development of a model predicting cardiovascular disease that included these two variables and the traditional Framingham variables. This model development was conducted in a developmental sample, consisting of two-thirds of the eligible subjects from the Antihypertensive and Lipid-Lowering Treatment to Prevent Heart Attack Trial (ALLHAT). The next step was to evaluate the predictive ability of the new model in comparison to a model including only the Framingham variables. This validation was conducted in publicly available data from two large cohort studies: ARIC and CHS. An additional validation will take place in the remaining one-third of ALLHAT subjects, the ALLHAT validation cohort. Model development ALLHAT ALLHAT was designed to evaluate the incidence of fatal CHD and nonfatal myocardial infarction (MI) in subjects with hypertension randomized to a calcium channel antagonist, an ACE inhibitor, an α-adrenergic blocker, or a thiazide-like diuretic (26). The study also included a cholesterol-lowering trial with some subjects randomized in both the hypertension and the cholesterol-lowering trial (26). The study received institutional review board approval, and participants provided written informed consent. Participants were men and women with hypertension, 55 years of age or older, with at least 1 additional risk factor for CHD (atherosclerotic cardiovascular disease, type II 14
diabetes, HDL cholesterol < 35 mg/dl, left ventricular hypertrophy, electrocardiogram changes indicative of ischemia, or current smoking) (26). Detailed inclusion and exclusion criteria as well as baseline characteristics have been described elsewhere (27). Of note, subjects were excluded if they had symptomatic MI, stroke, or angina pectoris in the prior 6 months or a serum creatinine 2 mg/dl (26). Exclusion criteria For this analysis, all subjects in ALLHAT were included with the following exceptions: (1) Patients over age 74 were excluded to allow for comparison with the Framingham equation (12). (2) Patients with missing data for HDL were excluded as this is necessary to calculate CHD risk using the Framingham equation (12). (3) Patients with missing data for both cldl and total cholesterol were excluded because at least one of these variables is necessary to calculate CHD risk using the Framingham equation (12). (4) Patients with a history of CHD as a response to a single question on the baseline questionnaire were excluded. CHD was defined as a history of MI, primary cardiac arrest, coronary revascularization, angina, coronary stenosis greater than 50%, or reversible coronary perfusion defect on noninvasive cardiac testing. The reason for excluding patients with baseline CHD is because the Framingham equation was developed to predict incident CHD events rather than recurrent. We considered using a combination of ALLHAT inclusion criteria (old or ageindeterminate MI or stroke, history of revascularization procedure, other 15
documented atherosclerotic cardiovascular disease, and major ST depression or T- wave inversion on ECG) as an exclusion criterion. A concern with using ALLHAT inclusion criteria to identify subjects with baseline CHD is a lack of specificity. For example, other documented atherosclerotic cardiovascular disease could be a history of transient ischemic attack. Also of concern is the lack of sensitivity. For instance, a physician could have noted that a potential subject had a history of diabetes and was therefore eligible and did not note a history of angioplasty, as only one inclusion criteria was required. The question regarding CHD on the baseline questionnaire is likely more sensitive and specific. Baseline measurements At baseline, participants had blood drawn for the measurement of fasting glucose, total cholesterol, HDL cholesterol, triglycerides, and creatinine. Levels of serum creatinine were measured in a single central laboratory using VITROS (Ortho Clinical Diagnostics, Rochester, New York), which had a coefficient of variation of approximately 2%. Creatinine measurements were calibrated to the Modification of Diet in Renal Disease (MDRD) laboratory, and no additional correction was found to be necessary (28). The simplified MDRD Study equation was used to estimate GFR (ml/min per 1.73m 2 ): 186.3 X (serum creatinine in mg/dl -1.154 ) X (age in years -0.203 ) X 1.212 (if black) X 0.742 (if female) (29). Patients were classified into 3 baseline categories of GFR, consistent with recommendations in the National Kidney Foundation Kidney Disease Outcomes Quality Initiative Clinical Practice Guidelines on Chronic Kidney Disease: normal or increased ( 90 ml/min per 1.73m 2 ), mild reduction (60 to 90 ml/min per 1.73m 2 ), and moderate or 16
severe reduction (<60 ml/min per 1.73m 2 ) (30). Because subjects with a serum creatinine level greater than 2 mg/dl were excluded, only a small percentage of patients (0.6%) had severe chronic kidney disease (estimated GFR 29 ml/min per 1.73m 2 ). Therefore, these subjects were grouped along with those with moderate chronic kidney disease (GFR 30 to 59 ml/min per 1.73m 2 ) (2, 8). For the present study, a participant was considered to have diabetes if a history of diabetes was noted on the baseline questionnaire or baseline fasting glucose was greater than or equal to 126mg/dL. For the present study, baseline blood pressures were used per standard ALLHAT definitions (26). Primary outcome The primary outcome for this study is the combined CHD outcome from ALLHAT, which included fatal CHD, nonfatal MI, coronary revascularization, or hospitalized angina. Coronary revascularization included coronary artery bypass grafting, percutanous angioplasty, insertion of stents, and atherectomy. This outcome was chosen to be consistent with the Framingham hard CHD outcome: recognized and unrecognized MI, coronary insufficiency, and coronary heart disease death (12). The full Framingham outcome included angina; a subject was deemed to have angina if two physicians interviewing the subject agreed it was definitely present. In Framingham, coronary insufficiency was diagnosed when a history of ischemic chest pain lasting one hour or more was associated with transient ischemic electrocardiographic criteria (31). 17
There are many advantages to the use of the ALLHAT combined CHD outcome. First, it was the primary outcome for ALLHAT, so the events have been well adjudicated. Second, there were a large number of events which provided significant power to our analyses. Finally, the combined CHD outcome closely resembles the Framingham hard CHD outcome for which beta-coefficients are available for predicting coronary death and MI at 5 years (15). The main disadvantage is a lack of direct comparison to the Framingham equation in use by clinicians. One additional primary outcome was considered, the combined CHD outcome plus treated angina which could be compared to the full Framingham CHD outcome. Beta coefficients are available for the full Framingham CHD outcome (12). Clinically, the outcomes are similar; the main difference would be the 5-yr risk for ALLHAT compared to the 10-yr risk for Framingham. Validation Samples Data sets from two cohort studies (ARIC and CHS) were obtained from NHLBI as part of their Limited Access Data Program and served as the primary validation cohorts. In addition, one third of eligible ALLHAT subjects were randomized to a validation sample. The ALLHAT validation sample will serve as a secondary validation set. Atherosclerosis Risk in Communities Study ARIC was a prospective cohort study designed to evaluate the etiology and consequences of atherosclerosis (32). Subjects aged 45-64 years from Forsyth County, North Carolina, Jackson, Mississippi, suburbs of Minneapolis, Minnesota, and Washington County, 18
Maryland were enrolled and monitored for CHD through review of all hospitalizations and deaths in those communities. Of note, the Jackson cohort was limited to blacks. For the present analyses, subjects with a history of CHD or angina were excluded. Baseline variables were obtained from visit one and include race, gender, age, smoking, diabetes, cholesterol, HDL, systolic and diastolic blood pressure, and creatinine. Because serum creatinine values vary from lab to lab, the ARIC creatinine values were adjusted to the Cleveland Clinic laboratory (used in the MDRD study) values by subtracting 0.24 mg/dl prior to calculating GFR using the MDRD equation (33). The primary CHD outcome for the present analyses includes definite fatal CHD, definite fatal MI, possible fatal CHD, and definite MI. Cardiovascular Health Study CHS was designed to evaluate the significance of traditional cardiovascular risk factors in older adults (34). CHS was a population based cohort study in adults aged 65 years and older. Subjects were enrolled from four communities: Forsyth County, North Carolina; Sacramento County, California; Washington County, Maryland; and Pittsburgh, Pennsylvania. Subjects were followed with formal examinations and interim contacts to evaluate the incidence of cardiovascular disease. For the present analyses, subjects with a history of CHD, angina, or MI were excluded. Baseline variables were obtained from visit one and include race, gender, age, smoking, diabetes, cholesterol, HDL, systolic and diastolic blood pressure, and creatinine. CHS creatinine values were adjusted to the Cleveland Clinic laboratory values by subtracting 0.11 mg/dl (33). The primary CHD 19
outcome for the present analyses includes MI, angioplasty, CABG, electrocardiographic MI, and other CHD deaths. Statistical Analyses Specific Aim 1 Develop a new model Cutoffs for total cholesterol (<160, 160 to 199, 200 to 239, 240 to 279, and 280 mg/dl) and HDL (<35, 35 to 44, 45 to 49, 50 to 59, and 60 mg/dl) were used in this analysis to be consistent with the original Framingham equation (12). Systolic blood pressure and diastolic blood pressure were classified according to the updated JNC VII guidelines. GFR was categorized as mentioned above and race was categorized as black or nonblack. In the ALLHAT developmental sample, Cox proportional hazards models were used to determine the effect of age, smoking status, diabetes, blood pressure, total cholesterol, HDL, and GFR on time to CHD in men and women. Tests for interaction will be performed. Four and six-year follow-up were used in the proportional hazards model and adapted to provide a 5-year CHD risk estimate. Informative censoring will be evaluated by performing Kaplan-Meier analyses of censored times. The proportional hazards assumption will be evaluated using Schoenfeld residuals. Each model will be assessed by examining deviance residuals and evaluating outliers. Two models were developed in the ALHAT sample. The first included only the original Framingham risk factors (age, blood pressure, total cholesterol, HDL, diabetes, and 20
smoking) with stratification by gender. The second included all previous risk factors plus CKD stage with stratification by gender and race. Specific Aim 2 Validate and compare the performance of the new model to a model including only the traditional Framingham risk factors Model evaluation Prediction model evaluation has traditionally involved a comparison of C-statistics, or the area under the receiver operating characteristic (ROC) curve, also known as AUC (35). Additionally, some authors appropriately include an evaluation of the overall model fit, that is, how well the predicted probabilities compare to observed probabilities within prespecified groups. Model fit is typically evaluated using the Hosmer-Lemeshow goodness-of-fit test. Overall model performance can also be evaluated using R 2, a measure of the proportion of variance in outcomes explained by each model. New methods for evaluating model performance have been developed and include net reclassification improvement (NRI) (24). Methods for model fit have been developed to assess fit within these reclassification categories using a test similar to the Hosmer- Lemeshow test (36). For these analyses, models were evaluated using an overall performance measure (R 2 ), two measures of discrimination (C-statistic and NRI), and two measures of calibration (the traditional Hosmer-Lemeshow test and goodness-of-fit within reclassification categories). R 2 and C-statistic were chosen because they are well known measures of model performance. C-statistics for models with and without race and CKD were 21
compared using a non-parametric approach (37). NRI was added because of the known insensitivity of the C-statistic to small changes in model improvement (24, 25). Measuring changes in C-statistics is an insensitive method for evaluating model improvement, especially for risk stratification models that already have moderate to high levels of discrimination, even with the addition of highly significant biomarkers (24, 25). Finally, the Hosmer-Lemeshow goodness-of-fit test was chosen because it is a commonly used measure of model fit and is easy to understand. As a secondary analysis, the goodness-of-fit within reclassified categories test was also conducted to enable a comparison of the two measures of fit. These analyses were conducted in the ARIC and CHS cohorts. Further validation and model assessment will take place in the ALLHAT validation cohort. There are several limitations to the proposed evaluation. The main drawback to the use of R 2 for these analyses is that it is unstable for subjects with events with very low predicted probabilities. Given the low probability of CHD in the cohorts, this is a concern. Another option for measuring overall performance would have been the Brier test. However, the Brier depends on the incidence which makes comparison across groups difficult. Limitations of NRI are that all reclassifications are considered equal and reclassification depends on where the category boundaries are located (38). R 2 and C-statistics were calculated using logistic regression with predicted probability of CHD as the independent variable. The Hosmer-Lemeshow goodness-of-fit test was also 22
calculated using logistic regression. The goodness-of-fit within reclassification categories was calculated using reclass.macro.sas (35). NRI was calculated as described below. Net reclassification improvement NRI assesses the ability of a model to classify individuals in comparison to a second model. For the present study, NRI assesses the ability of the new model including race and CKD to classify subjects into risk categories compared to a model including only the traditional Framingham risk factors. The NRI was calculated using the risk categories established by ATP III, adjusted for a 5-year rather than 10-year follow-up (<5%, 5-10%, and >10%) (11). The NRI was calculated as follows:(24) P up, events = P down, events = P up, nonevents = P down, nonevents = NRI = (P up, events P down, events ) (P up, nonevents P down, nonevents ) 23
A test for the null hypothesis of NRI = 0 based on the assumption of independence follows (24): z = Secondary Aims 1 and 2 Validate and compare the new model to a model including only the traditional Framingham risk factors in blacks and patients with CKD As in Specific Aim 2, the new model was compared to a model including only the traditional Framingham risk factors in black patients and patients with CKD from: a) ARIC and CHS and b) the ALLHAT validation cohort. RESULTS Baseline characteristics 42,418 subjects were randomized in ALLHAT. Of these, 25,177 were eligible for the current analyses (see Figure 1). Baseline blood pressures for the developmental sample are shown in Table 1. As expected, given that ALLHAT was a clinical trial of hypertension, a significant proportion of subjects had hypertension. Those with normal blood pressures were eligible for the study because they were receiving treatment for hypertension. Baseline characteristics for the ALLHAT developmental sample are shown in Table 2. In general, men were more likely than women to be smokers, especially amongst blacks, and had lower total cholesterol and HDL. Blacks had higher HDL than non-blacks. 24
Figure 1. CONSORT Diagram for ALLHAT samples Total Randomized in ALLHAT : 42,418 2,083 missing cholesterol measurement 10,360 with baseline CHD 4,798 over 74 years of age 25,177 eligible subjects 2/3 Developmental Sample: 16,785 1/3 Validation Sample: 8,392 Table 1. Baseline characteristics of the ALLHAT developmental sample according to JNC-V hypertension categories Systolic, mmhg Diastolic, mmhg Normal (including optimal) BP < 120 < 85 High Normal BP 120-139 85-89 Hypertension stage I 140-159 90-99 Hypertension stage II-IV 160 100 N (%) 654 (11.9) 815 (14.8) 2821 (51.3) 1211 (22.0) Non-black N (%) 483 (10.4) 712 (15.3) 2390 (51.3) 1075 (23.1) N (%) 339 (11.3) 490 (16.3) 1421 (47.3) 756 (25.2) Black N (%) 438 (12.1) 536 (14.8) 1736 (48.0) 908 (25.1) 25
Table 2. Baseline characteristics of the ALLHAT developmental cohort Non-black Black (n= 5501) (n= 4660) (n= 3006) (n= 3618) Age, mean 64.3 (n=5,501) 64.0 (n=4,660) 64.0 (n=3,006) 63.5 (n=3,618) Cigarette use, % No 74.8 (4,115) 75.7 (3,526) 65.3 (1,963) 77.1 (2,790) Yes 25.2 (1,386) 24.3 (1,134) 34.7 (1,043) 22.9 (828) Diabetes, % 41.1 (2,258) 41.7 (1,944) 42.5 (1,276) 47.8 (1,729) Total cholesterol, 208.1 (5,501) 228.3 (4,660) 206.7 (3,006) 225.3 (3,618) mg/dl cldl, mg/dl 131.2 (4,999) 139.9 (4,244) 132.4 (2,911) 143.3 (3,521) HDL, mg/dl 40.2 (5,501) 49.4 (4,660) 47.1 (3,006 ) 54.7 (3,618 ) Creatinine, mg/dl 1.06 (5,499) 0.85 (4,655) 1.19 (3,002) 0.94 (3,612) There were 15,732 subjects in ARIC and 5,888 subjects in CHS. Of these, 16,842 were eligible for the present analyses (see Figure 2). Figure 2. CONSORT Diagram for ARIC and CHS validation cohorts Total number of subjects: 15,732 ARIC 5,888 CHS 21,620 Total 2,876 with baseline CHD 2,064 over 74 years of age 318 missing data 16,842 eligible subjects Baseline blood pressures for ARIC and CHS are shown in Tables 3 and 4. As expected, subjects were less likely to have hypertension in these community cohort studies than subjects from ALLHAT. 26
Table 3. Baseline characteristics of the ARIC cohort according to JNC VII blood pressure categories, N (%) Non-black Black Blood pressure (n = 4698) (n = 5445) (n = 1471) (n = 2379) < 120/80 2339 (49.8) 3182 (58.4) 389 (26.4) 797 (33.5) 120-139/80-89 1741 (37.1) 1628 (29.9) 570 (38.8) 915 (38.5) 140-159/90-99 491 (10.5) 524 (9.6) 330 (22.4) 440 (18.5) 160/100 127 (2.7) 111 (2.0) 182 (12.4) 227 (9.5) Table 4. Baseline characteristics of the CHS cohort according to JNC VII blood pressure categories, N (%) Non-black Black Blood pressure (n = 1010) (n = 1651) (n = 194) (n = 295) < 120/80 128 (12.7) 311 (18.8) 22 (11.3) 30 (10.2) 120-139/80-89 432 (42.8) 718 (43.5) 74 (38.1) 93 (31.5) 140-159/90-99 302 (29.9) 445 (27.0) 54 (27.8) 93 (31.5) 160/100 148 (14.7) 177 (10.7) 44 (22.3) 79 (26.8) Baseline characteristics for the ARIC and CHS cohorts are shown in Tables 5 and 6. In general, blacks were more likely to have diabetes than non-blacks. Similar to the ALLHAT cohort, men had lower total cholesterol and HDL. Blacks had higher HDL than non-blacks. In comparison to the CHS cohort, the ARIC cohort was younger and had a better CHD risk profile with lower systolic blood pressures, fewer subjects who smoked, a lower percentage with diabetes, and higher average GFRs. These differences are reflected in the greater 5-year survival functions for the ARIC cohort. 27
Model development ALLHAT Cox proportional hazards models were used to develop models in the ALLHAT developmental cohort, stratifying by gender. First, a model was developed including only the traditional Framingham risk factors (Model 1). Then a model was developed that included CKD stage with additional stratification by race (Model 2). The beta coefficients for these models are shown in Table 7. The coefficients from the Framingham equation predicting the hard outcome at 5-years are shown for comparison (15). 28
Table 5. Baseline characteristics of the ARIC cohort* All Non-blacks Black (n = 6169) (n = 7824) (n = 4698) (n = 5445) (n = 1471) (n = 2379) Age, years 54.3 (5.8) 53.7 (5.7) 54.5 (5.7) 53.9 (5.7) 53.7 (5.9) 53.2 (5.7) Cigarette use, N (%) 1692 (27.4) 1934 (24.8) 1134 (24.1) 1349 (24.8) 558 (38.0) 585 (24.7) Diabetes, N (%) 519 (8.5) 735 (9.5) 310 (6.6) 321 (5.9) 209 (14.5) 414 (17.9) Systolic blood pressure, mmhg 122.5 (17.8) 120.1 (19.3) 120.1 (15.9) 116.8 (17.6) 130.2 (21.0) 127.7 (21.0) Diastolic blood pressure, mmhg 75.8 (11.2) 72.3 (10.9) 73.7 (9.9) 69.8 (9.8) 82.5 (12.7) 78.0 (11.3) Total cholesterol, mg/dl 210.4 (39.5) 217.6 (43.3) 210.2 (38.2) 217.8 (42.0) 211.1 (43.8) 217.0 (46.3) HDL, mg/dl 44.9 (14.0) 57.8 (17.2) 43.1 (12.4) 57.8 (17.1) 50.9 (16.9) 58.0 (17.4) Creatinine, mg/dl 1.0 (0.4) 0.8 (0.4) 1.0 (0.2) 0.7 (0.1) 1.1 (0.7) 0.8 (0.7) GFR, ml/min per 1.73m 2 92.4 (19.4) 94.2 (22.8) 89.6 (16.8) 89.9 (18.5) 101.8 (23.6) 104.2 (28.1) Baseline 5yr survival, (S(t)) 0.970 0.990 0.972 0.994 0.965 0.983 * mean (SD) unless noted Table 6. Baseline characteristics of the CHS cohort* All Non-blacks Black (n = 1214) (n = 1953) (n = 1010) (n = 1651) (n = 194) (n = 295) Age, years 70.1 (2.6) 69.8 (2.6) 70.1 (2.5) 69.8 (2.6) 70.1 (2.7) 69.8 (2.6) Cigarette use, N (%) 175 (14.4) 303 (15.5) 125 (12.4) 257 (15.6) 49 (25.4) 46 (15.7) Diabetes, N (%) 208 (17.2) 235 (12.2) 155 (15.4) 169 (10.3) 50 (26.0) 64 (22.3) Systolic blood pressure, mmhg 139.1 (19.3) 137.0 (19.9) 138.7 (18.9) 135.3 (18.9) 141.7 (21.3) 126.2 (22.4) Diastolic blood pressure, mmhg 69.8 (11.4) 74.9 (11.1) 74.4 (10.8) 68.9 (11.2) 77.5 (12.7) 74.7 (11.1) Total cholesterol, mg/dl 199.9 (34.7) 222.1 (38.4) 200.2 (34.4) 223.1 (38.0) 198.0 (35.8) 217.3 (40.4) HDL, mg/dl 48.2 (12.8) 59.7 (15.8) 47.5 (12.7) 59.4 (15.9) 51.8 (12.6) 61.6 (15.2) Creatinine, mg/dl 1.1 (0.4) 0.8 (0.2) 1.1 (0.3) 0.8 (0.2) 1.1 (0.7) 0.8 (0.3) GFR, ml/min per 1.73m 2 81.2 (23.4) 85.8 (24.7) 78.3 (20.5) 83.5 (22.8) 96.8 (31.1) 99.2 (30.5) Baseline 5yr survival, (S(t)) 0.876 0.954 0.869 0.958 0.906 0.931 * mean (SD) unless noted 29
Table 7. Beta coefficients for ALLHAT models without (Model 1) and with CKD and race (Model 2) and the Framingham equation. Model 1 Model 2 Framingham (15) All subjects Non-black Black All Subjects Variable Age, y 0.03 0.06 0.04 0.06 0.02 0.06 0.05 0.17 Age squared, y -.0004-0.002-0.001-0.001 0.0007-0.002-0.001 Total cholesterol, mg/dl < 160-0.21-0.38-0.33-0.41 0.05-0.32-0.38-0.21 160-199 -0.09-0.11-0.12-0.22 0.03 0.07 * * 200-239 * * * * * * 0.57 0.44 240-279 0.15 0.19 0.20 0.23 0.02 0.10 0.74 0.56 280 0.46 0.34 0.45 0.09 0.53 0.57 0.83 0.89 HDL, mg/dl < 35 0.27 0.20 0.32 0.10-0.09 0.76 0.61 0.73 35-44 * * * * * 0.38 0.37 0.60 45-49 0.04-0.37 0.08-0.47 0.03 0.12 * 0.60 50-59 -0.13-0.25-0.07-0.16-0.15 0.03 0.00 * 60-0.23-0.48 0.04-0.57-0.40 * -0.46-0.54 Blood pressure < 120/80 0.11 0.07 0.09 0.02 0.15 0.07 * * 120-139/80-89 -0.11-0.02 0.06 0.07-0.47-0.15 0.42 a -0.37 a 140-159/90-99 * * * * * * 0.66 0.22 160/100 0.13 0.24 0.22 0.27-0.05 0.17 0.90 0.61 Diabetes 0.34 0.57 0.33 0.68 0.45 0.45 0.53 0.87 Smoker 0.21 0.42 0.18 0.55 0.32 0.29 0.73 0.98 GFR, ml/min per 1.73m 2 90 NA NA -0.01-0.10 0.12 0.19 NA NA 60-89 NA NA * * * * NA NA < 60 NA NA 0.18 0.22 0.31 0.70 NA NA Baseline survival function at 5 years, S(t) 0.88 0.92 0.88 0.92 0.91 0.95 (a) Framingham used 5 blood pressure categories (normal and high normal were separated) 30
Model validation The models developed in the ALLHAT cohort were then evaluated in ARIC and CHS. The first step was to calculate a predicted probability of CHD for subjects using Model 1 (without CKD and race) and Model 2 (with CKD and race). For comparison purposes, predicted probabilities were also calculated using the Framingham equation for hard outcomes (15). For each subject, a linear function is calculated based on the subject s risk factors and model coefficients (12). The linear function is then corrected for the average risk for that subject s cohort (appropriately stratified by gender and race) and the result is exponentiated. The survival function is exponentiated by this value. Finally, this value is subtracted from 1 to calculate the 5 year probability of CHD. Model validation ARIC The C-statistics for Model 1 (without CKD and race) and Model 2 (with CKD and race) as well as the original Framingham model are shown in Table 8. Both Model 1 and Model 2 had poor discrimination with C-statistics less than 0.70 in all subgroups. Among men, inclusion of CKD and race (Model 2) resulted in worse discrimination than the model without these variables (Model 1). The opposite was true for women. Both models had significantly lower C-statistics than the Framingham model among men and women regardless of gender or race. Inclusion of CKD and race (Model 2) did not significantly improve discrimination in subjects with CKD. Similar results were seen when evaluating models using R 2 (see Table 9). The model without race and CKD (Model 1) explains approximately 1-3% of the variance among 31
men and women while the model including race and CKD (Model 2) explains less than 1% of the variance among men and between 1% and 7.5% of the variance in women. Again, the R 2 values for the Framingham model were higher than Model 1 and 2 for nearly all subgroups. Table 8. C-statistic for Models 1 and 2 (includes race and CKD) and Framingham by race and gender in ARIC All subjects Model 1 Model 2 Framingham Model 1 Model 2 Framingham All 0.628 0.553 * 0.725 0.506 0.538 * 0.844 Non-black 0.643 0.549 * 0.734 0.480 0.699 * 0.811 Black 0.591 0.547 0.663 0.604 0.625 * 0.825 Subjects with CKD Model 1 Model 2 Framingham Model 1 Model 2 Framingham All 0.578 0.547 0.683 0.622 0.550 0.792 * P < 0.01 vs Model 1, P < 0.01 vs Model 1 and 2, P < 0.05 vs Model 2 Table 9. Nagelkerke R 2 for Models 1 and 2 (includes race and CKD) and Framingham by race and gender in ARIC All subjects Model 1 Model 2 Framingham Model 1 Model 2 Framingham All 0.022 0.007 0.047 0.002 0.005 0.111 Non-black 0.030 0.009 0.056 0.014 0.075 0.074 Black 0.005 0.004 0.027 0.004 0.010 0.137 Subjects with CKD Model 1 Model 2 Framingham Model 1 Model 2 Framingham All 0.021 0.001 0.015 0.031 0.002 0.119 Net reclassification improvement was also assessed. An example of reclassification is shown in Table 10. Among all men in ARIC with an event, 31 were reclassified correctly 32
(up) and 5 were reclassified down for a net reclassification of 14.7% (26/177). Among all men in ARIC without an event, 740 were reclassified incorrectly (up) and 74 were reclassified downward for a net reclassification of -11.3% (666/5911). Therefore, NRI is equal to 3.4% (14.7%-11.3%) for all men. There was no significant improvement in risk classification with the inclusion of race and CKD (see Table 11). Table 10. CHD risk classification comparing models with (Model 2) and without race and CKD (Model 1) in all men in ARIC Model 2 risk category (with CKD and race) Model 1 risk category (without race and CKD) With events (n = 177) < 5% 5-10% > 10% < 5% 129 5 0 5-10% 24 12 0 > 10% 6 1 0 Without events (n = 5911) < 5% 4950 74 0 5-10% 620 145 0 > 10% 99 21 2 Table 11. Net reclassification improvement (NRI) from Model 1 to Model 2 (includes race and CKD) by race and gender in ARIC, % (P value) All subjects All 3.4 (0.32) -2.3 (0.46) Non-black 3.4 (0.31) -7.4 (0.15) Black 1.1 (0.90) -1.6 (0.66) Subjects with CKD All 0.2 (0.99) -2.2 ( 0.06) Hosmer-Lemeshow goodness-of-fit for Models 1 and 2 and the Framingham equation are shown in Table 12. There was evidence for lack of fit for Model 1 among all groups of women except those with CKD, as well as all men and black men. For Model 2 (includes 33
race and CKD), there was evidence of lack of fit in non-black men and all women. For Framingham, there was evidence of lack of fit among all groups except black men and women with CKD. Table 12. Hosmer-Lemeshow goodness-of-fit Test for Models 1 and 2 (includes race and CKD) and Framingham by race and gender in ARIC, Chi-square All subjects Model 1 Model 2 Framingham Model 1 Model 2 Framingham All 15.8 4.4 33.7 * 27.1 * 19.1 31.7 * Non-black 7.9 18.8 25.9 * 18.0 5.9 24.4 * Black 17.9 12.3 11.7 16.3 13.2 15.7 Subjects with CKD Model 1 Model 2 Framingham Model 1 Model 2 Framingham All 6.8 8.4 12.8 6.2 4.5 5.8 * P < 0.01, P < 0.05 The reclassification calibration statistics for Models 1 and 2 are shown in Table 13. For Model 1, there was only evidence of lack of fit for all women and black women. There was evidence for lack of fit for Model 2 among all men and women and black men and women. It is interesting to note the discordant findings between the traditional Hosmer- Lemeshow test and the reclassification calibration statistic. Of the 12 groups (all, nonblack, and black men and women for each Model), the two tests of calibration were concordant (either showing evidence for lack of fit or no evidence for lack of fit) in only half of the groups (all women for Models 1 and 2, black women and non-black men for Model 1, and non-black women for Model 2). 34
Table 13. Reclassification calibration statistic for Model 1 and Model 2 (includes race and CKD) by race and gender in ARIC, Chi-square (P value) All subjects Model 1 Model 2 Model 1 Model 2 All 5.3 15.8 * 9.7 * 7.3 Non-black 5.7 4.0 19.0 0.8 Black 2.8 12.6 * 7.7 * 6.6 * Subjects with CKD Model 1 Model 2 Model 1 Model 2 All 10.8 2.9 41.4 54.0 * P < 0.01, P < 0.05 Model validation CHS The C-statistics for Model 1 (without CKD and race) and Model 2 (with CKD and race) as well as the original Framingham model are shown in Table 14. Both Model 1 and Model 2 had poor discrimination with C-statistics less than 0.70 in all subgroups and even less than 0.60 among subgroups of men with the exception of Model 1 in men with CKD. Among men, inclusion of CKD and race (Model 2) resulted in worse discrimination than the model without these variables (Model 1), except for black men where there was no evidence of a significant difference. Among non-black women, there was evidence for improved discrimination with the addition of race and CKD. In general, both models had significantly lower C-statistics than the Framingham model among men and women regardless of gender or race. Among subjects with CKD, inclusion of CKD and race (Model 2) did not significantly improve discrimination. 35
Table 14. C-statistic for Models 1 and 2 (includes race and CKD) and Framingham by race and gender in CHS All subjects Model 1 Model 2 Framingham Model 1 Model 2 Framingham All 0.570 0.521 * 0.623 0.564 0.602 0.661 Non-black 0.574 0.512 * 0.643 0.584 0.631 * 0.657 * Black 0.573 0.536 0.502 0.499 0.470 0.664 Subjects with CKD Model 1 Model 2 Framingham Model 1 Model 2 Framingham All 0.614 0.537 0.628 0.530 0.574 0.684 * P < 0.05 vs Model 1, P 0.05 vs Model 1 and 2, P < 0.05 vs Model 2 Similar results were seen when evaluating models using R 2 (see Table 15). The model without race and CKD (Model 1) explains approximately 1-2% of the variance among men and women while the model including race and CKD (Model 2) explains less than 1% of the variance among men and between 1% and 4% of the variance in women. Again, the R 2 values for the Framingham model were higher than Model 1 and 2 for nearly all subgroups. Table 15. Nagelkerke R 2 for Models 1 and 2 (includes race and CKD) and Framingham by race and gender in CHS All subjects Model 1 Model 2 Framingham Model 1 Model 2 Framingham All 0.015 0.004 0.033 0.013 0.023 0.037 Non-black 0.017 0.002 0.048 0.017 0.039 0.032 Black 0.011 0.001 0.001 0.002 0.000 0.054 Subjects with CKD Model 1 Model 2 Framingham Model 1 Model 2 Framingham All 0.049 0.015 0.056 0.004 0.015 0.054 36
NRI from Model 1 to Model 2 in CHS is shown in Table 16. NRI is negative for men and significantly so for non-black men and men with CKD. Among women, there was no significant change in classification with the inclusion of race and CKD. The trend towards worse classification among men and improved classification among women corresponded to the trends seen in evaluations based on C-statistics and R 2. Table 16. Net reclassification improvement (NRI) from Model 1 to Model 2 by race and gender in CHS, % (P value) All subjects All -6.8 (0.09) 10.9 (0.08) Non-black -10.8 (0.005) 9.3 (0.19) Black -1.4 (0.94) 1.5 (0.90) Subjects with CKD All -13.5 (0.007) 7.0 (0.59) The Hosmer-Lemeshow goodness-of-fit tests for Models 1 and 2 and the Framingham equation in the CHS cohort are shown in Table 17. There was no evidence for lack of fit for any model in any subgroup, except for the Framingham model in non-black men. However, as shown in Table 18, goodness-of-fit as assessed by the reclassification calibration statistic revealed evidence for lack of fit in nearly all groups for both models with (Model 2) and without (Model 1) race and CKD. In the CHS cohort, as in ARIC, there was significant disagreement on goodness-of-fit between the two tests. The Hosmer-Lemeshow and the reclassification goodness-of-fit tests were concordant in only 4 out of 12 subgroups. 37
Table 17. Hosmer-Lemeshow goodness-of-fit test for Models 1 and 2 (includes race and CKD) and Framingham by race and gender in CHS, Chi-square All subjects Model 1 Model 2 Framingham Model 1 Model 2 Framingham All 2.4 6.4 13.5 13.4 11.6 11.7 Non-black 2.1 6.9 19.9 * 13.2 9.6 7.8 Black 8.7 8.7 4.0 4.7 8.1 2.6 Subjects with CKD Model 1 Model 2 Framingham Model 1 Model 2 Framingham All 7.3 11.4 7.8 9.3 8.1 3.4 * P 0.01 Table 18. Reclassification calibration statistic for Model 1 and Model 2 (includes race and CKD) by race and gender in CHS, Chi-square (P value) All subjects Model 1 Model 2 Model 1 Model 2 All 6.3 39.5 * 18.4 * 4.1 Non-black 7.5 38.2 * 14.7 * 1.5 Black 3.4 2.4 10.7 8.2 Subjects with CKD Model 1 Model 2 Model 1 Model 2 All 2.0 32.7 5.4 3.3 * P < 0.001, P < 0.05 DISCUSSION The current study shows that the addition of race and CKD to a CHD prediction model including the traditional Framingham risk factors does not result in a significant change in model performance, discrimination, or risk classification. However, there are significant limitations that need to be considered when evaluating these results. 38
Model performance The main limitation of the present analyses is the poor performance of both models developed in the ALLHAT population: Model 1 including only the traditional Framingham risk factors and Model 2 including the traditional risk factors plus race and CKD. For nearly all groups of patients in the ARIC and CHS validation cohorts, the new models had worse overall performance and significantly worse discrimination than the original Framingham equation. In general, poor validation could be due to a number of factors: missing data, differences in data collection between cohorts, differences in outcome definitions, and overfitting of the new models in the ALLHAT developmental cohort (39). The poor validation is not likely due to missing data for risk factors as this was minimal and subjects with missing data were excluded to be consistent with prior evaluations of the Framingham equation. While there may be differences in the methods for data collection for risk factors and outcomes, it is unlikely that the differences would bias towards better risk prediction for the original Framingham equation in both ARIC and CHS. Similarly for outcomes, while each study had slightly different definitions for CHD, the results consistently showed poor model performance for the prediction models developed in the ALLHAT cohort. Finally, it is possible that the models developed in the ALLHAT cohorts were subject to overfitting. However, the developmental sample was large and the number of covariates only included age, blood pressure, cholesterol, HDL, diabetes, tobacco use, and CKD stage with stratification on gender (Models 1 and 2) and race (Model 2). 39