Idiopathic pulmonary fibrosis (IPF) is a progressive fibrotic

Transcription

1 Original Research Annals of Internal Medicine A Multidimensional Index and Staging System for Idiopathic Pulmonary Fibrosis Brett Ley, MD; Christopher J. Ryerson, MD, MAS; Eric Vittinghoff, PhD; Jay H. Ryu, MD; Sara Tomassetti, MD; Joyce S. Lee, MD, MAS; Venerino Poletti, MD; Matteo Buccioli, BS; Brett M. Elicker, MD; Kirk D. Jones, MD; Talmadge E. King Jr., MD; and Harold R. Collard, MD Background: Idiopathic pulmonary fibrosis (IPF) is a progressive fibrotic lung disease with an overall poor prognosis. A simple-to-use staging system for IPF may improve prognostication, help guide management, and facilitate research. Objective: To develop a multidimensional prognostic staging system for IPF by using commonly measured clinical and physiologic variables. Design: A clinical prediction model was developed and validated by using retrospective data from 3 large, geographically distinct cohorts. Setting: Interstitial lung disease referral centers in California, Minnesota, and Italy. Patients: 228 patients with IPF at the University of California, San Francisco (derivation cohort), and 33 patients at the Mayo Clinic and Morgagni-Pierantoni Hospital (validation cohort). Results: Four variables were included in the final model: gender (G), age (A), and 2 lung physiology variables (P) (FVC and DLCO). A model using continuous predictors (GAP calculator) and a simple point-scoring system (GAP index) performed similarly in derivation (c-index of 7.8 and 69.3, respectively) and validation (c-index of 69.1 and 68.7, respectively). Three stages (stages I, II, and III) were identified based on the GAP index with 1-year mortality of 6%, 16%, and 39%, respectively. The GAP models performed similarly in pooled follow-up visits (c-index 71.9). Limitation: Patients were drawn from academic centers and analyzed retrospectively. Conclusion: The GAP models use commonly measured clinical and physiologic variables to predict mortality in patients with IPF. Primary Funding Source: University of California, San Francisco Clinical and Translational Science Institute. Measurements: The primary outcome was mortality, treating transplantation as a competing risk. Model discrimination was assessed by the c-index, and calibration was assessed by comparing predicted and observed cumulative mortality at 1, 2, and 3 years. Ann Intern Med. 212;156: For author affiliations, see end of text. Idiopathic pulmonary fibrosis (IPF) is a progressive fibrotic lung disease with an unknown cause characterized by the presence of usual interstitial pneumonia pattern on surgical lung biopsy (1). Idiopathic pulmonary fibrosis has a poor prognosis with a median survival of approximately 3 years (2), with most deaths related to IPF (3). However, patients with IPF demonstrate widely variable clinical courses and survival (3 5). Thus, predicting prognosis in patients with IPF is a challenge for clinicians. Staging systems have proven useful in estimating prognosis and guiding management decisions in other lung diseases, such as asthma (6), chronic obstructive pulmonary disease (7), and lung cancer (8). Although individual variables have been associated with mortality in IPF, none See also: Print Editors Notes Web-Only Appendix Appendix Tables Appendix Figure Online index and risk calculators Conversion of graphics into slides accurately predicts prognosis in isolation (9 11). Clinical prediction models that combine individual variables have been proposed for IPF (12 14) but have had little impact, perhaps because they are difficult to use and have not been externally validated. A simple-to-use staging system in IPF may accurately inform prognosis, help guide management decisions (for example, timing of lung transplantation), and allow for appropriate life planning. Such a system could also help facilitate future research in IPF by identifying patients who are at high risk for clinically important outcomes. In this study, we developed the multidimensional GAP (gender [G], age [A], and 2 lung physiology variables [P] [FVC and DLCO]) index and staging system and the GAP calculator that use variables that are commonly measured in clinical practice to predict mortality in IPF. We propose that the GAP index and staging system may be used as a quick and simple screening method for estimating risk in patients with IPF, and the GAP calculator may then be used to estimate individual risk for certain patients in whom a more precise estimate may affect clinical decision making (online GAP index and GAP calculator tools are available at METHODS The Appendix (available at includes more details on our methods. In brief, by using competing American College of Physicians

2 Index and Staging System for IPF Original Research risks regression modeling, we retrospectively screened potential predictors of mortality in a derivation cohort of patients with IPF, identifying a model consisting of 4 predictors. On the basis of these 4 predictors, we then developed an individual risk calculator (the GAP calculator) as well as a simple point-score model and staging system (the GAP index and staging system), and we retrospectively validated both in a separate validation cohort. Study Patients The derivation cohort included 228 patients enrolled in the University of California, San Francisco, Interstitial Lung Disease Program s longitudinal cohort study between 21 and 21. The validation cohort included 33 patients entered in the Mayo Clinic at Rochester (Rochester, Minnesota) (n 28) and the Morgagni-Pierantoni Hospital (Forli, Italy) (n 122) databases between 2 and 21. Patients were included in the baseline analysis for model derivation and validation if they had a diagnosis of IPF according to established criteria (1, 15) and had pulmonary function tests available within 6 months of the initial clinic visit. A subcohort of patients with follow-up data was identified from all 3 centers for model validation in follow-up. This cohort included 325 patients with 974 unique patient visits. Local institutional review boards approved these cohorts and databases, and all patients provided informed written consent. Each site did dual data entry and verification. Primary Outcome The primary outcome was mortality. Lung transplantation was treated as a competing risk. Vital status and date of death were verified by using the U.S. Social Security Death Index and the Italian death registry. Predictor Variables Predictor variables were considered if they were viewed as commonly measured and available at the time of initial consultation. Candidates included age, sex, body mass index, smoking status (those who had ever smoked versus those who had never smoked), use of long-term oxygen therapy (oxygen use), FVC, FEV 1, total lung capacity, and DLCO (corrected for hemoglobin level when available). Pulmonary function tests were done according to standard criteria (16 18). Values for DLCO were unavailable for 12 patients (5.3%) in the derivation cohort and 32 patients (9.7%) in the validation cohort, primarily because patients with very poor lung function could not do the test (defined as when the test is ordered and attempted, but the patient cannot complete it because of respiratory limitations). To address this, an indicator was created for those who could not do the DLCO test. Statistical Analysis All analyses were done by using STATA 12 (Stata Corp, College Station, Texas). Predictions of mortality risk were based on Fine Gray models for survival, treating transplantation as a competing risk (19). Candidate models Context Wide variation in the clinical course of patients with idiopathic pulmonary fibrosis makes predicting survival difficult. Contribution Using variables readily available at clinical encounters, a point-scoring stage model and a continuous calculator were derived and externally validated. Each performed well in predicting survival in patients with idiopathic pulmonary fibrosis. Implication The point-scoring and calculator models may be useful in identifying patients at heightened risk for death and in guiding care and clinical research. The Editors were screened by using 2 repetitions of 1-fold crossvalidation of the Harrell c-index, which is a measure of discrimination (2). To estimate the c-index in the presence of competing risks, we used the approach of Wolbers and colleagues (21). Age effects were modeled by using a 3-knot restricted cubic spline. Oxygen use was removed from consideration because it had substantially different effects in the derivation and validation cohorts, calling into question its generalizability. The final continuous model was selected to maximize the cross-validated c-index and face validity. From the selected continuous model, each continuous predictor was divided into clinically meaningful categories and then included as categorical predictors in a Fine Gray competing risks model for survival. Each category was assigned 1 to 3 points by rescaling then rounding the regression coefficients estimated by the categorical model. Next, a Fine Gray model was fit with the total point score for each patient as a continuous predictor. Finally, a staging system was created by grouping point scores into 3 groups comprising the 4% with the lowest estimated risk (stage I), the next 4% with intermediate risk (stage II), and the 2% with the highest risk (stage III). This categorization corresponded to estimated 1-year mortality risks of less than 1%, 1% to 3%, and greater than 3%, respectively. For both models, optimism-corrected estimates of the c-index were obtained, first internally by using 2 repetitions of 1-fold cross-validation within the derivation cohort, and then externally by using coefficient estimates from the derivation cohort to calculate the linear predictor for patients in the validation cohort. Bootstrap resampling with 5 repetitions was used to calculate 95% biascorrected bootstrap CIs for the c-index. In addition, calibration was assessed by comparing nonparametric estimates of cumulative mortality incidence at 1, 2, and 3 years, treating transplantation as a competing risk. Calibration was done by stage for both models; how May 212 Annals of Internal Medicine Volume 156 Number 1 685

3 Original Research Index and Staging System for IPF Table 1. Patient Characteristics Characteristic Derivation Cohort (n 228)* Validation Cohort (n 33) P Value Mean age (SD), y 69.7 (8.7) 66.3 (8.7).1 Male sex, % Ever smoked, % Mean BMI (SD), kg/m (4.8) 29.1 (4.5).1 Long-term oxygen therapy, % Biopsy-proven, % Mean FVC (SD), % predicted 68.8 (18.4) 68.2 (18.).68 Mean FEV 1 (SD), % predicted 77. (18.7) 73.2 (18.8).2 Mean TLC (SD), % predicted 67. (15.3) 67.2 (12.9).87 Mean DLCO (SD), % predicted 44.2 (16.6) 45.8 (14.).24 Treatment received, % Prednisone Azathioprine N-acetylcysteine BMI body mass index; TLC total lung capacity. * Exceptions for number of patients: BMI (n 221), FVC (n 226), FEV 1 (n 224), TLC (n 218), and DLCO (n 216). Exceptions for number of patients: sex (n 329), biopsy-proven (n 329), FEV 1 (n 329), TLC (n 286), DLCO (n 298), and treatment received (n 329). ever, for the continuous model, we further subdivided stage II into 3 groups of equal numbers (stages IIa, IIb, and IIc). The procedure was first done internally in the derivation cohort and then externally in the validation cohort by using predictions based on the derivation cohort. To improve calibration, shrinkage of the linear predictor toward the center was used, based on cross-validation (2). The continuous model was then updated by using coefficients and cumulative subdistribution hazards based on the combined cohort to produce the most generalizable estimates for clinical use (Appendix Figure, available at Finally, the predictive accuracy of the continuous and point-score models was assessed for patients during followup. Specifically, both models were used to estimate subsequent mortality risk among surviving patients, based on predictors assessed 6, 12, 18, and 24 months ( 3 months) after baseline for the primary analysis. Model performance was assessed by the same measures of discrimination and calibration used for the primary analysis. Role of the Funding Source The University of California, San Francisco Clinical and Translational Science Institute, provided funding for Table 2. Model Performance in the GAP Calculator (Continuous Model) Variable Derivation Cohort Validation Cohort* Combined Cohort Predicted Observed Predicted Observed Predicted Observed C-index (95% CI) 7.8 ( ) 69.1 ( ) 69.5 ( ) 1-y mortality, % Stage I Stage IIa Stage IIb Stage IIc Stage III y mortality, % Stage I Stage IIa Stage IIb Stage IIc Stage III y mortality, % Stage I Stage IIa Stage IIb Stage IIc Stage III GAP gender, age, and 2 lung physiology variables (FVC and DLCO). * Predicted estimates use shrinkage factor based on cross-validation in the derivation cohort (see Methods section for more information) May 212 Annals of Internal Medicine Volume 156 Number 1

4 Index and Staging System for IPF Original Research Figure 1. Distribution and attributed mortality risk of predictors in the GAP calculator. 3 Men 3 Women Age, y Age, y FVC, % FVC, % DLCO, % DLCO, % 1-y risk Patients who cannot do DLCO test Patients who can do DLCO test 2-y risk Patients who cannot do DLCO test Patients who can do DLCO test 3-y risk Patients who cannot do DLCO test Patients who can do DLCO test Histograms (bar plots) show the percentage of distribution of the predictor in the combined cohort. Curves (lines) show the mortality risk attributed to the predictor, whereas the other predictors are held constant at their sample means. The y-axis is scaled to represent both percentage of distribution for the histograms and percentage of mortality for the curves. Note: In the upper 4 panels, which represent age and FVC in men and women, the 1-y risk curve for patients who cannot do the DLCO test overlaps with the 2-y risk curve for patients who can perform the DLCO test, creating the appearance of a red-and-green dashed line. GAP gender, age, and 2 lung physiology variables (FVC and DLCO). the study. The funding source had no role in the study design, analysis, or interpretation or in the decision to submit the manuscript for publication. RESULTS Patient Characteristics Table 1 shows baseline characteristics of both cohorts. Eighty-nine deaths and 15 lung transplantations occurred in the derivation cohort, and 186 deaths and 2 lung transplantations occurred in the validation cohort. Median follow-up was 1.7 years (range,.3 to 9.1 years) in the derivation cohort and 2.4 years (range,.1 to 9. years) in the validation cohort. Median time to death or transplantation was 3.3 years in the derivation cohort and 3.2 years in the validation cohort. The proportion of patients receiving treatment at baseline differed between study cohorts; however, treatment was not associated with outcome on bivariate or multivariate analysis in either cohort (data not shown). Derivation and Validation of GAP Models Model screening selected the following 4 predictors for the continuous model (the GAP calculator): gender, age, and 2 lung physiology variables (FVC and DLCO) (Appendix Tables 1 and 2, available at The 15 May 212 Annals of Internal Medicine Volume 156 Number 1 687

5 Original Research Index and Staging System for IPF Figure 2. The GAP index and staging system. G A P Stage I II III Points Mortality 1-y 2-y 3-y Predictor Gender Female Male Age, y >65 Physiology FVC, % predicted > <5 DLCO, % predicted > Cannot perform Total Possible Points Points Points are assigned for each variable of the scoring system to obtain a total point score (range, 8). Patients should be scored in the Cannot perform category for DLCO if their symptoms or lung function prohibited performance of the DLCO maneuver. If DLCO is unavailable because it was not ordered or not completed because of nonrespiratory limitations, then the model cannot be applied. The total point score is used to classify patients as stage I ( 3 points), stage II (4 5 points), or stage III (6 8 points). Model-predicted 1-, 2-, and 3-y mortality is shown by stage. GAP gender, age, and 2 lung physiology variables (FVC and DLCO). c-index for the GAP calculator was 7.8 (95% CI, 63.7 to 75.) in the derivation cohort, 69.1 (CI, 64.3 to 73.5) in the validation cohort, and 69.5 (CI, 65.6 to 72.7) in the combined cohort. Calibration at 1 to 3 years was satisfactory in all 3 cohorts (Table 2). Figure 1 shows the functional form and distribution of each predictor in the GAP calculator. Points were assigned to variable categories to create a point-score model (the GAP index), as shown in Figure 2. The c-index for the GAP index was 69.3 (CI, 62.2 to 73.1) in the derivation cohort, 68.7 (CI, 64.9 to 72.7) in the validation cohort, and 69.7 (CI, 66.5 to 72.6) in the combined cohort. Total point scores were grouped into 3 stages (the GAP staging system). Calibration at 1 to 3 years was again satisfactory (Table 3). Cumulative mortality incidence differed substantially among stages for both models (Figure 3). Model performance was also evaluated later in the disease course. We identified 974 unique follow-up visits in 325 patients. The c-index in this pooled follow-up cohort was 71.9 (CI, 68.5 to 75.3) for the GAP calculator and 72.3 (CI, 67.3 to 76.3) for the GAP index. Discrimination was good at individual follow-up time points of 6 to 24 months, with a c-index of 68. or greater (Appendix Table 3, available at Calibration was slightly compromised, with some overprediction of risk, especially for the lower-risk groups. DISCUSSION We developed and validated the multidimensional GAP index and staging system and GAP calculator that use commonly measured clinical and physiologic variables to predict mortality in IPF. The index and staging system is based on an easily calculated point score that has discrim- Table 3. Model Performance in the GAP Index (Point-Score Model) Variable Derivation Cohort Validation Cohort* Combined Cohort* Predicted Observed Predicted Observed Predicted Observed C-index (95% CI) 69.3 ( ) 68.7 ( ) 69.7 ( ) 1-y mortality, % Stage I Stage II Stage III y mortality, % Stage I Stage II Stage III y mortality, % Stage I Stage II Stage III GAP gender, age, and 2 lung physiology variables (FVC and DLCO). * Predicted estimates use shrinkage factor based on cross-validation in the derivation cohort (see Methods section for more information) May 212 Annals of Internal Medicine Volume 156 Number 1

6 Index and Staging System for IPF Original Research inatory power similar to its more complex continuous model, as well as prognostic models used widely in other diseases (22 24). We propose that the GAP index and staging system be used as a simple risk screening method for patients with IPF, and that the GAP calculator be used for certain patients in whom more precise estimation of risk may change management (online index and calculator tools are available at We believe that the GAP index and staging system and the GAP calculator are complimentary and have important implications for both clinical practice and research (Table 4). Specifically, we believe that they provide clinicians and patients with a framework for discussing prognosis, policymakers with a tool for investigating stage-specific management options, and researchers with the ability to identify at-risk study populations that maximize the efficiency and power of clinical trials. To demonstrate the complimentary roles of the GAP models in management decision making, we describe their use in 4 theoretical patients who are being considered for lung transplantation. In these examples, we assume that the overall risk for mortality after lung transplantation is 2% to 25% at 1 year, 3% to 35% at 2 years, and 4% to 45% at 3 years (25). Patient 1 is a 6-year-old woman with an FVC of 65% and a DLCO of 4% (GAP index 2, GAP stage I). Further risk estimation using the GAP calculator is probably not indicated because the average risk for mortality for stage I patients is well below that of transplantation at 1, 2, and 3 years (6%, 11%, and 16%, respectively). Therefore, the patient probably does not require immediate listing for lung transplantation. Patient 2 is a 62-yearold man with an FVC of 68% and a DLCO of 4% (GAP index 4, GAP stage II). Because the average risk for mortality for stage II patients is near that of transplantation, more precise risk estimation may be appropriate to guide decision making. Applying the GAP calculator, the average estimated mortality of patients with this profile at 1, 2, and 3 years is 14%, 29%, and 41%, respectively. This risk is below that of transplantation at 1 and 2 years, and thus the Figure 3. Cumulative mortality in the combined cohort by stage for the GAP calculator and GAP index and staging system GAP Calculator Year Stage III Stage IIc Stage IIb Stage IIa GAP Index and Staging System Year Stage I Stage III Stage II Stage I GAP gender, age, and 2 lung physiology variables (FVC and DLCO). Table 4. Proposed Utility of the Staging System Stage Clinical Utility Research Utility Stage I Low risk for mortality at 1 y (5.6%) Close monitoring (every 6 mo) for evidence of disease progression may be appropriate May not require immediate listing for lung transplantation Aggressive management of symptoms and comorbid conditions Stage II Moderate risk for mortality at 1 y (16.2%) Close monitoring (every 3 6 mo) for evidence of disease progression Consider listing for lung transplantation based on patient preferences, evidence of disease progression, and individual risk assessment by using the GAP calculator Stage III High risk for mortality at 1 y (39.2%) List immediately for lung transplantation if appropriate Palliative care referral if not a transplant candidate May not be ideal for mortality-driven clinical trials because of infrequent events at 1 and 2 y May be better for symptom and quality-of-life driven clinical trials May be ideal for mortality-driven clinical trials because of the moderate number of events at 1 and 2 y May not be ideal for mortality-driven clinical trials, given advanced and possibly irreversible disease Would be appropriate for trials focused on pretransplantation and palliative care GAP gender, age, and 2 lung physiology variables (FVC and DLCO) May 212 Annals of Internal Medicine Volume 156 Number 1 689

7 Original Research Index and Staging System for IPF patient also may not warrant immediate listing for transplantation. Patient 3 is a 62-year-old woman with an FVC of 55% who attempted but could not do the DLCO test because of dyspnea (GAP index 5, GAP stage II), and individual risk estimation using the GAP calculator shows a higher average estimated risk for mortality at 1, 2, and 3 years (25%, 46%, and 62%, respectively). On the basis of the patient s comorbid conditions and preferences, she may warrant more urgent listing for lung transplantation. Patient 4 is a 63-year-old man with an FVC of 48% and a DLCO of 24% (GAP index 6, GAP stage III), which carries an average estimated risk for mortality at 1, 2, and 3 years of 39%, 62%, and 77%, respectively higher than that of lung transplantation. He should be listed for lung transplantation immediately, if appropriate. The decision to list a patient for lung transplantation is only 1 example of how the GAP models may be used together to inform clinical care. Clearly, this decision is more complex than comparing predicted mortality risks and involves weighing other considerations, such as comorbid conditions, quality of life, and the patient s values and preferences. As new therapies arise for IPF, the GAP models may similarly play a role in selecting patients who are most likely to benefit from those therapies while minimizing harm. The GAP models have important advantages compared with 3 previously developed prediction models for IPF (12 14). First, they are the only predictor models that have been externally validated in a distinct cohort of patients with IPF, which is a critically important step. Second, the predictors included in our models are both simple to obtain and multidimensional, incorporating clinical and physiologic variables. Third, the extension of the GAP index into a staging system provides a framework for stagespecific clinical and research recommendations. The GAP models are the first prediction models in IPF based on competing-risks analysis and, thus, may be more confidently used to help guide the timing of lung transplantation because they focus attention on the risk for death before transplantation. In our study, both the derivation and validation cohorts were retrospectively analyzed (although prospectively developed as part of ongoing registries), which may affect data quality. Both cohorts were also drawn from academic centers, and the patients who were included may differ from the general IPF population because of referral bias. In addition, we excluded variables that are not reliably available to clinicians at initial or subsequent patient visits, such as change in lung function. This exclusion was intentional to make the models more universally applicable. The models can be applied only to patients for whom a DLCO test was ordered by the clinician and attempted by the patient. They may not be applied to patients for whom the DLCO test was simply not ordered, attempted, or completed because of nonrespiratory limitations (for example, claustrophobia, facial muscle weakness, or technical or equipment failures). Although the GAP models had similar discriminatory performance in follow-up, calibration was slightly compromised, overestimating risk, especially in lower risk groups. This may be because of selection bias where healthier patients were more likely to make follow-up visits and merits further study with a prospective design. Lastly, the GAP models predict mortality; they do not directly predict other outcomes, such as acute deterioration or physiologic progression. In summary, we present the multidimensional GAP index and staging system and GAP calculator that predict mortality in patients with IPF. We suggest that the GAP models are complimentary and have both clinical and research utility. Future research should study the effect of the GAP models on disease management. Expanded models that incorporate more complex baseline variables (for example, radiologic features and biomarkers) and longitudinal measurements (for example, change in FVC or DLCO) should be evaluated against this simple model for their additive prognostic value before incorporation into clinical practice. From the University of California, San Francisco, San Francisco, California; Mayo Clinic, Rochester, Minnesota; and Morgagni-Pierantoni Hospital, Forli, Italy. Acknowledgment: The authors acknowledge the providers and staff of the University of California, San Francisco, Interstitial Lung Disease Program and Consortium, Mayo Clinic, and Morgagni-Pierantoni Hospital for their assistance in recruiting patients for this study, and the patients with IPF who, through their generosity and efforts, allow us to conduct clinical research studies in an effort to improve the lives of patients with IPF. Grant Support: By the University of California, San Francisco Clinical and Translational Science Institute Resident Research Award and the National Institutes of Health (HL86516). Potential Conflicts of Interest: Disclosures can be viewed at M Reproducible Research Statement: Study protocol: Not available. Data set and statistical code: Available from Dr. Ley ( , brett.ley@ucsf.edu). Requests for Single Reprints: Brett Ley, MD, 55 Parnassus Avenue, Box 111, San Francisco, CA, 94143; , brett.ley@ucsf.edu. Current author addresses and author contributions are available at References 1. Raghu G, Collard HR, Egan JJ, Martinez FJ, Behr J, Brown KK, et al; ATS/ERS/JRS/ALAT Committee on Idiopathic Pulmonary Fibrosis. An official ATS/ERS/JRS/ALAT statement: idiopathic pulmonary fibrosis: evidence-based guidelines for diagnosis and management. Am J Respir Crit Care Med. 211;183: [PMID: ] 2. Bjoraker JA, Ryu JH, Edwin MK, Myers JL, Tazelaar HD, Schroeder DR, et May 212 Annals of Internal Medicine Volume 156 Number 1

8 Index and Staging System for IPF Original Research al. Prognostic significance of histopathologic subsets in idiopathic pulmonary fibrosis. Am J Respir Crit Care Med. 1998;157: [PMID: 94453] 3. Fernández Pérez ER, Daniels CE, Schroeder DR, St Sauver J, Hartman TE, Bartholmai BJ, et al. Incidence, prevalence, and clinical course of idiopathic pulmonary fibrosis: a population-based study. Chest. 21;137: [PMID: ] 4. Kim DS, Collard HR, King TE Jr. Classification and natural history of the idiopathic interstitial pneumonias. Proc Am Thorac Soc. 26;3: [PMID: ] 5. Martinez FJ, Safrin S, Weycker D, Starko KM, Bradford WZ, King TE Jr, et al; IPF Study Group. The clinical course of patients with idiopathic pulmonary fibrosis. Ann Intern Med. 25;142: [PMID: ] 6. National Heart, Lung, and Blood Institute. National Asthma Education and Prevention Program. Expert Panel Report 3: Guidelines for the Diagnosis and Management of Asthma. Full Report 27. NIH Publication no Bethesda, MD: National Institutes of Health; 27. Accessed at on 2 February Global Strategy for the Diagnosis, Management and Prevention of COPD. Global Initiative for Chronic Obstructive Lung Disease (GOLD) 28. Accessed at on 2 February Goldstraw P, Crowley J, Chansky K, Giroux DJ, Groome PA, Rami-Porta R, et al; International Association for the Study of Lung Cancer International Staging Committee. The IASLC Lung Cancer Staging Project: proposals for the revision of the TNM stage groupings in the forthcoming (seventh) edition of the TNM Classification of malignant tumours. J Thorac Oncol. 27;2: [PMID: ] 9. Ley B, Collard HR, King TE Jr. Clinical course and prediction of survival in idiopathic pulmonary fibrosis. Am J Respir Crit Care Med. 211;183: [PMID: ] 1. Erbes R, Schaberg T, Loddenkemper R. Lung function tests in patients with idiopathic pulmonary fibrosis. Are they helpful for predicting outcome? Chest. 1997;111:51-7. [PMID: ] 11. Lederer DJ, Arcasoy SM, Wilt JS, D Ovidio F, Sonett JR, Kawut SM. Six-minute-walk distance predicts waiting list survival in idiopathic pulmonary fibrosis. Am J Respir Crit Care Med. 26;174: [PMID: ] 12. King TE Jr, Tooze JA, Schwarz MI, Brown KR, Cherniack RM. Predicting survival in idiopathic pulmonary fibrosis: scoring system and survival model. Am J Respir Crit Care Med. 21;164: [PMID: ] 13. Wells AU, Desai SR, Rubens MB, Goh NS, Cramer D, Nicholson AG, et al. Idiopathic pulmonary fibrosis: a composite physiologic index derived from disease extent observed by computed tomography. Am J Respir Crit Care Med. 23;167: [PMID: ] 14. du Bois RM, Weycker D, Albera C, Bradford WZ, Costabel U, Kartashov A, et al. Ascertainment of individual risk of mortality for patients with idiopathic pulmonary fibrosis. Am J Respir Crit Care Med. 211;184: [PMID: ] 15. American Thoracic Society. Idiopathic pulmonary fibrosis: diagnosis and treatment. International consensus statement. American Thoracic Society (ATS), and the European Respiratory Society (ERS). Am J Respir Crit Care Med. 2; 161: [PMID: ] 16. Macintyre N, Crapo RO, Viegi G, Johnson DC, van der Grinten CP, Brusasco V, et al. Standardisation of the single-breath determination of carbon monoxide uptake in the lung. Eur Respir J. 25;26: [PMID: ] 17. Miller MR, Hankinson J, Brusasco V, Burgos F, Casaburi R, Coates A, et al; ATS/ERS Task Force. Standardisation of spirometry. Eur Respir J. 25;26: [PMID: ] 18. Wanger J, Clausen JL, Coates A, Pedersen OF, Brusasco V, Burgos F, et al. Standardisation of the measurement of lung volumes. Eur Respir J. 25;26: [PMID: ] 19. Fine JP, Gray RJ. A proportional hazards model for the subdistribution of a competing risk. J Am Stat Assoc. 1999;94: Harrell FE Jr, Lee KL, Mark DB. Multivariable prognostic models: issues in developing models, evaluating assumptions and adequacy, and measuring and reducing errors. Stat Med. 1996;15: [PMID: ] 21. Wolbers M, Koller MT, Witteman JC, Steyerberg EW. Prognostic models with competing risks: methods and application to coronary risk prediction. Epidemiology. 29;2: [PMID: ] 22. Celli BR, Cote CG, Marin JM, Casanova C, Montes de Oca M, Mendez RA, et al. The body-mass index, airflow obstruction, dyspnea, and exercise capacity index in chronic obstructive pulmonary disease. N Engl J Med. 24;35: [PMID: ] 23. Fang MC, Go AS, Chang Y, Borowsky L, Pomernacki NK, Singer DE; ATRIA Study Group. Comparison of risk stratification schemes to predict thromboembolism in people with nonvalvular atrial fibrillation. J Am Coll Cardiol. 28;51:81-5. [PMID: ] 24. Wilson PW, D Agostino RB, Levy D, Belanger AM, Silbershatz H, Kannel WB. Prediction of coronary heart disease using risk factor categories. Circulation. 1998;97: [PMID: ] 25. Thabut G, Christie JD, Ravaud P, Castier Y, Dauriat G, Jebrak G, et al. Survival after bilateral versus single-lung transplantation for idiopathic pulmonary fibrosis. Ann Intern Med. 29;151: [PMID: ] EASY SLIDES Download tables and figures as PowerPoint slides at May 212 Annals of Internal Medicine Volume 156 Number 1 691

9 Annals of Internal Medicine Current Author Addresses: Drs. Ley, Lee, and Collard: Department of Medicine, University of California, San Francisco, 55 Parnassus Avenue, Box 111, San Francisco, CA Dr. Ryerson: St. Paul s Hospital, 181 Burrard Street, Ward 8B, Vancouver, British Columbia, Canada V6Z 1Y6. Dr. Vittinghoff: Department of Epidemiology and Biostatistics, University of California, San Francisco, 185 Berry Street, Box 56, San Francisco, CA Dr. Ryu: Division of Pulmonary and Critical Care Medicine, Gonda 18 South, Mayo Clinic, 2 1st Street, Southwest, Rochester, MN Drs. Tomassetti and Poletti and Mr. Buccioli: Department of Diseases of the Thorax, Pulmonology Unit, Morgagni-Pierantoni Hospital, via C. Forlanini 34, 471 Forli, Italy. Dr. Elicker: Department of Radiology, University of California, San Francisco, 55 Parnassus Avenue, Box 628, San Francisco, CA Dr. Jones: Department of Pathology, University of California, San Francisco, 55 Parnassus Avenue, Box 628, San Francisco, CA Dr. King: Department of Medicine, University of California, San Francisco, 55 Parnassus Avenue, Box 12, San Francisco, CA Author Contributions: Conception and design: B. Ley, J.H. Ryu, B.M. Elicker, T.E. King, H.R. Collard. Analysis and interpretation of the data: B. Ley, C.J. Ryerson, E. Vittinghoff, J.H. Ryu, V. Poletti, M. Buccioli, B.M. Elicker, T.E. King, H.R. Collard. Drafting of the article: B. Ley, C.J. Ryerson, E. Vittinghoff, J.H. Ryu, S. Tomassetti, V. Poletti, M. Buccioli, T.E. King, H.R. Collard. Critical revision of the article for important intellectual content: B. Ley, C.J. Ryerson, E. Vittinghoff, J.H. Ryu, J.S. Lee, V. Poletti, B.M. Elicker, K.D. Jones, T.E. King, H.R. Collard. Final approval of the article: B. Ley, C.J. Ryerson, E. Vittinghoff, J.H. Ryu, S. Tomassetti, J.S. Lee, V. Poletti, B.M. Elicker, K.D. Jones, T.E. King, H.R. Collard. Provision of study materials or patients: J.H. Ryu, S. Tomassetti, H.R. Collard. Statistical expertise: E. Vittinghoff, M. Buccioli, H.R. Collard. Obtaining of funding: B. Ley, T.E. King, H.R. Collard. Administrative, technical, or logistic support: B. Ley. Collection and assembly of data: B. Ley, C.J. Ryerson, J.H. Ryu, S. Tomassetti, J.S. Lee, V. Poletti, M. Buccioli, B.M. Elicker, K.D. Jones, H.R. Collard. 26. Greenland S, Finkle WD. A critical look at methods for handling missing covariates in epidemiologic regression analyses. Am J Epidemiol. 1995;142: [PMID: 75345] 27. Schoop R, Beyersmann J, Schumacher M, Binder H. Quantifying the predictive accuracy of time-to-event models in the presence of competing risks. Biom J. 211;53: [PMID: ] 28. Heagerty PJ, Zheng Y. Survival model predictive accuracy and ROC curves. Biometrics. 25;61: [PMID: ] APPENDIX: EXPANDED STATISTICAL METHODS Testing of DLCO was unavailable for 12 patients (5.3%) in the derivation cohort and 32 (9.7%) in the validation cohort, primarily because patients with very poor lung function could not do the test (defined as when the test is ordered and attempted, but the patient cannot complete it because of respiratory limitations). To accommodate this feature of the data, we used 2 variables: First, an indicator for those who could not do the test, and second, 1 DLCO (in percent); the second variable was set to for patients who could not do the test, so that their outcomes did not affect the estimates for the association of measured DLCO with mortality. In contrast to problematic cases where the true value is simply unobserved, opening the door for residual confounding when a missing value category is used; in this case, patients with missing DLCO values are a distinct group with very high mortality risk (26). The prediction models and staging system were based on Fine Gray models for survival, treating transplantation as a competing risk (19). To select a model with good discrimination and calibration without overfitting, we screened candidate models with 1 to 8 predictors by using 2 repetitions of 1-fold crossvalidation of the Harrell c-index, a measure of discrimination (2). To estimate the c-index in the presence of competing risks, we used the approach of Wolbers and colleagues (21). We also cross-validated the Brier score, a global measure of prediction accuracy adapted for the competing-risks model (27). Based on exploratory analysis, age effects were modeled by using a 3-knot restricted cubic spline. Oxygen use was removed from consideration because it had substantially different effects in the derivation and validation cohorts, calling into question its generalizability to new patients. Dyspnea was omitted because it was unavailable in the validation cohort. The selected GAP model had cross-validated Brier score and c-index near the maximum; omitting sex or adding tobacco use improved 1 or both measures trivially. Once the continuous model had been selected, we divided each continuous predictor into clinically meaningful categories, and then fit a Fine Gray competing risks model for survival by using the resulting set of categorical predictors. Next, for ease of scoring, 1 to 3 points were assigned to each category. Point scores were obtained by rescaling then rounding the regression coefficients estimated by the categorical model. We then fit a Fine Gray model with the total point score for each patient as a continuous predictor. We checked for violation of the log-linearity assumption by treating the point score as a 3- or 4-knot restricted cubic spline (both P values for departure from linearity were.15). Finally, a staging system was created by dividing point scores into 3 groups comprising the 4% with the lowest estimated risk (stage I), the next 4% with intermediate risk (stage II), and the 2% with the highest risk (stage III). This categorization corresponded to 1-year mortality risks of less than 1%, 1% to 3%, and greater than 3%, respectively, motivated by clinical considerations previously discussed. For both models, we obtained optimism-corrected estimates of the c-index, first internally by using 2 repetitions of 1-fold cross-validation within the derivation cohort, and then externally by using coefficient estimates from the derivation cohort to calculate the linear predictor for patients in the validation cohort. Under the proportional hazards assumption, the c-index depends only on the linear predictor (28). We checked this assumption by assessing interaction between the linear predictor and log time. Although evidence was found for long-term violations, the assumption holds approximately (P.1 for both models) for the first 3 years of follow-up, where 79% of deaths occurred and where our survival estimates are focused. The c-index was essentially unchanged if deaths more than 3 years after baseline were W May 212 Annals of Internal Medicine Volume 156 Number 1

10 censored. Bootstrap resampling with 5 repetitions was used to calculate 95% bias-corrected bootstrap CIs for the c-index. In addition, calibration was assessed by comparing nonparametric estimates of cumulative mortality incidence at 1, 2, and 3 years, treating transplantation as a competing risk, with predictions based on the Fine Gray continuous and point-score models, under the assumption of proportional subdistribution hazards. Calibration was by stage for both models. For the continuous model, we further subdivided stage II into 3 groups of equal numbers (stage IIa, IIb, and IIc). The procedure was first done internally in the derivation cohort and then externally in the validation cohort by using predictions based on the derivation cohort. To improve calibration, we used shrinkage of the linear predictor toward the center, based on cross-validation (2). We also conducted sensitivity analyses in which model variables were estimated with censoring of deaths that occurred after the first 3 years; predicted mortality and calibration were essentially unchanged. The continuous model was then updated by using coefficients and cumulative subdistribution hazards based on the combined cohort to produce the most generalizable estimates for clinical use. Finally, we assessed the predictive accuracy of the continuous and point-score models for patients later in the course of disease. Specifically, both models were used to estimate subsequent mortality risk among surviving patients, based on predictors assessed 6, 12, 18, and 24 months ( 3 months) after baseline for the primary analysis. We also pooled the resulting follow-up data sets. Model performance was assessed by using the same measures of discrimination and calibration used for the primary analyses; to account for clustering in the pooled analysis, we did bootstrap resampling for the c-index CI by patient May 212 Annals of Internal Medicine Volume 156 Number 1 W-237

11 Appendix Figure. Calculating mortality risks by using the GAP calculator. Step 1: Calculate S S = [.337 (GENDER).15 (FVC ) +.92 (AGE ).52 (AGE2) (DLCO1) +.24 (DLCO2)] x.99 Where: 1) GENDER = a..293 if the patient is male b..77 if the patient is female 2) FVC = forced vital capacity, % predicted 3) AGE1 = patient s age, y 4) AGE2 = refer to the table below. Enter the value of AGE2 that corresponds to the patient s age. AGE AGE2 AGE AGE2 AGE AGE2 AGE AGE ) DLCO1 = a..921 if the patient could not do the DLCO test b..79 if the patient could do the DLCO test 6) DLCO2 = a if the patient could not do the DLCO test b. ( the patient s DLCO) if the patient could do the test Step 2: Calculate risk using S: 1-y risk = 1 x [1 exp( exp (S) x.225)] 2-y risk = 1 x [1 exp( exp (S) x.486)] 3-y risk = 1 x [1 exp( exp (S) x.768)] Example: 6-year-old man with an FVC of 65% predicted and a DLCO of 4% predicted Step 1: Calculate S S = [.337 (.293).15 ( ) +.92 ( ).52 (.236) (.921) +.24 ( )] x.99 =.471 Step 2: Calculate risk using S 1-y risk = 1 x [1 exp( exp (.471) x.225)] = 13.1% 2-y risk = 1 x [1 exp( exp (.471) x.486)] = 26.3% 3-y risk = 1 x [1 exp( exp (.471) x.768)] = 38.1% Note: Age is modeled by using a 3-knot restricted cubic spline function. For simplicity, we represent this by using 2 age variables (AGE1 and AGE2) in the figure, utilizing a value look-up table for AGE2. AGE1 and AGE2 age variables after 3-knot restricted cubic spline (knots 12.68,.32, and 1.32, after centering age on its mean (67.68 y); GAP gender, age, and 2 lung physiology variables (FVC and DLCO). W May 212 Annals of Internal Medicine Volume 156 Number 1

12 Appendix Table 1. The Continuous Model Based on the Derivation Cohort Appendix Table 2. The Continuous Model Based on the Combined Cohort Predictor Hazard Ratio (95% CI) P Value Male sex 1.66 ( ).72 Age AGE ( ).1 AGE2.91 (.83.98).2 Physiology FVC, % predicted.98 (.97.99).1 Inverse DLCO, % predicted 1.3 ( ).2 Cannot do DLCO, % 17.5 ( ).1 AGE1 and AGE2 age variables after 3-knot restricted cubic spline (knots of 12.68,.32, and 1.32), after centering age on its mean (67.68 y). Predictor Hazard Ratio (95% CI) P Value Male sex 1.4 ( ).28 Age AGE1 1.1 ( ).1 AGE2.95 (.91.99).17 Physiology FVC, % predicted.99 (.98.99).1 Inverse DLCO, % predicted 1.2 ( ).1 Cannot do DLCO, % 9.37 ( ).1 AGE1 and AGE2 age variables after 3-knot restricted cubic spline (knots of 12.68,.32, and 1.32), after centering age on its mean (67.68 y). Appendix Table 3. Performance of the GAP Calculator and GAP Index in Follow-up* Variable 6 mo (n 317) 12 mo (n 243) 18 mo (n 232) 24 mo (n 182) Pooled (n 974) Predicted Observed Predicted Observed Predicted Observed Predicted Observed Predicted Observed GAP calculator C-index (95% CI) 72.9 ( ) 72.1 ( ) 73. ( ) 69.1 ( ) 71.9 ( ) 2-y mortality, % Stage I Stage IIa Stage IIb Stage IIc Stage III GAP index C-index (95% CI) 73.8 ( ) 74.4 ( ) 71.7 ( ) 68. ( ) 72.3 ( ) 2-y mortality, % Stage I Stage II Stage III GAP gender, age, and 2 lung physiology variables (FVC and DLCO). * Each time point represents data at follow-up after the initial clinic visit 3 mo. The pooled cohort represents all unique patient visits (n 974) in the follow-up analysis May 212 Annals of Internal Medicine Volume 156 Number 1 W-239