Prospective Derivation and Validation of a Clinical Prediction Rule for Recurrent Clostridium difficile Infection

GASTROENTEROLOGY 2009;136:1206 1214 Prospective Derivation and Validation of a Clinical Prediction Rule for Recurrent Clostridium difficile Infection MARY Y. HU,* KIANOOSH KATCHAR,* LORRAINE KYNE, SEEMA MAROO,* SANJEEV TUMMALA,* VALLEY DREISBACH,* HUA XU,* DANIEL A. LEFFLER,* and CIARÁN P. KELLY* *Division of Gastroenterology, Department of Medicine, Beth Israel Deaconess Medical Center, Harvard Medical School, Boston, Massachusetts; Department of Medicine for the Older Person, Mater Misericordiae University Hospital, University College Dublin, Dublin, Ireland Podcast interview: www.gastro.org/gastropodcast; see editorial on page 1152. Background & Aims: Prevention of recurrent Clostridium difficile infection (CDI) is a substantial therapeutic challenge. A previous prospective study of 63 patients with CDI identified risk factors associated with recurrence. This study aimed to develop a prediction rule for recurrent CDI using the above derivation cohort and prospectively evaluate the performance of this rule in an independent validation cohort. Methods: The clinical prediction rule was developed by multivariate logistic regression analysis and included the following variables: age >65 years, severe or fulminant illness (by the Horn index), and additional antibiotic use after CDI therapy. A second rule combined data on serum concentrations of immunoglobulin G (IgG) against toxin A with the clinical predictors. Both rules were then evaluated prospectively in an independent cohort of 89 patients with CDI. Results: The clinical prediction rule discriminated between patients with and without recurrent CDI, with an area under the curve of the receiveroperating-characteristic curve of 0.83 (95% confidence interval [CI]: 0.70 0.95) in the derivation cohort and 0.80 (95% CI: 0.67 0.92) in the validation cohort. The rule correctly classified 77.3% (95% CI: 62.2% 88.5%) and 71.9% (95% CI: 59.2% 82.4%) of patients in the derivation and validation cohorts, respectively. The combined rule performed well in the derivation cohort but not in the validation cohort (area under the curve of the receiver-operating-characteristic curve, 0.89 vs 0.62; diagnostic accuracy, 93.8% vs 69.2%, respectively). Conclusions: We prospectively derived and validated a clinical prediction rule for recurrent CDI that is simple, reliable, and accurate and can be used to identify highrisk patients most likely to benefit from measures to prevent recurrence. Clostridium difficile was first reported to cause antibiotic-associated pseudomembranous colitis in 1978 and has since become the leading known cause of hospital-acquired infectious diarrhea in the developed world. 1 4 Despite advances in knowledge of the pathogenesis of C difficile infection (CDI), the organism continues to afflict millions of patients every year and is associated with increasing morbidity and mortality. Most patients with CDI respond well to medical therapy including withdrawal of the offending antibiotic and treatment with metronidazole or vancomycin. However, approximately 20% of individuals experience recurrence despite successful treatment of the initial episode, and the risk may be as high as 65% for those with a history of prior CDI. 1,5 13 Multiple recurrences are common, and patients suffering 10 or more episodes have been described. 1,7,12,13 Recurrent CDI typically occurs within 1 to 3 weeks after completion of treatment for the initial episode, but late recurrences up to 2 months are not infrequent. 1,7,9,11 13 Various strategies have been proposed for the management of recurrent CDI, including longer courses of metronidazole or vancomycin, tapered and pulsed schedules of vancomycin, toxin-binding agents such as cholestyramine, probiotics such as Saccharomyces boulardii, and immunotherapy. 1,5 9,12 24 Nevertheless, no approach is universally successful, and recurrent CDI remains a substantial therapeutic challenge. In a previous investigation, we prospectively studied a cohort of hospitalized patients with CDI at our institution and identified risk factors associated with recurrence. 25 The independent clinical predictors of recurrent CDI included age 65 years, severe or fulminant underlying illness as determined by the Horn index, and use of additional antibiotics after discontinuation of therapy for the initial CDI episode. Furthermore, we determined that high serum concentrations of immunoglobulin M (IgM) against toxin A on day 3 after the onset of the initial CDI episode or high concentrations of immunoglobulin G (IgG) against toxin A on day 12 were associated with a reduced risk of recurrence. Other investiga- Abbreviations used in this paper: AUC, area under the curve; CDI, Clostridium difficile infection; NPV, negative predictive value; PPV, positive predictive value; ROC, receiver-operating-characteristic curve. 2009 by the AGA Institute 0016-5085/09/$36.00 doi:10.1053/j.gastro.2008.12.038

April 2009 PREDICTION RULE FOR C difficile RECURRENCE 1207 tors similarly reported old age and exposure to new antibiotics as negative prognostic factors for recurrent CDI. 12,13,26,27 In addition, several studies found that inadequate host immune response to C difficile toxins predisposed patients to recurrence. 20,28 30 In the current study, we first developed a clinical prediction rule for recurrent CDI based on data from the previous cohort (derivation cohort). We then prospectively evaluated the performance of this rule in an independent cohort of hospitalized patients with CDI at our institution (validation cohort). Results from this study may help to identify a subset of patients at high risk for recurrent CDI who may benefit from early preventive and therapeutic interventions. Materials and Methods Patient Populations and Definitions In the previous investigation, we prospectively studied 63 hospitalized patients with CDI at Beth Israel Deaconess Medical Center, Boston, Massachusetts, between January and May 1998. 25 Data from this cohort were used to develop a clinical prediction rule for recurrent CDI (derivation cohort). In the current study, an independent cohort of patients was investigated under a protocol almost identical to that used in the previous study. All adult patients with CDI hospitalized at Beth Israel Deaconess Medical Center between December 2004 and May 2006 were eligible for study entry. Data from this second cohort were used to evaluate the performance of the prediction rule (validation cohort). The study was approved by the Institutional Review Board. Informed consent was obtained from all patients or their health care proxies. Diarrhea was defined as a change in bowel habit with 3 or more unformed bowel movements a day for at least 2 days. CDI was defined as diarrhea coupled with a positive stool C difficile toxin assay and not attributed to any other cause. The main outcome of interest was recurrent CDI, defined as a new episode of diarrhea confirmed by a positive stool C difficile toxin assay, after resolution of the initial CDI episode for at least 2 days and after discontinuation of therapy with metronidazole or vancomycin. Data Collection Cases of CDI were identified by daily review of stool C difficile toxin assays performed at the clinical microbiology laboratory. Potential subjects were assessed within 1 day of a positive assay result. Demographic data including age, gender, and race were recorded. Other baseline characteristics, such as residence prior to admission, immunosuppression (congenital immunodeficiency, HIV, or AIDS; malignancy; organ transplant; use of corticosteroids, chemotherapy agents, or other immunosuppressive medications), comorbid conditions (determined by the Charlson comorbidity score), seriousness of underlying illness (determined by the modified Horn disease severity index), and history of prior CDI, were documented. 25,31 34 The Horn index rates disease severity as 1 of 4 categories on the basis of clinical judgment: mild (single mild illness), moderate (more severe illness but uncomplicated recovery expected), severe (major complications or multiple conditions requiring treatment), and fulminant (catastrophic life-threatening illness). 34 Patients were followed for a total of 60 days. Additional clinical information, such as presence of CDI on admission, severity and duration of CDI, therapy for CDI, intensive care unit stay, length of hospital stay, and additional antibiotic use after discontinuation of CDI therapy, were collected. Resolution of CDI was monitored on a daily basis until discharge. Patients were asked to keep a stool diary if they were discharged before the end of the study. For patients discharged to other health care facilities, the nursing staff at those facilities was asked to keep the diary. Patients were interviewed at 30 and 60 days of follow-up. Laboratory Studies Serum samples for measurement of antibody response to C difficile toxins were obtained during hospitalization and follow-up. All laboratory studies were performed on coded specimens by investigators blinded to patient data. C difficile toxins were purified from the culture supernatant of strain VPI 10463 (American Type Culture Collection 43255). Concentrations of antibodies against C difficile toxins were measured by enzyme-linked immunosorbent assay (ELISA) as previously described. 25,31 For the derivation cohort, convalescent sera from CDI patients with high antitoxin antibody titers were pooled and used as a standard. 25 For the validation cohort, the original standard was exhausted, and a new standard was prepared again using high-titer sera from CDI patients. The new standard was calibrated by comparing antibody concentrations in a group of control subjects with data obtained in a similar control group using the original standard (under the assumption that antibody concentrations in control populations had remained constant). Both control groups consisted of contemporaneous hospitalized patients without CDI. Antibody concentrations of patients in the derivation cohort were divided at the 75th percentile (equivalent to 1.29 ELISA units) on the basis of the distribution of antibody concentrations at the onset of CDI. 25 Clinical Prediction Rule and Statistical Analyses To develop a clinical prediction rule for recurrent CDI, we built a multivariable logistic regression model in the derivation cohort using the following variables:

1208 HU ET AL GASTROENTEROLOGY Vol. 136, No. 4 age 65 years, Horn s index severe or fulminant, and additional antibiotic use after discontinuation of CDI therapy (clinical rule). To explore any potential added value of antibody response, a second rule including day 12 serum antitoxin A IgG level 75th percentile (1.29 ELISA units) was also constructed (combined rule). Day 12 antitoxin A IgG was chosen over day 3 antitoxin A IgM because of its stronger association with recurrent CDI as described in the previous study. 25 Missing values of variables were dummy coded. Each predictor in the model was assigned an integer point proportional to its coefficient. Risk scores for patients were calculated by summing the points for all predictors present. Patients were divided into high- and low-risk groups based on their scores and associated predicted probability of recurrence, using a cut-off of 40%. The performance of the models was assessed by the receiver-operating-characteristic (ROC) curve and the area under the curve (AUC). 35 The sensitivity, specificity, positive predictive value (PPV), negative predictive value (NPV), and diagnostic accuracy of the rules were calculated. Both clinical and combined prediction rules were tested in the validation cohort. Because of difficulties inherent in collecting the day 12 serum samples for antibody measurement (most patients had been discharged), the time criterion in the validation cohort was relaxed to include all samples drawn on day 12 and at anytime during follow-up until day 60. The observed and predicted risks of recurrent CDI were calculated. The ROC curves for the models were compared with those from the derivation cohort. The sensitivity, specificity, PPV, NPV, and accuracy of the rules were also compared with those from the derivation cohort. All statistical analyses were performed using the SAS software system, version 9.1 (SAS Institute Inc, Carey, NC), except ROC curve analyses, which were performed using the SPSS software system, version 13.0 (SPSS Inc, Chicago, IL). Results Patient Characteristics As previously described, the derivation cohort contained 63 patients, of whom 19 (30%) died during follow-up and could not be assessed for recurrent CDI. Among the remaining 44 patients, 22 (50%) had recurrence. 25 In the validation cohort, 276 patients with CDI were identified, and 89 (32%) were enrolled. Among the 187 patients not enrolled, 34 were discharged or died before the initial assessment, 33 were incompetent and had no available health care proxies, and 120 declined to participate (49 declined because they were critically ill). Among the 89 patients enrolled, 18 (20%) could not be assessed for recurrence: 15 died during follow-up, and 3 were lost to follow-up. Another 7 (8%) were excluded because they could not be classified as either recurrence or no recurrence: 5 had recurrent diarrhea without documentation of C difficile toxin in the stool but were empirically treated with metronidazole or vancomycin; 2 had fever and leukocytosis without recurrent diarrhea in the setting of positive stool C difficile toxin assays and were treated with metronidazole. Of the remaining 64 patients included in analysis, 13 (20%) had recurrent CDI: 5 as outpatients and 8 as inpatients. Characteristics of patients in the derivation and validation cohorts are compared in Table 1. The 2 populations were similar with respect to age, gender, race, residence prior to admission, comorbidities (Charlson score), and proportion of patients with prior CDI. Compared with the derivation cohort, the validation cohort contained more immunosuppressed patients (16% vs 45%, respectively, P.001) but fewer patients with serious underlying illness as determined by the Horn index (50% vs 25%, respectively, P.008). There was no difference in the proportion of patients with CDI on admission, severity or duration of CDI, or additional antibiotic use after discontinuation of CDI therapy. In contrast to the derivation cohort, patients in the validation cohort were more likely to be treated with both metronidazole and vancomycin (0% vs 22%, respectively, P.0009) instead of metronidazole alone (95% vs 75%, respectively, P.005), and the duration of metronidazole treatment was in general longer (10 vs 14 days, respectively, P.0002). Most of these patients were given metronidazole as the initial therapy, and vancomycin was added or substituted because of inadequate response. Although more patients in the validation cohort required intensive care unit stay (9% vs 30%, respectively, P.01), the overall length of hospital stay was similar. In the derivation cohort, all serum samples for antibody measurement were drawn on day 12 after the onset of CDI; the timing of samples from patients in the validation cohort was more varied (median, day 17; range, days 12 59, P.0001). Patients in general had lower antitoxin A IgG levels in the validation cohort (0.97 vs 0.5 ELISA units, respectively, P.009), but the proportion with levels 75th percentile or 1.29 ELISA units was not different. Interestingly, when compared with the derivation cohort, the frequency of recurrent CDI in the validation cohort was significantly lower (50% vs 20%, respectively, P.001). Derivation of the Clinical Prediction Rule From the derivation cohort, we generated a clinical prediction model for recurrent CDI (clinical rule) as shown in Table 2. Each predictor age 65 years, Horn index severe or fulminant, and additional antibiotic use was assigned 1 point. The model discriminated patients with and without recurrence well with an AUC of the ROC curve of 0.83 (95% confidence interval [CI]: 0.70 0.95) as shown in Figure 1A. The model passed the Hosmer Lemeshow goodness-of-fit test (P.07). When sorted by risk scores, the clinical rule identified groups of patients with increasing probability of recur-

April 2009 PREDICTION RULE FOR C difficile RECURRENCE 1209 Table 1. Characteristics of Patients in the Derivation and Validation Cohorts Variable Derivation cohort (n 44) Validation cohort (n 64) P value Age, y; median (range) 69, 19 97 69, 22 94.45 65,n(%) 25 (57) 37 (58).92 Male sex, n (%) 17 (39) 27 (42).71 White race, n (%) 36 (84) (n 43) 51 (80).60 Residence before admission, n (%) Home 28 (64) 40 (63).90 Nursing home or rehabilitation center 14 (32) 14 (22).25 Other hospital 2 (5) 10 (16).12 Immunosuppression, n (%) 7 (16) 29 (45).001 Charlson score, median (range) 3, 0 12 3, 0 10.65 Horn index severe or fulminant, n (%) 22 (50) 16 (25).008 Prior CDI, n (%) 3 (7) 10 (16).17 CDI on admission, n (%) 21 (48) 32 (50).82 Severe CDI, n (%) 15 (34) 21 (33).89 CDI therapy, n (%) Supportive 2 (5) 0.16 Metronidazole 42 (95) 48 (75).005 Vancomycin 0 2 (3).51 Metronidazole and vancomycin 0 14 (22).0009 Duration of therapy (day), median (range) Metronidazole 10, 8 22 14, 4 42.0002 Vancomycin 17, 12 40 Duration of CDI (day), median (range) 9, 1 29 9, 2 56.44 After initiation of therapy 7, 0 27 4, 1 56.80 ICU stay, n (%) 4 (9) 19 (30).01 LOS (day), median (range) 11, 1 55 8, 1 69.27 Additional antibiotic use, n (%) 20 (45) 24 (38).41 Time of antitoxin A IgG (day), median (range) 12 (n 16) 17, 12 59 (n 26).0001 Antitoxin A IgG (ELISA unit), median (range) 0.97, 0.35 23.53 (n 16) 0.5, 0.043 6.04 (n 26).009 1.29, n (%) 9 (56) 20 (77).19 Recurrent CDI, n (%) 22 (50) 13 (20).001 ICU, intensive care unit; LOS, length of hospital stay. rent CDI as shown in Table 3. For example, a patient with no risk factor would have a score of 0 with 0 probability of developing recurrence. On the other hand, a patient with all 3 risk factors would have a score of 3 and 87.5% probability of developing recurrence. Using the prespecified cut-off of 40%, patients were classified as high risk if their score was 2 and low risk if their score was 2. As shown in Table 4, the 44 patients in the derivation cohort were equally distributed between the 2 risk groups. Of the 22 patients in the high-risk group, 17 had recurrence (77.3%). Conversely, of the 22 patients in the low-risk group, 5 had recurrence (22.7%). The sensitivity, specificity, PPV, and NPV of the rule were identical at Table 2. Predictors of Recurrent C difficile Infection in the Derivation Cohort Variable coefficient OR 95% CI Points Age 65 y 1.81 6.12 1.03 36.58 1 Horn index severe or 2.26 9.56 1.19 76.68 1 fulminant Additional antibiotic use 2.31 10.03 1.47 68.26 1 Antitoxin A IgG 1.29 3.96 52.54 1.49 1000 2 NOTE. The clinical prediction model excluded antitoxin A IgG. 77.3% (95% CI: 54.6% 92.2%). The diagnostic accuracy was also 77.3% (95% CI: 62.2% 88.5%). Derivation of the Combined Prediction Rule The combined prediction model for recurrent CDI (combined rule) was generated by incorporation of antibody response with the other clinical predictors in the derivation cohort. Each clinical predictor was assigned 1 point as above; serum antitoxin A IgG level 1.29 ELISA units was assigned 2 points (Table 2). The discriminative ability of the model was demonstrated by the ROC curve in Figure 1B with an AUC of 0.89 (95% CI: 0.80 0.99). The Hosmer Lemeshow goodness-of-fit test indicated that the model was well calibrated (P.49). Patients were in general grouped by increasing probability of recurrent CDI based on calculated risk scores (Table 5). A patient without any risk factor would have a score of 0 and 0 probability of recurrence, whereas a patient with all 4 risk factors would have a score of 5 and 100% probability of recurrence. Using the 40% cut-off, a score of 4 was considered high risk, and a score of 4 was considered low risk. A total of 16 patients in the derivation cohort had available antitoxin A IgG data, 8 in each risk group (Table 6).

1210 HU ET AL GASTROENTEROLOGY Vol. 136, No. 4 47.4% 99.7%), respectively. The rule predicted recurrent CDI with an accuracy of 93.8% (95% CI: 69.8% 99.8%). Figure 1. (A) Receiver operating characteristic curves for the clinical prediction model of recurrent C difficile infection. The area under the curve for the derivation cohort was 0.83 (95% CI: 0.70 0.95), and the area-under-the-curve for the validation cohort was 0.80 (95% CI: 0.67 0.92). The dashed line represents a test of no discrimination. (B) Receiver operating characteristic curves for the combined prediction model of recurrent C difficile infection. The area under the curve for the derivation cohort was 0.89 (95% CI: 0.80 0.99), and the area under the curve for the validation cohort was 0.62 (95% CI: 0.44 0.80). The dashed line represents a test of no discrimination. All 8 patients in the high-risk group had recurrence (100%), whereas only 1 of the 8 patients in the low-risk group had recurrence (12.5%). The sensitivity and specificity of the rule were 88.9% (95% CI: 51.8% 99.7%) and 100% (95% CI: 59.0% 100%), respectively; the PPV and NPV were 100% (95% CI: 63.1% 100%) and 87.5% (95% CI: Validation of the Prediction Rules When applied to the validation cohort, the clinical rule successfully stratified the 64 patients according to their risk of recurrent CDI as shown in Table 3. As shown in Table 4, using the calculated score of 2 for high risk, recurrence occurred in 7 out of 19 patients in the highrisk group (36.8%) and 6 out of 45 patients in the lowrisk group (13.3%). The sensitivity and specificity of the rule were 53.8% (95% CI: 25.1% 80.8%) and 76.5% (95% CI: 62.5% 87.2%), respectively, with PPV and NPV of 36.8% (95% CI: 16.3% 61.6%) and 86.7% (95% CI: 73.2% 95.0%), respectively. The diagnostic accuracy was 71.9% (95% CI: 59.2% 82.4%). The ROC curve for the model yielded an AUC of 0.80 (95% CI: 0.67 0.92) as shown in Figure 1A. The combined rule also roughly stratified the 26 patients with available antitoxin A IgG data in the validation cohort by their risk of recurrent CDI as shown in Table 5. When divided by risk groups (score 4 for high-risk), recurrence occurred in 3 out of 6 patients in the high-risk group (50%) and 5 out of 20 patients in the low-risk group (25%) as shown in Table 6. The sensitivity and specificity were 37.5% (95% CI: 8.5% 75.5%) and 83.3% (95% CI: 58.6% 96.4%), respectively; the PPV and NPV were 50% (95% CI: 11.8% 88.2%) and 75% (95% CI: 50.9% 91.3%), respectively. The rule predicted recurrent CDI with an accuracy of 69.2% (95% CI: 48.2% 85.7%). The AUC of the ROC curve for the model was 0.62 (95% CI: 0.44 0.80) as shown in Figure 1B. Comparison of the Prediction Rules Although the combined prediction model for recurrent CDI demonstrated a high discriminative ability in the derivation cohort, its performance was less ideal in the validation cohort (AUC, 0.89 vs 0.62, respectively) as shown in Figure 1B. Accordingly, the diagnostic accuracy of the prediction rule based on the model also decreased (93.8% vs 69.2%, respectively) as shown in Table 6.Onthe other hand, the performance of the clinical prediction Table 3. Risks of Recurrent C difficile Infection by the Clinical Prediction Rule Score Predicted (derivation cohort) (n 44) Recurrence Observed (validation cohort) (n 64) n % n % 0 0/7 0 0/9 0 1 5/15 33.3 6/36 16.7 2 10/14 71.4 5/16 31.3 3 7/8 87.5 2/3 66.7 NOTE. Patients with scores 2 were classified as high risk.

April 2009 PREDICTION RULE FOR C difficile RECURRENCE 1211 Table 4. Performance of the Clinical Prediction Rule for Recurrent C difficile Infection Derivation cohort (n 44) Validation cohort (n 64) Recurrence No recurrence Recurrence No recurrence High risk, n 17 5 7 12 Low risk, n 5 17 6 39 Sensitivity 77.3% 95% CI: 54.6 92.2 53.8% 95% CI: 25.1 80.8 Specificity 77.3% 95% CI: 54.6 92.2 76.5% 95% CI: 62.5 87.2 PPV 77.3% 95% CI: 54.6 92.2 36.8% 95% CI: 16.3 61.6 NPV 77.3% 95% CI: 54.6 92.2 86.7% 95% CI: 73.2 95.0 Accuracy 77.3% 95% CI: 62.2 88.5 71.9% 95% CI: 59.2 82.4 model was nearly identical in the derivation and validation cohorts (AUC, 0.83 vs 0.80, respectively) as shown in Figure 1A. Likewise, the diagnostic accuracy of the prediction rule displayed only slight degradation when tested in the validation cohort (77.3% vs 71.9%, respectively) as shown in Table 4. Discussion Recurrence is a common and challenging problem in CDI, affecting 20% of patients treated with metronidazole or vancomycin and in some instances causing multiple episodes over many months. Previously, we identified clinical and host immune risk factors associated with recurrence including age 65 years, severe or fulminant underlying illness as determined by the Horn index, additional antibiotic use after discontinuation of metronidazole or vancomycin therapy for the initial CDI episode, and low serum antitoxin A IgG concentration ( 1.29 ELISA units) on day 12 after the onset of CDI. 25 In this study, we used the clinical risk factors to develop a prediction rule for recurrent CDI and evaluated the performance of this rule in an independent prospective cohort of patients. The clinical prediction model successfully discriminated between patients with and without recurrence with an AUC of the ROC curve of 0.83 (95% CI: 0.70 0.95) in the derivation cohort and 0.80 (95% CI: 0.67 0.92) in the validation cohort. When dichotomized according to patients with 2 vs 2 risk factors, the rule Table 5. Risks of Recurrent C difficile Infection by the Combined Prediction Rule Score Predicted (derivation cohort) (n 16) Recurrence Observed (validation cohort) (n 26) n % n % 0 0/1 0 0/1 0 1 0/2 0 1/3 33.3 2 1/3 33.3 1/4 25 3 0/2 0 3/12 25 4 3/3 100 1/3 33.3 5 5/5 100 2/3 66.7 NOTE. Patients with scores 4 were classified as high risk. correctly classified 77.3% (95% CI: 62.2% 88.5%) of patients in the derivation cohort and 71.9% (95% CI: 59.2% 82.4%) of patients in the validation cohort. An interesting finding in our study was the difference in recurrence rates in the derivation and validation cohorts. Recurrent CDI occurred in 20% (13/64) of patients in the validation cohort, an average rate cited by most studies historically. 1,5 13 However, this rate was far lower than the 50% (22/44) observed in the derivation cohort. 25 The main explanation for this discrepancy is a healthier study population in the validation cohort because many critically ill patients were not recruited. As a result, the validation cohort contained considerably fewer patients with serious underlying illness as determined by the Horn index, an important independent predictor for recurrent CDI. Another explanation for the lower rate in the validation cohort may be the exclusion of the 7 patients who perhaps had recurrence but did not meet the strict study definition of recurrence: 5 of them had recurrent diarrhea but without documentation of C difficile toxin in the stool, and the other 2 had positive stool C difficile toxin assays without recurrent diarrhea. Nonetheless, inclusion of these patients would only increase the recurrence rate to 28% (20/71). The 3 clinical risk factors included in our prediction rule for recurrent CDI were in keeping with findings of other investigators. Old age and exposure to new antibiotics have both been independently identified as predictors for recurrence. 12,13,26,27 The Horn index has not been previously associated with recurrence, but similar surrogate markers of poor underlying health status (such as chronic renal insufficiency) have been described. 36 Additional risk factors reported by other investigators include low quality of life, history of prior CDI, high white blood cell count during the initial CDI episode, onset of CDI in the spring, and short length of hospital stay after CDI. 12,13,26,36 However, these factors were not included in our study because they did not emerge as significant predictors in our analysis, were not reproduced widely, or were difficult to justify clinically. The association between host immune response to C difficile toxins and recurrent CDI has been demonstrated by a number of investigators including us. 20,25,28 30 In this study, we explored the role of low serum antitoxin A

1212 HU ET AL GASTROENTEROLOGY Vol. 136, No. 4 Table 6. Performance of the Combined Prediction Rule for Recurrent C difficile Infection Derivation cohort (n 16) Validation cohort (n 26) Recurrence No recurrence Recurrence No recurrence High risk, n 8 0 3 3 Low risk, n 1 7 5 15 Sensitivity 88.9% 95% CI: 51.8 99.7 37.5% 95% CI: 8.5 75.5 Specificity 100% 95% CI: 59.0 100 83.3% 95% CI: 58.6 96.4 PPV 100% 95% CI: 63.1 100 50% 95% CI: 11.8 88.2 NPV 87.5% 95% CI: 47.4 99.7 75% 95% CI: 50.9 91.3 Accuracy 93.8% 95% CI: 69.8 99.8 69.2% 95% CI: 48.2 85.7 IgG level during an initial episode of CDI in predicting recurrence in combination with the other clinical predictors. The combined rule performed well in the derivation cohort but deteriorated significantly in the validation cohort (AUC of the ROC curve, 0.89 vs 0.62; diagnostic accuracy 93.8% vs 69.2%, respectively). Several explanations exist to account for this disappointing result. First, the number of serum samples available for antibody measurement was small in both the derivation and validation cohorts, thereby limiting the stability of the model and its power to accurately predict recurrence. Second, the timing of antibody measurement differed considerably in the validation cohort, which may in turn have resulted in disparity in antibody concentrations measured, because antibody response has been shown to be dynamic during an episode of CDI. 25,31 In fact, serum antitoxin A IgG levels were substantially lower in the validation cohort compared with the derivation cohort. Third, a new antibody standard was used in the validation cohort, which may have further contributed to differences in antibody concentrations between the 2 cohorts. Last, the epidemiology of CDI has changed significantly from the time of the derivation cohort in 1998 to the time of the validation cohort in 2004 2006, especially in the setting of recent outbreaks in North America and Europe. 9,37 39 Whether this change in the epidemiology of CDI influenced the host immune response to C difficile is not entirely clear and may warrant further investigation. In contrast to the combined rule, the clinical rule performed consistently and well in both the derivation and validation cohorts (diagnostic accuracy, 77.3% and 71.9%, respectively). This result is encouraging, especially after considering the temporal and clinical differences between the 2 cohorts as discussed above. The omission of antitoxin antibody measurement and the use of accessible predictors with straightforward calculations also facilitate broader and logistically simpler bedside application of the rule. One limitation of our analysis was the relatively small number of recurrent CDI episodes in the validation cohort, which reduced the power of the study to evaluate the performance of the rule. In an exploratory analysis, we included the 7 patients with possible but unproven recurrence in the validation cohort and found no substantial difference in the performance of the rule (the sensitivity, specificity, PPV, NPV, and diagnostic accuracy were 45%, 76.5%, 42.9%, 78%, and 67.6%, respectively). Another potential limitation of the rule was the inclusion of a risk factor that required subjective assessment, ie, the Horn index. However, in our previous studies, we validated the use of the Horn index as a predictor for CDI and found that the interobserver reliability was excellent with coefficient of correlation 0.9 for all categories of disease severity. 25,31,40 Our rule was imperfect with respect to its modest sensitivity, 77.3% in the derivation cohort and only 53.8% in the validation cohort, indicating that between 22.7% and 46.2% of patients with recurrent CDI would have been missed. In an effort to reduce this miss rate, we refined the rule by shifting the cut-off for high-risk group from 2 risk factors to 1 risk factor. The result was an improvement of sensitivity to 100% in both the derivation and validation cohorts. As a trade-off, the specificity decreased from 77.3% to 31.8% in the derivation cohort and from 76.5% to 17.6% in the validation cohort. Nevertheless, this alternative application may be useful in circumstances in which high sensitivity is paramount. In summary, in this study, we prospectively derived and validated a clinical prediction rule for recurrent CDI that is simple, reliable, and accurate. This rule is valuable in clinical practice because it defines a high-risk population in whom awareness of the risk can facilitate more prompt recognition, diagnosis, and treatment of recurrent CDI. These patients are also most likely to benefit from interventions to prevent recurrence, such as infection control precautions, prudent use of antibiotics, prolongation of metronidazole or vancomycin therapy, and use of probiotics or other prophylactic measures. In addition, this rule will be of great value in selecting highrisk patients for clinical trials of novel agents to prevent recurrent CDI, eg, alternative antibiotics such as rifaximin, C difficile toxin-binding agents such as tolevamer, and passive or active immunizations against C difficile toxins. 20 24,41,42

April 2009 PREDICTION RULE FOR C difficile RECURRENCE 1213 References 1. Kelly CP, Pothoulakis C, LaMont JT. Clostridium difficile colitis. N Engl J Med 1994;330:257 262. 2. Kyne L, Farrell RJ, Kelly CP. Clostridium difficile. Gastroenterol Clin North Am 2001;30:753 777. 3. Bartlett JG, Chang TW, Gurwith M, et al. Antibiotic-associated pseudomembranous colitis due to toxin-producing clostridia. N Engl J Med 1978;298:531 534. 4. Larson HE, Price AB, Honour P, et al. Clostridium difficile and the aetiology of pseudomembranous colitis. Lancet 1978;1:1063 1066. 5. Fekety R. Guidelines for the diagnosis and management of Clostridium difficile-associated diarrhea and colitis. American College of Gastroenterology, Practice Parameters Committee. Am J Gastroenterol 1997;92:739 750. 6. Kyne L, Kelly CP. Recurrent Clostridium difficile diarrhoea. Gut 2001;49:152 153. 7. Maroo S, Lamont JT. Recurrent Clostridium difficile. Gastroenterology 2006;130:1311 1316. 8. Aslam S, Hamill RJ, Musher DM. Treatment of Clostridium difficile-associated disease: old therapies and new strategies. Lancet Infect Dis 2005;5:549 557. 9. Bartlett JG. Narrative review: the new epidemic of Clostridium difficile-associated enteric disease. Ann Intern Med 2006;145: 758 764. 10. Wilcox MH, Spencer RC. Clostridium difficile infection: responses, relapses and re-infections. J Hosp Infect 1992;22: 85 92. 11. Olson MM, Shanholtzer CJ, Lee JT Jr, et al. Ten years of prospective Clostridium difficile-associated disease surveillance and treatment at the Minneapolis VA Medical Center, 1982-1991. Infect Control Hosp Epidemiol 1994;15:371 381. 12. Fekety R, McFarland LV, Surawicz CM, et al. Recurrent Clostridium difficile diarrhea: characteristics of and risk factors for patients enrolled in a prospective, randomized, double-blinded trial. Clin Infect Dis 1997;24:324 333. 13. McFarland LV, Surawicz CM, Rubin M, et al. Recurrent Clostridium difficile disease: epidemiology and clinical characteristics. Infect Control Hosp Epidemiol 1999;20:43 50. 14. Tedesco FJ. Treatment of recurrent antibiotic-associated pseudomembranous colitis. Am J Gastroenterol 1982;77:220 221. 15. Tedesco FJ, Gordon D, Fortson WC. Approach to patients with multiple relapses of antibiotic-associated pseudomembranous colitis. Am J Gastroenterol 1985;80:867 868. 16. McFarland LV, Elmer GW, Surawicz CM. Breaking the cycle: treatment strategies for 163 cases of recurrent Clostridium difficile disease. Am J Gastroenterol 2002;97:1769 1775. 17. Surawicz CM, McFarland LV, Elmer G, et al. Treatment of recurrent Clostridium difficile colitis with vancomycin and Saccharomyces boulardii. Am J Gastroenterol 1989;84:1285 1287. 18. McFarland LV, Surawicz CM, Greenberg RN, et al. A randomized placebo-controlled trial of Saccharomyces boulardii in combination with standard antibiotics for Clostridium difficile disease. JAMA 1994;271:1913 1918. 19. Surawicz CM, McFarland LV, Greenberg RN, et al. The search for a better treatment for recurrent Clostridium difficile disease: use of high-dose vancomycin combined with Saccharomyces boulardii. Clin Infect Dis 2000;31:1012 1017. 20. Leung DY, Kelly CP, Boguniewicz M, et al. Treatment with intravenously administered globulin of chronic relapsing colitis induced by Clostridium difficile toxin. J Pediatr 1991;118:633 637. 21. Salcedo J, Keates S, Pothoulakis C, et al. Intravenous immunoglobulin therapy for severe Clostridium difficile colitis. Gut 1997; 41:366 370. 22. Wilcox MH. Descriptive study of intravenous immunoglobulin for the treatment of recurrent Clostridium difficile diarrhoea. J Antimicrob Chemother 2004;53:882 884. 23. Sougioultzis S, Kyne L, Drudy D, et al. Clostridium difficile toxoid vaccine in recurrent C difficile-associated diarrhea. Gastroenterology 2005;128:764 770. 24. McPherson S, Rees CJ, Ellis R, et al. Intravenous immunoglobulin for the treatment of severe, refractory, and recurrent Clostridium difficile diarrhea. Dis Colon Rectum 2006;49:640 645. 25. Kyne L, Warny M, Qamar A, et al. Association between antibody response to toxin A and protection against recurrent Clostridium difficile diarrhoea. Lancet 2001;357:189 193. 26. Pepin J, Alary ME, Valiquette L, et al. Increasing risk of relapse after treatment of Clostridium difficile colitis in Quebec, Canada. Clin Infect Dis 2005;40:1591 1597. 27. Nair S, Yadav D, Corpuz M, et al. Clostridium difficile colitis: factors influencing treatment failure and relapse a prospective evaluation. Am J Gastroenterol 1998;93:1873 1876. 28. Aronsson B, Granstrom M, Mollby R, et al. Serum antibody response to Clostridium difficile toxins in patients with Clostridium difficile diarrhoea. Infection 1985;13:97 101. 29. Bacon AE III, Fekety R. Immunoglobulin G directed against toxins A and B of Clostridium difficile in the general population and patients with antibiotic-associated diarrhea. Diagn Microbiol Infect Dis 1994;18:205 209. 30. Warny M, Vaerman JP, Avesani V, et al. Human antibody response to Clostridium difficile toxin A in relation to clinical course of infection. Infect Immun 1994;62:384 389. 31. Kyne L, Warny M, Qamar A, et al. Asymptomatic carriage of Clostridium difficile and serum levels of IgG antibody against toxin A. N Engl J Med 2000;342:390 397. 32. Charlson ME, Sax FL, MacKenzie CR, et al. Assessing illness severity: does clinical judgment work? J Chronic Dis 1986;39: 439 452. 33. Charlson ME, Pompei P, Ales KL, et al. A new method of classifying prognostic comorbidity in longitudinal studies: development and validation. J Chronic Dis 1987;40:373 383. 34. Horn SD, Sharkey PD, Bertram DA. Measuring severity of illness: homogeneous case mix groups. Med Care 1983;21:14 30. 35. Hanley JA, McNeil BJ. The meaning and use of the area under a receiver operating characteristic (ROC) curve. Radiology 1982; 143:29 36. 36. Do AN, Fridkin SK, Yechouron A, et al. Risk factors for early recurrent Clostridium difficile-associated diarrhea. Clin Infect Dis 1998;26:954 959. 37. Pepin J, Valiquette L, Alary ME, et al. Clostridium difficile-associated diarrhea in a region of Quebec from 1991 to 2003: a changing pattern of disease severity. CMAJ 2004;171:466 472. 38. Loo VG, Poirier L, Miller MA, et al. A predominantly clonal multiinstitutional outbreak of Clostridium difficile-associated diarrhea with high morbidity and mortality. N Engl J Med 2005;353:2442 2449. 39. McDonald LC, Killgore GE, Thompson A, et al. An epidemic, toxin gene-variant strain of Clostridium difficile. N Engl J Med 2005; 353:2433 2441. 40. Kyne L, Sougioultzis S, McFarland LV, et al. Underlying disease severity as a major risk factor for nosocomial Clostridium difficile diarrhea. Infect Control Hosp Epidemiol 2002;23:653 659. 41. Johnson S, Schriever C, Galang M, et al. Interruption of recurrent Clostridium difficile-associated diarrhea episodes by serial therapy with vancomycin and rifaximin. Clin Infect Dis 2007;44:846 848. 42. Louie TJ, Peppe J, Watt CK, et al. Tolevamer, a novel nonantibiotic polymer, compared with vancomycin in the treatment of mild

1214 HU ET AL GASTROENTEROLOGY Vol. 136, No. 4 to moderately severe Clostridium difficile-associated diarrhea. Clin Infect Dis 2006;43:411 420. Received June 9, 2008. Accepted December 11, 2008. Reprint requests Address requests for reprints to: Ciarán P. Kelly, MD, Associate Professor of Medicine, Harvard Medical School, Division of Gastroenterology, Department of Medicine, Beth Israel Deaconess Medical Center, 330 Brookline Ave, Dana 601, Boston, Massachusetts 02215. e-mail: ckelly2@bidmc.harvard.edu; fax: 617-667-8144. Acknowledgements The authors thank the subjects who participated in this study for their generous contributions. Conflicts of interest The authors disclose the following: During the past 2 years, Ciarán P. Kelly has acted as a scientific consultant for Acambis, Actelion, BioHelix, Genzyme, Replidyne, Salix, and ViroPharm and has received research grant funding from Actelion Inc, Genzyme Inc, Massachusetts Biologics Laboratories, Medarex Inc, and Salix Pharmaceuticals, companies that are producing or developing treatments for C difficile infection. No potential conflicts of interest exist for other authors. Funding Supported by grants from the National Institutes of Health (RO-1 AI053069 (to C.P.K.); K30-HL04095 for the Scholars in Clinical Science Program at Harvard Medical School (in which Mary Y. Hu was enrolled); and T32-DK0776 (to S.M. and D.A.L.); and the Irish Health Research Board (RP/2005/72, to L.K.).