Received 26 March 2013; returned 21 April 2013; revised 22 July 2013; accepted 26 July 2013

J Antimicrob Chemother 2014; 69: 211 218 doi:10.1093/jac/dkt340 Advance Access publication 29 August 2013 Method-specific performance of vancomycin MIC susceptibility tests in predicting mortality of patients with methicillin-resistant Staphylococcus aureus bacteraemia Shey-Ying Chen 1,2, Chun-Hsing Liao 3, Jiun-Ling Wang 4, Wen-Chu Chiang 1,2, Mei-Shu Lai 2, Wei-Chu Chie 2, Shan-Chwen Chang 5 and Po-Ren Hsueh 5,6 * 1 Department of Emergency Medicine, National Taiwan University Hospital, National Taiwan University College of Medicine, Taipei, Taiwan; 2 Institute of Epidemiology and Preventive Medicine, College of Public Health, National Taiwan University, Taipei, Taiwan; 3 Department of Internal Medicine, Far Eastern Memorial Hospital, Taipei, Taiwan; 4 Department of Internal Medicine, E-Da Hospital, I-Shou University, Kaohsiung, Taiwan; 5 Department of Internal Medicine, National Taiwan University Hospital, National Taiwan University College of Medicine, Taipei, Taiwan; 6 Department of Laboratory Medicine, National Taiwan University Hospital, National Taiwan University College of Medicine, Taipei, Taiwan *Corresponding author. Tel: +886-2-23123456, ext. 65355; Fax: +886-2-23224263; E-mail: hsporen@ntu.edu.tw Received 26 March 2013; returned 21 April 2013; revised 22 July 2013; accepted 26 July 2013 Objectives: Emerging evidence shows that methicillin-resistant Staphylococcus aureus (MRSA) infections caused by isolates with higher vancomycin MICs within the susceptibility range are associated with adverse outcomes. No study, however, has examined different susceptibility tests in predicting treatment outcomes of MRSA infections. Methods: This retrospective cohort study included 393 patients with MRSA bacteraemia. Vancomycin MICs for all MRSA isolates were determined simultaneously by agar dilution and the Etest, and using the MicroScan, VITEK-2 and Phoenix automated systems, and categorized into low- and high-mic isolates at a breakpoint of 2 mg/l. The essential and categorical agreement between testing methods was compared. The method-specific ability to predict in-hospital mortality was examined by multivariate logistic regression analysis controlling for other potential confounders using clinical data from 310 vancomycin-treated MRSA bacteraemia patients. Results: The agar dilution, Etest, MicroScan, VITEK-2 and Phoenix methods assessed 14.2% (56/393), 9.7% (38/393), 28.8% (113/393), 22.6% (89/393) and 3.1% (12/393) of MRSA isolates as having high ( 2 mg/l) vancomycin MICs. The essential and categorical agreement between testing methods ranged from 98.5% to 100% and from 73.8% to 91.9%, respectively. High vancomycin MICs for isolates determined using agar dilution and the Etest independently predicted mortality when controlling for confounding factors [adjusted OR, 2.321; 95% CI, 1.160 4.641; and adjusted OR, 3.121; 95% CI, 1.293 7.536, respectively]. High vancomycin MICs determined using all three automated systems failed to predict mortality. Conclusions: Vancomycin MICs generated by the agar dilution and Etest methods, but not the automated systems, independently predicted mortality among vancomycin-treated MRSA bacteraemia patients. Clinicians should incorporate this information with clinical assessment for decisions on appropriate anti-mrsa treatment. Keywords: susceptibility testing methods, essential agreement, categorical agreement Introduction Vancomycin is the most commonly used antimicrobial agent in the treatment of methicillin-resistant Staphylococcus aureus (MRSA) infection in many countries because of its predictable activity against MRSA. 1 The clinical use of vancomycin is, however, threatened by increasing reports of a higher likelihood of mortality or treatment failure among patients infected with MRSA presenting with a vancomycin MIC at the high end of the susceptible range. 2 5 On the basis of pharmacodynamic plausibility and available clinical evidence, alternative anti-mrsa agents have been suggested if a vancomycin MIC of 2 mg/l is identified for the causative MRSA isolate, especially for patients with persistent disease or critical illness. 6 8 In applying this recommendation in daily practice it is important to use a susceptibility testing method that can reliably detect in vitro vancomycin resistance # The Author 2013. Published by Oxford University Press on behalf of the British Society for Antimicrobial Chemotherapy. All rights reserved. For Permissions, please e-mail: journals.permissions@oup.com 211

Chen et al. and efficiently predict in vivo treatment outcomes. However, the conclusions of previous MRSA outcome studies supporting the effect of high vancomycin MIC on mortality have been determined predominantly from Etest results. 8 Because previous studies have shown significant differences in vancomycin MICs obtained from different susceptibility testing methods, 9 14 it is difficult to generalize the results observed with the Etest to other testing methods in the treatment of patients infected with an MRSA strain that shows laboratory evidence of reduced vancomycin susceptibility. In this study, we hypothesized that different susceptibility testing methods possess different MIC detection and outcome prediction capabilities owing to their intrinsic method-specific design, and are therefore able to be compared in parallel at the same cut-off value. Five susceptibility testing methods agar dilution, the Etest and three commonly used automated systems were simultaneously evaluated in this study. We aimed to understand the agreement between tests and, more importantly, to evaluate the outcome prediction capabilities of different susceptibility testing methods for better guidance on MRSA treatment. Patients and methods Study design, setting and patient selection This retrospective cohort study was conducted in the Emergency Department (ED) of National Taiwan University Hospital. This 2500 bed university-affiliated teaching hospital provides both primary and tertiary care in northern Taiwan and has an average of 67000 patient discharges and 100000 patient ED visits annually. All ED patients aged 15 years diagnosed with MRSA bacteraemia from 1 January 2001 to 31 December 2011 were initially included. For patients with repeated MRSA bacteraemia during the study period, only the first episode was considered. Patients with MRSA bacteraemia were then excluded if the bloodstream isolate was not available for antimicrobial susceptibility testing or the patient was subsequently hospitalized and treated outside the study hospital. These patients with MRSA bacteraemia constituted the main study groups, and their bloodstream isolates were used for the comparison of vancomycin MIC results between different testing methods. To evaluate the performance of specific MIC susceptibility tests in predicting clinical outcomes, only patients with MRSA bacteraemia who received vancomycin as empirical therapy (patients with early mortality 3 days after obtaining the index blood culture) or definite antimicrobial therapy (patients who survived.3 days after obtaining the index blood culture) were included in the outcome predictability analysis. This study was approved by the institutional review board of the hospital, and the requirement for informed consent from each patient was waived. Data collection, and information on variables and outcomes Patient data included demographics, healthcare-associated exposure history within the past year, pre-existing comorbidities, initial presentations and empirical antimicrobial therapy in the ED. The subsequent hospitalization course, definitive antimicrobial therapy, primary site of infection, vancomycin trough level (if available) and all-cause mortality were determined from the medical records. A full description of the definition of healthcare-associated exposure has previously been reported. 15 The severity of pre-existing comorbidities was assessed using the modified Charlson comorbidity score. 16 Initial vital signs on ED presentation were classified according to the International Sepsis Definitions. The severityof acute illness asassessed bythe sepsis syndrome criteria was graded as: no sepsis or simple sepsis; severe sepsis; and septic shock. 17,18 Inadequate vancomycin therapy was defined as a vancomycin trough level of,10 mg/l measured 3 days but,1 week after starting vancomycin therapy. All-cause in-hospital mortality was used as the main outcome variable in this retrospective study. Microbiological procedures and determination of vancomycin MICs Blood culture specimens were inoculated into BACTEC standard culture bottles or BACTEC PLUS culture bottles in a BACTEC 9000 system (Becton Dickinson, Sparks, MD, USA). Clinical S. aureus isolates were identified by colony morphology, results of Gram staining and a positive slide or tube coagulase test. Methicillin-resistant isolates were detected by the standard disc diffusion method. A 30 mg cefoxitin disc (BBL Microbiology Systems, Cockeysville, MD, USA) was used to detect MRSA after 2006. 19,20 Clinical MRSA isolates were collected and stored at 2708C in trypticase soy broth (Difco Laboratories, Detroit, MI, USA) supplemented with 15% glycerol. All available MRSA isolates were thawed and subcultured twice before testing. Vancomycin MIC values were determined simultaneously by the agar dilution method, according to CLSI guidelines, 21 and the Etest (AB BIODISK, Solna, Sweden) and three automated systems: MicroScan (Siemens Healthcare Diagnostics), VITEK-2 (biomérieux, Durham, NC, USA) and Phoenix (Becton Dickinson Diagnostic Systems, Sparks, MD, USA) according to the manufacturers instructions. Etest MICs were determined using a standard inoculum (turbidity equivalent to that of a 0.5 McFarland standard), Mueller Hinton agar and commercialized strips with concentrations ranging from 0.016 to 256 mg/l. For the agar dilution method, the final inoculum of each isolate on Mueller Hinton agar was 10 4 cfu per spot. 21 All laboratory procedures were performed by experienced microbiology laboratory technicians who were blinded to the clinical outcome of the study patients. S. aureus ATCC 29213 was used as a control strain for all methods. Statistical analysis Binary variables were compared using the x 2 test. Age and the Charlson comorbidity score were compared using the independent Student t-test. Vancomycin values of all testing methods were categorized into low- and high-mic groups at cut-off values of,2 mg/l and 2 mg/l, respectively. We used three approaches essential agreement, categorical agreement and very major error to examine the inter-test agreements between the different testing methods. Essential agreement between the agar dilution method and the other four methods (Etest, MicroScan, VITEK-2 and Phoenix) was defined as a difference between MICs of +1 log 2 dilution or less using the agardilution method asthe reference standard. 22 Categorical agreements between the five testing methods were evaluated as the percentage of isolates that had concordant test results in determining high ( 2 mg/l) and low (,2 mg/l) vancomycin MIC values. A very major error for the Etest, MicroScan, VITEK-2 or Phoenix was defined as a high-mic isolate ( 2 mg/l) determined using the agar dilution method that was interpreted asa low-micisolate (,2 mg/l) bythe indicated testing method. Sensitivity, specificity, positive predictive value and negative predictive value with associated 95% CIs of different MIC testing methods in capturing cases of mortality were obtained using standard definitions and methods. 23 We performed forward, backward and manual stepwise multivariate logistic regression modelling to understand the independent role of method-specific high vancomycin MIC values in predicting in-hospital mortality of vancomycin-treated patients with MRSA bacteraemia. Age was included throughout the model construction process because of its potential effect on mortality. The ability to predict mortality for different susceptibilitytests was evaluatedseparatelybyincluding the highvancomycin MIC isolate variable determined using a specific testing method with other potential confounding factors identified on univariate analysis (P 0.10) into the multivariate regression modelling. Finally, the discriminatory ability of different MIC testing methods was assessed by the area under 212

Predicting mortality by different MIC tests JAC the receiver operating characteristic curve of the final model integrating the method-specific high vancomycin MIC isolate variable. 24 Data were analysed with SPSS for Windows version 16.0 (SPSS, Chicago, IL, USA) and CART software (Salford Systems). All P values are two-sided, and P,0.05 was considered statistically significant. Results Patients We initially considered 432 non-duplicate episodes of MRSA bacteraemia recorded during the 11 year study period. Nine patients were excluded because they received treatment outside the study hospital, and 30 patients were excluded because their bloodstream isolates were not available for antimicrobial susceptibility testing. In total, 393 patients with MRSA bacteraemia were finally enrolled in this study for the evaluation of measurement agreement between different MIC testing methods. To evaluate the performance of a specific MIC testing method in predicting treatment outcome, only clinical data from 310 patients with MRSA bacteraemia who received vancomycin therapy were used in the outcome predictability analysis. The clinical characteristics of all study patients (n¼393) and of the 310 vancomycin-treated patients are detailed in Table 1. Method-specific vancomycin MIC values and measurement agreement Table 2 shows the vancomycin MIC values for the different test methods. The Phoenix system identified the lowest percentage of MRSA isolates with a vancomycin MIC 2 mg/l (12/393, 3.1%) and the MicroScan system the highest (113/393, 28.8%). Within a 2-fold dilution, the four commercial susceptibility testing methods (Etest, MicroScan, VITEK-2 and Phoenix) showed 98.5% 100% essential agreement with the agar dilution method. When categorical agreement (high versus low vancomycin MIC) between different testing methods was evaluated, the Phoenix system had the highest (88.3%) and the MicroScan system the lowest (75.3%) measurement agreement with the agar dilution method (Table 3). Very major errors occurred in 9.9% (39/393), 5.1% (20/393), 4.8% (19/393) and 11.5% (45/393) of the Etest, MicroScan, VITEK-2 and Phoenix results, respectively. Table 1. Clinical characteristics of 393 patients with MRSA bacteraemia who were treated with or without vancomycin Characteristic All patients (n¼393) Treated with vancomycin (n¼310) Not treated with vancomycin (n¼83) P Age (years), mean+sd 66.2+16.2 66.1+16.6 66.5+15.1 0.860 Male sex 239 (60.8) 183 (59.0) 56 (67.5) 0.162 Healthcare-associated exposure within the past year 338 (86.0) 266 (85.8) 72 (86.7) 0.826 Comorbid medical condition diabetes mellitus 165 (42.0) 134 (43.2) 31 (37.3) 0.335 malignancy 89 (22.6) 65 (21.0) 24 (28.9) 0.124 end-stage renal disease 104 (26.5) 94 (30.3) 10 (12.0) 0.001 liver cirrhosis 38 (9.7) 35 (11.3) 3 (3.6) 0.036 congestive heart failure 47 (12.0) 38 (12.3) 9 (10.8) 0.724 cerebrovascular accident 91 (23.2) 67 (21.6) 24 (28.9) 0.161 AIDS 2 (0.5) 2 (0.6) 0 (0) 1.000 intravenous drug user 11 (2.8) 11 (3.5) 0 (0) 0.130 Presence of central vascular catheters 117 (29.8) 99 (31.9) 18 (21.7) 0.070 Charlson score, mean+sd 4.1+2.7 4.1+2.7 4.0+2.9 0.832 High Charlson score (.4) 164 (41.7) 128 (41.3) 36 (43.4) 0.732 Infection focus vascular device related 94 (23.9) 85 (27.4) 9 (10.8) 0.002 skin and soft tissue 87 (22.1) 71 (22.9) 16 (19.3) 0.480 lower respiratory tract infection 81 (20.6) 54 (17.4) 27 (32.5) 0.003 orthopaedic infection 58 (14.8) 47 (15.2) 11 (13.3) 0.663 endovascular infection 53 (13.5) 48 (15.5) 5 (6.0) 0.025 other infection sites 33 (8.4) a 28 (9.0) 5 (6.0) 0.380 no focus identified 54 (13.7) b 35 (11.3) 19 (22.9) 0.014 Data are n (%) of subjects unless otherwise indicated. a Includes genitourinary tract infection (24), brain abscess (4), endophthalmitis (2), splenic abscess (1), enterocutaneous fistula with intra-abdominal abscess formation (1) and percutaneous transhepatic cholangiography and drainage tube-associated infection (1). b Includes primary bacteraemia (38) and bacteraemia with no identified focus due to rapidly fatal course (16). 213

Chen et al. Method-specific performance in outcome prediction The overall in-hospital mortality rate of the 310 vancomycintreated patients with MRSA bacteraemia was 30.3% (94/310). The sensitivity of a high vancomycin MIC, measured by the five study methods, at predicting in-hospital mortality was highest for the MicroScan (27/94, 28.7%), followed by the VITEK-2 (25/ 94, 26.6%), agar dilution (22/94, 23.4%), Etest (14/94, 14.9%) and Phoenix (3/94, 3.2%) (Table 4). Comparison of the clinical characteristics of patients who survived to hospital discharge and those who did not is shown in Table 5. The agar dilution and Etest methods demonstrated statistically differences between the survival and non-survival patient groups in the detection of isolates with high vancomycin MICs. In the multivariate logistic regression analysis controlling for other independent confounders, including age, malignancy, thrombocytopenia, endovascular infection and lower respiratory tract infection, a high vancomycin MIC isolate was an independent predictor of mortality only when the variable was assessed by agar dilution or the Etest [adjusted OR (aor), 2.321; 95% CI, 1.160 4.641; and aor, 3.121; 95% CI, 1.293 7.536, respectively] (Table 6). The AUCs of agar dilution- and Etest-incorporated Table 2. Vancomycin MIC results for 393 MRSA bloodstream isolates by different testing methods Test method Number of isolates with indicated vancomycin MIC (mg/l) 0.5 1 1.5 2 3 4 Percentage of isolates with a high vancomycin MIC ( 2 mg/l) Agar dilution 11 326 54 2 14.2 Etest 0 144 211 33 5 0 9.7 MicroScan 1 279 111 2 28.8 VITEK-2 138 166 88 1 22.6 Phoenix 129 252 12 0 3.1 regression models in predicting mortality were 0.743 (95% CI, 0.683 0.803) and 0.751 (95% CI, 0.692 0.810), respectively (Figure 1). Finally, if the clinical data from 81 patients who did not have measured vancomycin trough levels and 48 patients who received inadequate vancomycin therapy were further excluded, a high vancomycin MIC isolate identified by either the agar dilution or the Etest method remained an independent predictor of mortality (aor, 2.722; 95% CI, 1.071 6.918; and aor, 3.306; 95% CI, 1.060 10.306, respectively) in multivariate logistic regression models controlling for other confounders including age, malignancy, thrombocytopenia, endovascular infection and lower respiratory tract infection. Discussion We specifically evaluated the performance of different vancomycin susceptibility testing methods in predicting the in-hospital mortality of vancomycin-treated patients with MRSA bacteraemia. There are three notable findings in this study. First, although remarkably good essential agreements were observed, categorical agreements differed significantly across different susceptibility testing methods for vancomycin MICs for MRSA isolates. Second, vancomycin MICs determined using the three automated susceptibility testing systems failed to predict the mortality of patients with MRSA bacteraemia. Third, both agar dilution and the Etest effectively stratified the risk of in-hospital mortality of patients with MRSA bacteraemia independently of the effect from other confounding factors. Furthermore, at the same cut-off value ( 2 mg/l), the Etest method had a higher predictability for treatment outcome of MRSA infections compared with the agar dilution method. Many previous studies have evaluated the accuracy of different susceptibility tests in determining vancomycin MICs using different reference standards. 9 14 Although poor measurement agreement between susceptibility tests has been observed, as shown again by this study, no study has comprehensively evaluated the clinical performance of different tests in predicting the clinical outcome Table 3. Essential and categorical agreement of vancomycin MICs obtained by the five study susceptibility testing methods for 393 MRSA bloodstream isolates Number of isolates for which the MICs determined using the indicated method differed from the agar dilution MICs by the following number of log 2 dilutions Categorical agreement b, n (%) of isolates 23 22 21 same +1 +2 +3 essential agreement a, n (%) of isolates agar dilution Etest MicroScan VITEK-2 Agar dilution ref Etest 0 0 5 179 206 3 0 390 (99.2) 333 (84.7) MicroScan 0 0 22 282 89 0 0 393 (100.0) 296 (75.3) 302 (76.8) VITEK-2 0 3 145 189 56 0 0 390 (99.2) 322 (81.9) 330 (84.0) 303 (77.1) Phoenix 0 6 157 225 5 0 0 387 (98.5) 347 (88.3) 361 (91.9) 290 (73.8) 314 (79.9) a Essential agreement between the agar dilution method and the other four susceptibility testing methods was defined as the difference between MICs of +1 log 2 dilution or less using the agar dilution method as the reference standard (ref). b Categorical agreement between the five studysusceptibility testing methodswas measured asthe percentage of isolatesthathad concordant test results when determining high ( 2 mg/l) and low (,2 mg/l) vancomycin MICs. 214

Predicting mortality by different MIC tests JAC Table 4. Parameters for method-specific high vancomycin MICs a in predicting in-hospital mortality among 310 vancomycin-treated patients with MRSA bacteraemia Sensitivity Specificity Positive predictive value Negative predictive value % 95% CI % 95% CI % 95% CI % 95% CI Agar dilution 23.4 15.5 33.5 88.9 83.7 92.6 47.8 33.1 62.9 72.7 66.9 77.9 Etest 14.9 8.7 24.1 94.0 89.7 96.6 51.9 32.4 70.8 71.7 66.0 76.8 MicroScan 28.7 20.1 39.1 72.2 65.7 78.0 31.0 21.8 42.0 70.0 63.4 75.8 VITEK-2 26.6 18.3 36.9 77.8 71.5 83.0 34.2 23.8 46.4 70.9 64.6 76.5 Phoenix 3.2 0.8 9.7 97.2 93.8 98.9 33.3 9.0 69.1 69.8 64.2 74.8 a Defined as 2 mg/l for all susceptibility testing methods. Table 5. Clinical characteristics and comparison of vancomycin MIC results between MRSA bacteraemia patients with and without survival to hospital discharge Characteristic Survival (n¼216) Non-survival (n¼94) P Age (years), mean+sd 64.8+17.2 69.2+14.6 0.022 Male sex 129 (59.7) 54 (57.4) 0.708 Comorbid medical condition diabetes mellitus 94 (43.5) 40 (42.6) 0.875 malignancy 34 (15.7) 31 (33.0) 0.001 end-stage renal disease 67 (31.0) 27 (28.7) 0.686 liver cirrhosis 19 (8.8) 16 (17.0) 0.035 congestive heart failure 31 (14.4) 7 (7.4) 0.088 cerebrovascular accident 50 (23.1) 17 (18.1) 0.319 High Charlson score (.4) 81 (37.5) 47 (50.0) 0.040 Thrombocytopenia 33 (15.3) 31 (33.0),0.001 Severe sepsis or septic shock 64 (29.6) 45 (47.9) 0.002 Inadequate vancomycin therapy a 36 (21.6) b 12 (19.4) c 0.716 Endovascular infection 25 (11.6) 23 (24.5) 0.004 Low respiratory tract infection 31 (14.4) 23 (24.5) 0.031 Vascular device-related infection 64 (29.6) 21 (22.3) 0.186 non-removable catheter 5 (2.3) 4 (4.3) 0.462 time to remove infected vascular catheter (days), mean+sd 4.0+4.4 4.4+4.5 0.737 Length of stay (days), mean+sd 37.6+27.9 24.4+22.3,0.001 High vancomycin MIC isolate d, identified by agar dilution 24 (11.1) 22 (23.4) 0.005 Etest 13 (6.0) 14 (14.9) 0.011 MicroScan 60 (27.8) 27 (28.7) 0.865 VITEK-2 48 (22.2) 25 (26.6) 0.404 Phoenix 6 (2.8) 3 (3.2) 1.000 Data are n (%) of subjects, unless otherwise indicated. a Defined as a vancomycin trough level,10 mg/l measured 3 days and,1 week after starting vancomycin therapy. b Vancomycin trough level was available in 167 patients. c Vancomycin trough level was available in 62 patients. d Defined as an isolate with an MIC 2 mg/l for all susceptibility testing methods. 215

Chen et al. Table 6. Outcome predictability performance of different MRSA vancomycin MIC testing methods by multivariate logistic regression modelling evaluation Multivariate models, with method-specific MIC results included a Univariate, OR (95% CI) agar dilution, aor (95% CI) Etest, aor (95% CI) other test methods b, aor (95% CI) Age (per year increase) 1.017 (1.001 1.033) c 1.032 (1.013 1.051) d 1.033 (1.014 1.052) d 1.033 (1.015 1.052) d Male sex 0.910 (0.557 1.488) Diabetes mellitus 0.961 (0.589 1.568) Malignancy 2.634 (1.497 4.633) d 3.607 (1.904 6.834) d 3.695 (1.950 7.003) d 3.682 (1.957 6.929) d End-stage renal disease 0.896 (0.527 1.525) Liver cirrhosis High Charlson score (.4) 2.127 (1.041 4.347) c 1.667 (1.022 2.718) c Thrombocytopenia 2.729 (1.547 4.814) d 2.622 (1.399 4.914) d 2.556 (1.370 4.768) d 2.483 (1.338 4.610) d Severe sepsis or septic shock 2.181 (1.324 3.593) d Endovascular infection 2.475 (1.320 4.640) d 4.108 (1.994 8.462) d 4.003 (1.930 8.304) d 4.029 (1.958 8.290) d Lower respiratory tract infection 1.933 (1.056 3.540) c 2.021 (1.035 3.946) c 2.293 (1.174 4.479) c 2.170 (1.122 4.198) c Inadequate vancomycin therapy 0.732 (0.362 1.479) High vancomycin MIC MRSA isolate e agar dilution 2.444 (1.291 4.630) d 2.321 (1.160 4.641) c NE NE Etest 2.733 (1.230 6.070) c NE 3.121 (1.293 7.536) c NE MicroScan 1.048 (0.612 1.792) NE NE NS f VITEK-2 1.268 (0.725 2.217) NE NE NS g Phoenix 1.154 (0.282 4.715) NE NE NS h NE, not entered; NS, not significant. a Only results from one vancomycin MIC testing method was entered into the multivariate logistic regression modelling process when evaluating methodspecific outcome prediction performance. b Included the high vancomycin MICvariable assessed by the MicroScan, VITEK-2 and Phoenixautomated systems. All were not an independent predictor in the multivariate regression model when adjusted by other confounders. c P,0.05. d P,0.01. e Defined as an isolate with an MIC 2 mg/l for all susceptibility testing methods. f aor was 1.221 and 95% CI were 0.675 2.210 if MIC data from the MicroScan system were forced into the final regression model. g aor was 1.256 and 95% CI were 0.677 2.328 if MIC data from the Vitek-2 system were forced into the final regression model. h aor was 1.302 and 95% CI were 0.290 5.848 if MIC data from the Phoenix system were forced into the final regression model. of patients with MRSA infection. A study by Hsu et al. 11 found better outcome predictability for patients with MRSA infection with the Etest method than the VITEK Legacy automated system. However, the study was limited by the small sample size, the use of end-of-treatment response rather than mortality as the outcome variable and failure to control the severity of chronic comorbidity. Our study therefore provides strong clinical evidence supporting the previous observation that strains of staphylococci with reduced vancomycin susceptibility are best detected by nonautomated quantitative testing methods. 16 The failure of MIC determination by the three automated methods to predict mortality in the current study is an important finding and deserves further discussion. Both the MicroScan and Phoenix automated systems use the broth microdilution method and determine the MIC by turbidity. However, the MicroScan system interprets the MIC values after a full 24 h of incubation and the Phoenix system interprets them at 6 16 h (12 h on average). The difference in MIC reading times might influence the turbidity of the incubating broth and therefore the MIC results. Different from the MicroScan and Phoenix systems, the VITEK-2 system continuously monitors bacterial growth on susceptibility cards containing different vancomycin concentrations during an abbreviated incubation period. The VITEK-2 system infers the vancomycin MIC result by comparing the growth curve of the index isolate with the curves of reference isolates in a computerized database, which is different from the standard method recommended by the CLSI. The method-specific design of different automated susceptibility testing systems might therefore contribute to their performance in predicting outcome in patients with MRSA bacteraemia. 216

Predicting mortality by different MIC tests JAC Sensitivity 1.0 0.8 0.6 0.4 0.2 Agar-dilution MIC-incorporated model (AUC, 0.743; 95% CI, 0.683 0.803) Etest MIC-incorporated model (AUC, 0.751; 95% CI, 0.692 0.810) Basic model (AUC, 0.738; 95% CI, 0.679 0.798) 0.0 0.0 0.2 0.4 0.6 0.8 1.0 1 Specificity Figure 1. Receiver operating characteristic curves and associated AUCs for different outcome prediction models. In the current study, although both the agar dilution and Etest methods were useful in predicting mortality, we found that the Etest had better ability to predict mortality than the agar dilution method, as selected by final multivariate logistic regression modelling. Because vancomycin MIC results generated by the Etest tend to be 0.5 1 log 2 dilution higher than those generated by the CLSI broth or agar dilution methods, 12 it was postulated that an effect of high vancomycin MIC at 2 mg/l on patient mortality would be more easilyobserved in studies using the Etest as the susceptibility testing method given in studies with a limited number of cases. This might explain why previous outcome studies supporting an effect of high vancomycin MIC have predominantly been based on the results of the Etest rather than the broth or agar dilution methods. Although we postulated that a high vancomycin MIC affects patient outcomes by reducing the probability of achieving an optimal ratio of AUC at 24 h (AUC 24h ) to MIC target, 25,26 other explanations do exist. First, isolates with a high vancomycin MIC may contain a high percentage of heteroresistant vancomycinintermediate S. aureus and contribute significantly to patient mortality. 27,28 Second, MIC may be a surrogate marker for strainspecific virulence that affects patient mortality. 18,29 It is especially important to understand the interactive effect of vancomycin MIC and strain type on patient outcome because more and more MRSA infections occurring in hospitals are due to novel communityassociated MRSA strains. 30 There are limitations in this study. First, this was a single-centre study conducted in Taiwan. The generalizability of the study findings to other communities, where the circulating MRSA strains might differ, requires further confirmation. Second, the clinical data from this observational cohort were retrospectively collected and therefore subject to information bias. However, we tried to minimize such bias by using in-hospital mortality as the primary study outcome, for which information could be accurately obtained for all study patients. Third, the vancomycin MIC values in this study were determined using MRSA isolates stored for.6 months from the time of isolation. Recent evidence has shown that frozen storage significantly affects the MICs for MRSA isolates, and has an inconsistent effect if different antimicrobial agents are tested or susceptibility test methods adopted. 31 Our study findings therefore need to be validated in a prospective manner using MIC values measured at the time of isolation. Finally, though agar dilution is an accepted CLSI reference method, the broth microdilution method is more commonly used as the reference standard in contemporary studies. Though discrepancy between the two reference methods in MIC determination has been observed, many studies have also shown that agar dilution has very good agreement with broth microdilution for many antibiotics, including vancomycin. One study even found that the essential agreement between the two reference methods for vancomycin MIC was 96.4+2.5% and categorical agreement was 100%. 32 In conclusion, a subtle increase in vancomycin MICvalues within the current susceptibility window is associated with a higher risk of mortality among patients with MRSA bloodstream infections. The effect on patient mortality is independently predicted by both the agar dilution and Etest methods. This is especially important for patients with MRSA infection with vancomycin MICs 2 mg/l identified either by agar dilution or the Etest. Clinicians should consider these MRSA strains to be vancomycin non-susceptible for patients with MRSA bloodstream or invasive infections. None of the automated susceptibility testing systems showed predictability for mortality in vancomycin-treated patients with MRSA bacteraemia. Because the categorical measurement agreement between different susceptibility testing methods in identifying a high vancomycin MIC MRSA isolate is generally low, the results from automated systems cannot simply be inferred from those of non-automated quantitative testing methods when deciding on antimicrobial therapy. Acknowledgements We acknowledge the statistical assistance provided by the National Translational Medicine and Clinical Trial Resource Center [which is funded by the National Research Program for Biopharmaceuticals (NRPB) of the National Science Council of Taiwan; NSC101-2325-B-002-078] and the Department of Medical Research, National Taiwan University Hospital. Funding This study was supported by internal funding. Transparency declarations None to declare. References 1 Levine DP. Vancomycin: a history. Clin Infect Dis 2006; 42 Suppl 1: S5 12. 2 Sakoulas G, Moise-Broder PA, Schentag J et al. Relationship of MIC and bactericidal activity to efficacy of vancomycin for treatment of 217

Chen et al. methicillin-resistant Staphylococcus aureus bacteremia. J Clin Microbiol 2004; 42: 2398 402. 3 Hidayat LK, Hsu DI, Quist R et al. High-dose vancomycin therapy for methicillin-resistant Staphylococcus aureus infections: efficacy and toxicity. Arch Intern Med 2006; 166: 2138 44. 4 Soriano A, Marco F, Martinez JA et al. Influence of vancomycin minimum inhibitory concentration on the treatment of methicillin-resistant Staphylococcus aureus bacteremia. Clin Infect Dis 2008; 46: 193 200. 5 Lodise TP, Graves J, Evans A et al. Relationship between vancomycin MIC and failure among patients with methicillin-resistant Staphylococcus aureus bacteremia treated with vancomycin. Antimicrob Agents Chemother 2008; 52: 3315 20. 6 Gould IM, Cauda R, Esposite S et al. Management of serious methicillinresistant Staphylococcus aureus infections: what are the limits?. Int J Antimicrob Agents 2011; 37: 202 9. 7 Liu C, Bayer A, Cosgrove SE et al. Clinical practice guidelines by the Infectious Diseases Society of America for the treatment of methicillinresistant Staphylococcus aureus infections in adults and children. Clin Infect Dis 2011; 52: e18 55. 8 van Hal SJ, Lodise TP, Paterson DL. The clinical significance of vancomycin minimal inhibitory concentration in Staphylococcus aureus infections: a systematic review and meta-analysis. Clin Infect Dis 2012; 54: 755 71. 9 Tenover FC, Lancaster MV, Hill BC et al. Characterization of staphylococci with reduced susceptibilities to vancomycin and other glycopeptides. J Clin Microbiol 1998; 36: 1020 7. 10 Walsh TR, Bolmström A, Qwärnström Aet al. Evaluation of current methods for detection of staphylococci with reduced susceptibility to glycopeptides. J Clin Microbiol 2001; 39: 2439 44. 11 Hsu DI, Hidayat LK, Quist R et al. Comparison of method-specific vancomycin minimum inhibitory concentration values and their predictability for treatment outcome of methicillin-resistant Staphylococcus aureus (MRSA) infections. Int J Antimicrob Agents 2008; 32: 378 85. 12 Prakash V, Lewis JS II, Jorgensen JH. Vancomycin MICs for methicillinresistant Staphylococcus aureus isolates differ based upon the susceptibility test method used. Antimicrob Agents Chemother 2008; 52: 4528. 13 Keel RA, Sutherland CA, Aslanzadeh J et al. Correlation between vancomycin and daptomycin MIC values for methicillin-susceptible and methicillin-resistant Staphylococcus aureus by 3 testing methodologies. Diagn Microbiol Infect Dis 2010; 68: 326 9. 14 Bland CM, Porr WH, Davis KA et al. Vancomycin MIC susceptibility testing of methicillin-susceptible and methicillin-resistant Staphylococcus aureus isolates: a comparison between Etest and an automated testing method. South Med J 2010; 103: 1124 8. 15 Wang JL, Chen SY, Wang JT et al. Comparison of both clinical features and mortality risk associated with bacteremia due to communityacquired methicillin-resistant Staphylococcus aureus and methicillinsusceptible S. aureus. Clin Infect Dis 2008; 46: 799 806. 16 Deyo RA, Cherkin DC, Ciol MA. Adapting a clinical comorbidity index for use with ICO-9-CM administrative databases. J Clin Epidemiol 1992; 45: 613 9. 17 Rangel-Frausto MS, Pittet D, Costigan M et al. The natural history of the systemic inflammatory response syndrome (SIRS). A prospective study. JAMA 1995; 273: 117 23. 18 Chen SY, Liao CH, Wang JL et al. Methicillin-resistant Staphylococcus aureus (MRSA) staphylococcal cassette chromosome mec genotype effects outcomes of patients with healthcare-associated MRSA bacteremia independently of vancomycin minimum inhibitory concentration. Clin Infect Dis 2012; 55: 1329 37. 19 National Committee for Clinical Laboratory Standards. Performance Standards for Antimicrobial Susceptibility Testing: Fifteenth Informational Supplement M100-S15. NCCLS, Wayne, PA, USA, 2005. 20 Clinical and Laboratory Standards Institute. Performance Standards for Antimicrobial Susceptibility Testing: Seventeenth Informational Supplement M100-S17. CLSI, Wayne, PA, USA, 2007. 21 Clinical and Laboratory Standards Institute. Performance Standards for Antimicrobial Susceptibility Testing: Twenty-second Informational Supplement M100-S22. CLSI, Wayne, PA, USA, 2012. 22 National Committee for Clinical and Laboratory Standards. Development of In Vitro Susceptibility Testing Criteria and Quality Control Parameters Second Edition: Approved Guideline M23-A2. NCCLS, Wayne, PA, USA, 2001. 23 Diagnosis. In: Fletcher RH, Fletcher SW, Wagner EH, eds. Clinical Epidemiology: The Essentials. 3rd edn. Philadelphia, PA: Lippincott Williams & Wilkins, 1996: 43 74. 24 McNeil BJ, Hanley JA. Statistical approaches to the analysis of receiver operating characteristic (ROC) curves. Med Decis Making 1984; 4: 137 50. 25 Patel N, Pai MP, Rodvold KA et al. Vancomycin: we can t get there from here. Clin Infect Dis 2011; 52: 969 74. 26 Kullar R, Davis SL, Levine DP et al. Impact of vancomycin exposure on outcomes in patients with methicillin-resistant Staphylococcus aureus bacteremia: support for consensus guidelines suggested targets. Clin Infect Dis 2011; 52: 975 81. 27 van Hal SJ, Wehrhahn MC, Barbagiannakos T et al. Performance of various testing methodologies for detection of heteroresistant vancomycinintermediate Staphylococcus aureus in bloodstream isolates. JClinMicrobiol 2011; 49: 1489 94. 28 van Hal SJ, Paterson DL. Systematic review and meta-analysis of the significance of heterogeneous vancomycin-intermediate Staphylococcus aureus isolates. Antimicrob Agents Chemother 2011; 55: 405 10. 29 Holmes NE, Turnidge JD, Munckhof WJ et al. Antibiotics choice may not explain poorer outcomes in patients with Staphylococcus aureus bacteremia and high vancomycin minimum inhibitory concentrations. J Infect Dis 2011; 204: 340 7. 30 Popovich KJ, Weinstein RA, Hota B. Are community-associated methicillin-resistant Staphylococcus aureus (MRSA) strains replacing traditional nosocomial MRSA strains? Clin Infect Dis 2008; 46: 787 94. 31 Edwards B, Milne K, Lawes T et al. Is vancomycin MIC creep method dependent? Analysis of methicillin-resistant Staphylococcus aureus susceptibility trends in blood isolates from North East Scotland from 2006 to 2010. J Clin Microbiol 2012; 50: 318 25. 32 Baker CN, Stocker SA, Culver DH et al. Comparison of the E test to agar dilution, broth microdilution, and agar diffusion susceptibility testing techniques by using a special challenge set of bacteria. J Clin Microbiol 1991; 29: 533 8. 218