Antibiotic-impregnated shunt systems versus standard shunt systems: a meta- and cost-savings analysis

Transcription

1 J Neurosurg Pediatrics 8: , 8: , 2011 Antibiotic-impregnated shunt systems versus standard shunt systems: a meta- and cost-savings analysis Clinical article Paul Klimo Jr., M.D., M.P.H., 1 3 Clinton J. Thompson, M.S., 4 Brian T. Ragel, M.D., 5 and Frederick A. Boop, M.D Semmes-Murphey Neurologic & Spine Institute; 2 Department of Neurosurgery, University of Tennessee Health Science Center; 3 St. Jude Children s Research Hospital, Memphis, Tennessee; 4 School of Public Health and Health Services, George Washington University, Washington, DC; and 5 Department of Neurosurgery, Oregon Health & Science University, Portland, Oregon Object. Infection is a serious and costly complication of CSF shunt implantation. Antibiotic-impregnated shunts (AISs) were introduced almost 10 years ago, but reports on their ability to decrease the infection rate have been mixed. The authors conducted a meta-analysis assessing the extent to which AISs reduce the rate of shunt infection compared with standard shunts (SSs). They also examined cost savings to determine the degree to which AISs could decrease infection-related hospital expenses. Methods. After conducting a comprehensive search of multiple electronic databases to identify studies that evaluated shunt type and used shunt-related infection as the primary outcome, 2 reviewers independently evaluated study quality based on preestablished criteria and extracted data. A random effects meta-analysis of eligible studies was then performed. For studies that demonstrated a positive effect with the AIS, a cost-savings analysis was conducted by calculating the number of implanted shunts needed to prevent a shunt infection, assuming an additional cost of $400 per AIS system and $50,000 to treat a shunt infection. Results. Thirteen prospective or retrospective controlled cohort studies provided Level III evidence, and 1 prospective randomized study provided Level II evidence. Shunt infection was generally uniformly defined among the studies, but the availability and detail of baseline demographic data for the control (SS) and treatment (AIS) groups within each study were variable. There were 390 infections (7.0%) in 5582 procedures in the control group and 120 infections (3.5%) in 3467 operations in the treatment group, yielding a pooled absolute risk reduction (ARR) and relative risk reduction (RRR) of 3.5% and 50%, respectively. The meta-analysis revealed the AIS to be statistically protective in all studies (risk ratio = 0.46, 95% CI ) and in single-institution studies (risk ratio = 0.38, 95% CI ). There was some evidence of heterogeneity when studies were analyzed together (p = 0.093), but this heterogeneity was reduced when the studies were analyzed separately as single institution versus multiinstitutional (p > 0.10 for both groups). Seven studies showed the AIS to be statistically protective against infection with an ARR and RRR ranging from 1.7% to 14.2% and 34% to 84%, respectively. The number of shunt operations requiring an AIS to prevent 1 shunt infection ranged from 7 to 59. Assuming 200 shunt cases per year, the annual savings for converting from SSs to AISs ranged from $90,000 to over $1.3 million. Conclusions. While the authors recognized the inherent limitations in the quality and quantity of data available in the literature, this meta-analysis revealed a significant protective benefit with AIS systems, which translated into substantial hospital savings despite the added cost of an AIS. Using previously developed guidelines on treatment, the authors strongly encourage the use of AISs in all patients with hydrocephalus who require a shunt, particularly those at greatest risk for infection. (DOI: / PEDS11346) Key Words antibiotic cerebrospinal fluid shunt infection meta-analysis Hydrocephalus is one of the most common problems faced by neurosurgeons. Despite the application of endoscopic procedures for treatment, Abbreviations used in this paper: ACC = American College of Cardiology; AHA = American Heart Association; AIS = antibiotic-impregnated shunt; ARR = absolute risk reduction; ATS = American Thoracic Society; EVD = external ventricular drain; GRADE = Grades of Recommendation, Assessment, Development, and Evaluation; NNT = number needed to treat; RR = risk ratio; RRR = relative risk reduction; SS = standard shunt. most adults and children with hydrocephalus will require the implantation of a permanent CSF diversion device. Shunts have been used since the 1950s but continue to be plagued with complications, including infection. The rate of infection generally ranges from 5% to 15% but can be much higher in certain patient subgroups, such as neonates with posthemorrhagic hydrocephalus. 15,17,21,32,42,56 Shunt infection comes with obvious undesirable burdens to the patient, family, neurosurgeon, other health care providers, and the health care system, with both shortand long-term consequences. Infection often causes shunt 600 J Neurosurg: Pediatrics / Volume 8 / December 2011

2 Meta-analysis of shunt systems malfunction, placing the patient at risk for the potential consequences. It can also lead to scarring and loculation of the ventricles, making the patient s hydrocephalus more complex, and may result in a lower IQ, an increased risk of seizures, and psychomotor retardation. 13,36,44,69,71 Furthermore, the purported cost of treating a shunt infection is upward of $50,000 in the US, making it one of the most costly implant-related infections. 18 As such, the prevention of shunt infection should be paramount. A shunt-related infection, by definition, is any infection associated with the implantation of a shunt, with the most serious in terms of potential morbidity and mortality being infected CSF or ventriculitis. A wide range of practices has been designed to prevent shunt infection, 30 and rigid adherence to a shunt surgery protocol has repeatedly been shown to decrease shunt infection rates. 14,40,41,49,55 The most common pathogens in shunt infections are gram-positive skin organisms acquired at the time of surgery, namely Staphylococcus epidermidis and S. aureus. Although antibiotic-impregnated silastic catheters were first introduced by Roger Bayston in 1977 and then were considered more specifically with shunts in 1989, 10 these devices did not become available for clinical use until about 10 years ago. The first, and still the only available, AIS was introduced in This AIS is impregnated with 0.054% rifampin and 0.15% clindamycin (Bactiseal, Codman, Johnson & Johnson). Although it does not reduce bacterial adherence, this combination of antibiotics kills bacteria and thus prevents colonization by the most common pathogens for up to 56 days in in vitro studies and up to 127 days in vivo. 9,11,46 The AIS has also been shown to be nonepileptogenic. 1 Since the introduction of the AIS, there have been a number of studies evaluating its effectiveness compared with SSs. Some investigators have shown that the use of AISs decreases the risk of shunt infection, but others have not. Our primary objective in the present study was to combine data from existing studies to maximize their power to determine whether a difference in shunt infection rates truly exists, that is, to minimize the chance of incorrectly concluding that there is no difference. Secondly, in this era of escalating health care expenditures, we believe it is important to present cost-savings data on AIS systems. We hope to clarify whether AISs can lower the infection risk and identify the added costs and potential savings, so that surgeons can best determine whether it is clinically and economically indicated to use the AIS at their institutions. Methods Search Strategy Our systematic search strategy involved an electronic database search, a manual search of journals, examination of bibliographies of relevant articles, and consultation with the senior author (F.A.B.). We electronically searched MEDLINE (via NLM Gateway), PubMed, The Cochrane Library, Web of Knowledge, and Scopus to find English-language articles published from January 2000 to April 2011 while using the following terms in various J Neurosurg: Pediatrics / Volume 8 / December 2011 combinations: antibiotic-impregnated, shunt, catheter, system, infection, and hydrocephalus. Articles were also searched using the Related Articles function on PubMed and by reviewing the references from articles identified in the aforementioned searches. We excluded so-called grey literature, such as conference proceedings, abstracts, and trial registries. Inclusion Criteria, Data Extraction, End Points, and Definitions The goal of the search strategy was to identify studies published in the English language that satisfied the following criteria: 1) the study had a group of patients (adult or pediatric) that was treated with an AIS (treatment group); 2) the study had a group of patients (adult or pediatric) that was treated with an SS (control group); 3) the chosen implanted shunt system represented the only intentional treatment difference between the 2 patient groups (that is, no other changes were indicated, such as a difference in surgical technique); and 4) the minimum data included the total number of procedures performed in each group (treatment and control) and the number of shunt infections. Studies were excluded if they contained data that had been previously published (duplicated data) or if the authors used an AIS system other than the Codman Bactiseal system. Two individuals (P.K. and B.T.R.) independently screened all potential articles and extracted data from eligible articles. For all studies, we collected the following data (for the AIS and SS groups) if available, in addition to what was stated in the inclusion criteria above: study type, study population, number of patients, number of patients younger than 1 year, number of shunt operations, average age, age range, initial or revision operation, recent shunt infection (usually within the last 6 months), type of hydrocephalus (communicating, obstructive, or unclear), and cause of hydrocephalus (congenital; posthemorrhagic, including postsubarachnoid hemorrhage and germinal matrix hemorrhage of prematurity; spina bifida; normal pressure hydrocephalus; posttraumatic; tumor; and postmeningitic). Although the primary outcome for the purpose of this meta-analysis was shunt infection, its definition was, of course, determined by the authors of the studies that met our entry criteria. In general, a CSF shunt/catheterrelated infection was present if a patient had signs and symptoms of shunt malfunction or infection with an organism cultured from the CSF, shunt apparatus, purulence from the shunt wound(s), or abdominal fluid/pseudocyst. Some investigators also included patients in whom the clinical suspicion was very high (for example, raised CSF white blood cell count, clinical improvement after shunt removal, and treatment with antibiotic therapy) but positive CSF cultures were lacking. 33,38 The shunt infection rate was calculated per procedure, rather than per patient, for 2 reasons. First, we judged that it was more clinically relevant because some patients undergo multiple shunt revisions, and second, in some studies, the total number of patients was not provided. Each study that met our inclusion criteria was carefully reviewed independently, and the authors conclusions 601

3 P. Klimo Jr. et al. were verified based on the data provided. Disagreements in study selection and data abstraction were resolved through discussion. The quality of the evidence provided in each study was then graded I IV (Table 1). 4 We used 2 classification systems to grade the strength of our recommendations on the use of AISs based on the results of our meta-analysis (Tables 2 and 3). 58,68 The Meta-Analysis For each study, we identified the number of infections resulting from SSs and AISs and then computed the risk of an infection with the AIS relative to the SS, yielding an RR. An RR < 1 indicates protection against infection with the AIS. The overall risk ratio was computed using the method of DerSimonian and Laird. 20 We conducted a random effects meta-analysis of the selected studies. A random effects model as opposed to a fixed effects model does not assume that the measure of association (that is, the RR) is uniform across strata (that is, among studies) and consequently yields a more conservative estimate of the effect. We assessed heterogeneity using the chi-square test of heterogeneity and the I 2 statistic, where the former returns a chi-square distributed test statistic and corresponding p value and the latter returns a value bound between 0% and 100%, with higher values denoting increasing heterogeneity. We regarded a chi-square test of heterogeneity p value less than α = 0.10 and an I 2 value in the range of 30% 60% as suggestive of moderate heterogeneity. 16,19 To examine the source of heterogeneity, we categorized the studies based on their institutional status (single vs multiinstitutional) and analyzed each group separately (sensitivity analysis). We hypothesized that this stratification would account for some of the observed heterogeneity. We initially excluded studies that presented pooled data from multiple institutions because we judged that single-institution data were cleaner, that is, any difference in the infection rate between the treatment and control groups was more likely a result of the intervention (switching from an SS to an AIS) than a result of any one or more of the large number of variables that could positively or negatively affect the primary outcome (that is, confounders). Nonetheless, we reasoned that the data from the multiinstitutional studies were important enough to be included, although we elected to analyze them separately (see Results). We also assessed the presence of publication bias via a funnel plot. 22,66,67 All statistical analysis was conducted using Stata/SE 11.2 software. The Cost-Benefit Analysis For each study in which a statistically significant reduced infection rate was demonstrated for the AIS, we calculated an ARR and RRR. The number of AISs that would need to be implanted to prevent 1 shunt infection, or the number needed to treat (NNT), was calculated for each study as the inverse of the ARR. We then calculated the cost of preventing 1 shunt infection as the NNT multiplied by the additional cost of an AIS system, which is approximately $400. Next, assuming a cost of $50,000 to a treat shunt infection, we calculated the savings per NNT and the savings per annum, assuming that the institution performs 200 shunt operations per year. Results The initial search strategy identified 22 studies, but several articles were disqualified from analysis. Izci et al. 34 used a silver-impregnated polyurethane ventricular catheter that has not been evaluated by others and is not commercially available in the US. Two groups of authors presented similar data in multiple publications; we chose the publication that provided the most detailed data. The group from Johns Hopkins University has published at least 8 studies that detail their experience with AIS over different but overlapping time periods. 8,26,27,45,59 62 Two publications were selected for this analysis because they had a large number of patients over extended time periods, with each study focusing on pediatric or adult patients only. 26,45 Likewise, Eymann and colleagues 24,25 had 2 publications with shared data, and the one used for this meta-analysis was selected because it included both adult and pediatric patients and also had a cost-benefit analysis. Thus, 14 studies met our inclusion criteria. 2,7,24,26,29,31,33,37, 38,45,47,53,54,65 Note, however, that the study by Eymann et al. 24 had separate data for adult and pediatric patients, and thus each population was analyzed and listed separately. TABLE 1: Levels for classification of evidence Level I II III IV Definition prospective, randomized, controlled clinical trial w/ masked outcome assessment in representative population; requires clearly defined primary outcome(s), clearly defined exclusion/inclusion criteria, adequate accounting for dropouts & crossovers w/ numbers sufficiently low to have minimal potential for bias, & relevant baseline characteristics presented & substantially equivalent among treatment groups or appropriate statistical adjustment for differences prospective matched group cohort study in representative population w/ masked outcome assessment that meets requirements listed above OR a randomized controlled trial in representative population that lacks 1 of the criterion listed above all other controlled trials including well-defined natural history controls or patients serving as own controls in representative population in which outcome assessment is independently assessed or independently derived by objective outcome measures (that is, an outcome measure that is unlikely to be affected by an observer s (patient, treating physician, or investigator) expectation or bias (for example, blood tests or administrative outcome data) evidence from uncontrolled studies, case series, case reports, or expert opinion 602 J Neurosurg: Pediatrics / Volume 8 / December 2011

4 Meta-analysis of shunt systems TABLE 2: Strength of recommendation used by ACC and AHA Class I II a b III Definition conditions for which there is evidence &/or general agreement that given procedure or treatment is useful & effective conditions for which there is conflicting evidence &/or divergence of opinion about usefulness/efficacy of procedure or treatment weight of evidence/opinion in favor of usefulness/efficacy usefulness/efficacy less well established by evidence/opinion conditions for which there is evidence &/or general agreement that the procedure/treatment is not useful/effective & in some cases may be harmful Characteristics of Eligible Studies There were 2 prospective studies (1 controlled cohort and 1 randomized), 2 ambidirectional cohort studies (data collected both retro- and prospectively), and 10 retrospective cohort studies (Table 4). Eleven studies involved patients from a single institution, and 3 studies included data from multiple institutions. Two of these multiinstitutional studies were from the United Kingdom. Although the study by Richards et al. 53 used data from the UK Shunt Registry in which all major neurosurgery centers in the British Isles contribute, we could not confidently assume that the data from the 3 neurosurgery units presented in the Kandasamy et al. 38 study were also used in the Richards et al. study. Furthermore, Kandasamy et al. only reported on pediatric patients. Therefore, it was decided to include both studies. Seven studies contained both adult and pediatric patients, 5 had just pediatric patients, and 2 had only adults. All studies except 1 were graded as having Level III data. The study by Govender et al. 29 was a prospective randomized trial but was downgraded to Level II data quality because of serious methodological and data interpretation flaws, including the lack of clearly defined primary outcome and demographic data for the treatment and control groups (see Discussion). The demographic data for patients in the SS and AIS groups within each study are detailed in Tables 5 and 6. Tests for Evidence of Publication Bias There is a trend toward a modest publication bias in our analysis, although this bias was not statistically significant (p = 0.103). The absence of studies in the lower TABLE 3: Strength of recommendation used by the ATS and the GRADE system Grade of Recommendation Clarity of Risk/Benefit Implications strong recommendation w/ high-quality evidence moderate-quality evidence low-quality evidence very-low-quality evidence weak recommendation w/ high-quality evidence moderate-quality evidence low-quality evidence very-low-quality evidence benefit clearly outweighs harms & burdens or vice versa benefits closely balanced w/ harms & burdens benefits closely balanced w/ harms & burdens uncertainty in estimates of benefits, harms, & burdens; benefits may be closely balanced w/ harms & burdens major uncertainty in estimates of benefits, harms, & burdens; benefits may or may not be balanced w/ harms & burdens recommendation can apply to most patients in most circumstances; further research very unlikely to change confidence in estimate of effect recommendation can apply to most patients in most circumstances; further research (if performed) likely to have important impact on our confidence in estimate & may change estimate recommendation may change when higher-quality evidence becomes available; further research (if performed) likely to have important impact on confidence in estimate & may change estimate recommendation may change when higher-quality evidence becomes available; any estimate of effect, for at least 1 critical outcome, very uncertain best action may differ depending on circumstances or patients or societal values; further research very unlikely to change confidence in estimate of effect alternative approaches likely better for some patients under some circumstances; further research (if performed) likely to have important impact on confidence in estimate of effect & may change estimate other alternatives may be equally reasonable; any estimate of effect, for at least 1 critical outcome, very uncertain other alternatives may be equally reasonable; any estimate of effect, for at least 1 critical outcome, very uncertain J Neurosurg: Pediatrics / Volume 8 / December

5 P. Klimo Jr. et al. TABLE 4: Summary of studies used in meta-analysis Authors & Year Study Design Study Population Study Site Farber et al., 2011 retrospective adult single institution III Steinbock et al., 2010 prospective pediatric + adult multiinstitutional, multinational III Gutiérrez-González et al., 2010 retrospective pediatric + adult single institution III Kandasamy et al., 2011 ambispective pediatric multiinstitutional, uninational III Richards et al., 2009 retrospective matched pair pediatric + adult multiinstitutional, uninational III Parker et al., 2009 retrospective pediatric single institution III Albanese et al., 2009 retrospective adult single institution III Eymann et al., 2008 retrospective pediatric + adult single institution III Hayhurst et al., 2008 retrospective pediatric single institution III Pattavilakom et al., 2007 ambispective pediatric + adult single institution III Ritz et al., 2007 retrospective pediatric + adult single institution III Kan & Kestle, 2007 retrospective pediatric single institution III Aryan et al., 2005 retrospective pediatric single institution III Govender et al., 2003 prospective, randomized, blinded pediatric + adult single institution II Data Level right quadrant of the funnel plot (Fig. 1) indicates that small, negative studies have not been published and thus are not included in this meta-analysis. Meta-Analysis: Shunt Infection and Sensitivity Analysis Among the 14 studies, 7 showed AISs to be protective in preventing a shunt malfunction and 7 documented no statistical benefit (Table 7). There were 5582 procedures involving a standard catheter system and 390 infections, yielding a pooled infection rate of 7.0%. In the population receiving AISs, there were 120 infections among 3467 shunt operations, for an overall infection rate of 3.5%. When analyzing only the single-institution studies (12 study populations), the overall RR was 0.38 (95% CI , p < 0.001; Fig. 2). In other words, a shunt infection was 2.63 times more likely when using an SS than an AIS. If all studies were included (15 studies, including both data sets from Eymann et al. 24 ), then the overall RR was 0.46 (95% CI , p < 0.001), making shunt infection 2.18 times more likely with an SS system. There was evidence of some heterogeneity when all of the studies were analyzed together (Q = 21.33, df = 14, p = 0.093), but when we examined the studies according to their institutional status, the heterogeneity was reduced to statistically nonsignificant levels (p > 0.10 for both single- and multiinstitutional studies). The stratification of our analysis a sensitivity analysis according to study institutional status indicated that the observed heterogeneity was partially explained by institutional status. Furthermore, the I 2 statistic decreased from 34.4% for all studies to 29.0% for single-institution studies, suggesting that the institutional status accounted for some of the overall heterogeneity, with the remaining heterogeneity attributable to differences between the single-institution studies. Our recommendation on the use of AISs is strong evidence with low- to moderate-quality evidence based on the GRADE system/ats guidelines and Class IIa evidence based on the ACC/AHA guidelines. Cost-Savings Analysis As the difference in infection rates between patients who had an SS and those who had an AIS increases, the NNT to prevent 1 shunt infection consequently decreases (Table 8). For example, the study by Gutiérrez-González et al. 31 showed a decrease in the infection rate from 17% to 2.8%, yielding an NNT of 7. A lower NNT translates into a lower additional cost for switching to the AIS (assuming an additional hospital cost of $400 per AIS kit). Assuming a cost of $50,000 to treat a shunt infection, the cost savings per shunt infection prevented for the various studies is shown in Table 8. The estimated annual savings, assuming 200 shunt operations performed, ranges from just under $90,000 to well over $1.3 million. Discussion Literature Review Of the 14 studies that satisfied our inclusion and exclusion criteria, 7 revealed AISs to be statistically protective, 24,26,31,38,45,47,53 whereas 7 did not. 2,7,29,33,37,54,65 Among the studies with negative statistical findings, one 2 had such a small number of patients (18 patients, 6 with AISs) that no conclusion could be made, although the authors believed that the AIS could be effective. The authors of 2 studies 29,33 made somewhat misleading comments in their respective abstracts that required us to classify them as finding no benefit with the AIS. Hayhurst et al. 33 stated that AIS catheters can reduce the number of shunt infections and had a significant impact on the neonatal hydrocephalic population, when in fact their results showed no difference in the shunt infection rate overall and within any subgroup, including neonates. Proponents of AISs have given much credence to the prospective randomized trial by Govender et al. 29 These authors provided a definition of shunt infection, even differentiating between an internal and an external infection, and described their inclusion and exclusion criteria, surgical 604 J Neurosurg: Pediatrics / Volume 8 / December 2011

6 Meta-analysis of shunt systems TABLE 5: Characteristics of patients who received an SS* Authors & Year No. of Pts No. of Shunt Ops Mean (yrs) Pt Age Sex (no.) No. <1 Yr Range M F Initial No. of Shunt Procedures Revision No. of Prior Shunt Infections Type of Hydrocephalus (no.) Etiology of Hydrocephalus (no.) Noncomm Comm Unclear Congenital PHH Spina Bifida NPH Tumor Trauma Meningitis Farber et al., NA NA NA NA NA NA NA NA NA NA 195 NA NA NA Steinbock et al., NA NA NA NA NA NA NA NA NA Gutiérrez-González et al., NA 8 NA NA NA NA NA 8 NA NA Kandasamy et al., 2011 NA NA 1 d 16 yrs NA NA NA NA NA NA NA NA NA NA NA NA NA Richards et al., NA NA NA NA NA NA NA Parker et al., 2009 NA NA 1 d 20 yrs NA NA 36 Albanese et al., yrs Eymann et al., yrs NA NA NA NA NA NA 71 NA NA NA Eymann et al., NA 1 d 72 mos NA NA NA NA NA NA NA NA Hayhurst et al., NA 11 NA NA NA NA NA NA NA NA NA NA 0 NA NA NA Pattavilakom et al., NA 551 NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA 2007 Ritz et al., NA NA NA NA NA 50/ /172 8/172 NA NA NA NA NA NA NA Kan & Kestle, NA NA NA 8 NA NA NA NA NA NA Aryan et al., 2005 NA 46 NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA Govender et al., NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA * Comm = communicating; d = day; NA = not available; Noncomm = noncommunicating; NPH = normal pressure hydrocephalus; PHH = posthemorrhagic hydrocephalus; Pt = patient. Less than or equal to 6 months of age. Data are from a single paper, but the authors analyzed pediatric and adult patients separately, and so the data are listed separately. J Neurosurg: Pediatrics / Volume 8 / December

7 P. Klimo Jr. et al. TABLE 6: Characteristics of patients who received an AIS Authors & Year No. of Pts No. of Shunt Ops Mean (yrs) Pt Age Sex (no.) No. <1 Yr Range M F Initial No. of Shunt Procedures Revision No. of Prior Shunt Infections Type of Hydrocephalus (no.) Etiology of Hydrocephalus (no.) Noncomm Comm Unclear Congenital PHH Spina Bifida NPH Tumor Trauma Meningitis Farber et al., NA NA NA NA NA NA NA NA NA NA 183 NA NA NA Steinbock et al., NA NA NA NA NA NA NA NA NA NA Gutiérrez-González NA 6* NA NA NA NA NA 12 NA NA et al., 2010 Kandasamy et al., NA d 16 yrs NA NA NA NA NA NA NA NA NA NA NA NA NA 2011 Richards et al., NA NA NA NA NA NA NA Parker et al., 2009 NA NA 1 d 19.8 yrs NA NA 32 Albanese et al., yrs Eymann et al., yrs NA NA NA NA NA NA 135 NA NA NA 2008 Eymann et al., NA mos NA NA NA NA NA NA NA NA 2008 Hayhurst et al., NA 33 1 d 16 yrs NA NA NA NA NA NA NA NA NA 0 NA NA NA 2008 Pattavilakom et al., NA 23 NA NA NA NA NA NA NA 53/178 33/178 0/178 14/ NA Ritz et al., NA NA NA NA NA 17/86 68/86 1/86 NA NA NA NA NA NA NA Kan & Kestle, NA NA NA 16 NA NA NA NA NA NA Aryan et al., NA 6 mos 17 yrs NA NA NA NA NA NA NA NA NA NA NA NA NA Govender et al., NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA NA 2003 * Less than or equal to 6 months of age. Data are from a single paper, but the authors analyzed pediatric and adult patients separately, and so the data are listed separately. 606 J Neurosurg: Pediatrics / Volume 8 / December 2011

8 Meta-analysis of shunt systems Fig. 1. Funnel plot with pseudo 95% confidence limits showing unequal distribution of studies, indicative of a lack of small negative studies within the literature. S.E. = standard error. procedure, sample size calculation, and follow-up protocol. Nevertheless, their study suffered from a number of critical flaws. The investigators failed to provide a clear definition of their primary end point; they did not discuss the method of randomization and whether any known shunt infection risk factors, such as prematurity, would be controlled for in the randomization process or data analysis; and they did not provide the demographic makeup of the treatment or control groups to demonstrate whether they were balanced. They focused their conclusion on their finding of a decreased shunt infection rate in the first 2 months in the AIS group, but the overall shunt infection rate, which we believe is more clinically relevant, was not statistically different between the AIS (5%) and SS (13.3%) groups. Furthermore, this study was conducted in a region of South Africa where the prevalence of HIV is the highest, and the patients, as stated by the authors, often have an extremely poor socioeconomic status, are severely malnourished, and have poor immunocompetency. These factors, therefore, limit the external validity, or generalizability, of this study as compared with other studies in the literature from more socioeconomically advanced countries. There was a mild publication bias in the articles used in the meta-analysis, as depicted by the lack of uniform distribution within the inverted V of the funnel plot (Fig. 1), which suggests that small, negative studies have not been published in the literature. Furthermore, as mentioned in the Results, authors from Johns Hopkins University have published multiple articles with duplicate data, all of which have revealed a positive effect with AISs. 8,26,27,45,59 62 This duplication has the effect of flooding the literature with results that could have been coalesced into fewer studies. To control for this effect, we selected only those studies that had distinct data 1 study that had adult patients and 1 study that contained only pediatric patients. Even more concerning is the fact that 2 authors who are associated with all of the studies are self-admitted paid consultants for Codman, which could raise questions regarding the unbiased nature of the studies and results. Meta-Analysis and Limitations Our meta-analysis showed AISs to be protective against shunt infection. The pooled infection rate de- TABLE 7: Number of shunt infections and rates per study SS % Infected Per Procedure AIS % Infected Per Procedure Authors & Year No. of Pts Infected No. of Pts Infected Study Conclusions* Farber et al., AIS reduced shunt infection rate Steinbock et al., no difference in shunt infection rate Gutiérrez-González et al., AIS reduced shunt infection rate Kandasamy et al., AIS reduced shunt infection rate Richards et al., AIS reduced shunt infection rate Parker et al., AIS reduced shunt infection rate Albanese et al., no difference in shunt infection rate Eymann et al., AIS reduced shunt infection rate & resulted in significant hospital cost savings Eymann et al., AIS reduced shunt infection rate & resulted in significant hospital cost savings Hayhurst et al., no difference in shunt infection rate Pattavilakom et al., AIS reduced shunt infection rate Ritz et al., no difference in shunt infection rate Kan & Kestle, no difference in shunt infection rate Aryan et al., no difference in shunt infection rate Govender et al., no difference in shunt infection rate * Conclusions put forth by authors of the individual papers were not automatically accepted and were verified by 2 authors (P.K. and B.T.R.) of the present paper. Data are from a single paper, but the authors analyzed pediatric and adult patients separately, and so the data are listed separately. J Neurosurg: Pediatrics / Volume 8 / December

9 P. Klimo Jr. et al. Fig. 2. Forest plots of all studies (multi- and single-institution) with their respective RRs, events (infections), treatments (procedures), and cumulative RRs. creased from 7.0% in the patients in the SS group to 3.5% in those in the AIS group ARR and RRR of 3.5% and 50%, respectively. Regardless of whether all studies were included or only those from a single institution, the odds of a shunt infection developing was more than 2 times greater in patients with an SS than in those who received an AIS. Since a meta-analysis is a summation of trials, it is only as good as the trials that are combined within it. Although the trials used in this meta-analysis shared certain core components, as defined by our inclusion and exclusion criteria, and all had Class III data, with the exception of 1 study with Class II data, there was a considerable degree of heterogeneity among them. For example, some studies included only adult patients, others had only children, and still others included both. Surgical technique is obviously a factor that cannot be controlled for in such an analysis and, as discussed previously, may be somewhat more standardized in a study from a single institution than in a multiinstitutional study. Even the definition of a shunt infection was not identical from study to study. The availability of composition data for the control and treatment groups within each of the studies varied considerably, even for seemingly basic data such as sex and age (Tables 5 and 6). Some studies gave little to no compositional data for the treatment or control groups. 7,47,54 The most important implication regarding group composition TABLE 8: Cost analysis for studies that demonstrated a benefit in switching from an SS to an AIS Authors & Year SS Infection Rate (%) AIS Infection Rate (%) ARR (%) RRR (%) NNT Cost to Prevent 1 Shunt Infection (US$)* Cost Savings Per NNT (US$) Cost Savings Per Annum (US$) Farber et al., ,400 35, ,360 Gutiérrez-González et al., ,800 47,200 1,349,920 Kandasamy et al., ,800 35, ,080 Richards et al., ,600 26,400 89,760 Parker et al., ,200 44, ,920 Eymann et al., ,600 38, ,960 Eymann et al., ,000 46, ,000 Pattavilakom et al., ,600 42, ,200 * NNT $400. $50,000 (NNT $400). (200/NNT) [$50,000 (NNT $400)]. Data are from a single paper, but the authors analyzed pediatric and adult patients separately, and so these data are listed separately. 608 J Neurosurg: Pediatrics / Volume 8 / December 2011

10 Meta-analysis of shunt systems is whether collectively the treatment and control groups are balanced for some of the known prognostic factors (that is, confounders) for shunt infection. Various primarily nonmodifiable preoperative patient characteristics are thought to be risk factors for shunt infection. Newborns (younger than 6 12 months of age) and premature infants in particular (< 40 weeks gestation), with their immature immune systems, thin skin, and high bacterial skin flora, have frequently been shown in the literature to be high-risk groups, with infection rates of 10% 15% or higher. 3,23,28,43,50,63,70 A few reports have not shown age to be a risk factor. 42,65 Simon et al. 63 also identified female sex, African-American race, public insurance, cause of intraventricular hemorrhage, and respiratory complex chronic condition as risk factors. Kestle et al. 39 found an alarmingly high overall reinfection rate of 26% in patients who were treated for a recent CSF shunt infection and 29% in those infected with S. epidermidis. Ritz et al. 54 assumed a number of shunt infection risk factors as part of their data analysis, including age (< 1 and > 80 years), premature birth, EVD, former shunt infection, former systemic infection, disturbance of consciousness, and former radiation or chemotherapy. Prusseit et al. 51 listed a number of confirmed risk factors, which included among others low gestational age and preterm birth, young age at shunt placement, and cause of hydrocephalus (increased risk after intraventricular hemorrhage, infectious etiology, or children with malignant disease, chemotherapy-associated immunosuppression, or long-term application of steroids). High-risk subgroups as defined by Parker et al. 45 were characterized by prematurity (< 35 weeks gestational age), placement of shunts immediately after meningitis, conversion of an EVD to a shunt, and shunt replacement due to nosocomial infection in patients requiring prolonged hospital stays (> 1 month). Pattavilakom et al. 47 listed similar risk factors such as cause of hydrocephalus, previous revisions, extended hospital stay, positive CSF cultures prior to implantation, and the preoperative occurrence of CSF leakage or the use of an EVD. What this demonstrates is that there are undoubtedly a number of known and unknown preoperative risk factors or confounders for shunt infection, with varying degrees of agreement among neurosurgeons collectively. It is impossible based on the studies that qualified for our meta-analysis to know whether the SS and AIS groups were balanced with respect to even the more commonly cited risk factors simply because such data were not available in all studies. Our hope is that with so many patients and procedures in each treatment group (5582 and 3467 procedures in the SS and AIS groups, respectively), the influence that any differences between the groups as regards known and unknown confounders would be lessened, and thus the statistically significant and substantial reduction in the infection rate that we demonstrated is a true finding and not a false positive or a Type I error. Steps may be taken at the design stage or in the analysis stage of a clinical study to reduce the impact of disproportionately distributed confounders. Matching groups on certain key confounders, as done in the study by Richards et al., 53 can eliminate the impact of only those confounders J Neurosurg: Pediatrics / Volume 8 / December 2011 that were matched, but matching on multiple confounders in a large cohort trial can be economically and logistically impracticable. Although some authors have called for a prospective, blinded, randomized controlled trial, 37 such an analysis would require, as correctly stated by Richards et al., 53 very large patient numbers, which would necessitate multicenter cooperation, the establishment of a standard protocol, and considerable funding. For example, if we assumed that the shunt infection rates in our meta-analysis were true, with a b of 20% and an a of 5%, more than 500 patients would be needed in each group. More importantly, it may be difficult to recruit centers into such a trial because, anecdotally, some neurosurgeons and centers have developed a strong bias toward using the AIS (that is, lack of clinical equipoise). It is easy to understand why. For the neurosurgeon, converting to the AIS requires no change in surgical technique or added surgical time and may reduce the risk of what is arguably one of the most adverse complications of shunt surgery. Furthermore, there have been no reported deleterious consequences of implanting an AIS. Although there has been some concern that AIS systems may mask or delay shunt infections or even increase the rate or virulence of such an infection, Sciubba et al. 60 showed that AISs did not increase the incidence of late CSF shunt infection. There have been no reports of postsurgical hypersensitivity, and Abed et al. 1 demonstrated that AISs are nonepileptogenic. Nonetheless, the Hydrocephalus Clinical Research Network developed and implemented a standardized shunt surgery protocol that excluded the use of AISs in 4 centers with 21 neurosurgeons. 41 In the absence of a well-designed prospective cohort trial or randomized trial, we believe that our meta-analysis provides the best evidence-based appraisal of the current literature. Using the ATS/GRADE system, we strongly recommend the use of AISs based on low- to moderatequality evidence (Class II and III data). Similarly, we give our findings a Grade IIb on the ACC/AHA scale, which indicates that, although there is some conflicting evidence, the weight of the evidence favors AIS use. Cost Analysis and Limitations The implantation of shunts is a very common procedure and thus uses tremendous monetary resources. Each year, more than $2 billion dollars are spent treating pediatric hydrocephalus, with an estimated cost of $36,000 $40,000 per admission. 48,64 The cost to treat a shunt infection may be upward of $50,000 or more. 18 Therefore, the impact of a measure to decrease the risk of shunt infection can translate into substantial health care savings. The greater the risk reduction with a conversion to AISs, the lower the number of AIS implants needed to prevent 1 infection and thus the less additional cost to prevent 1 shunt infection, assuming the commonly quoted additional cost of $400 for an AIS system (at the primary author s hospital [P.K.], the difference is $412.48). With a lower NNT and thus lower additional expenditure to prevent 1 shunt infection, the cost savings per NNT (assuming a cost of $50,000 to treat a shunt infection) and cost savings per annum (assuming 200 shunt operations/year) increases accordingly. The savings per annum 609

11 P. Klimo Jr. et al. in trials that showed a benefit with AIS systems ranged from just less than $90,000 to over $1.3 million (Table 8). The cost savings are dependent on several factors and assumptions. The biggest factor is the decrease in the shunt infection rate with a change to the AIS (ARR). In some institutions, the shunt infection rates are so low that converting to AISs would not be financially worthwhile. Choux et al. 14 reported a per-procedure rate of 0.17% after the introduction of a strict protocol for shunt surgery. With the addition of intrathecal vancomycin and gentamycin at the time of surgery, Ragel et al. 52 saw their infection rate fall to 0.4%. Pirotte et al. 49 reported the lowest infection rate in the literature, 0%, in a consecutive series of 100 patients undergoing 115 surgeries with the implementation of their own perioperative protocol. Thus, if an institution s shunt infection rate is already well below 5% with the use of standard catheters, converting to AISs may not be cost effective or may best be limited to patients at greatest risk for shunt infection (for example, premature infants). A significant assumption in our analysis was the cost of treating a shunt infection. Darouiche 18 estimated the medical and surgical cost of treating a shunt infection to be $50,000. Attenello et al. 8 recently reported the average hospital cost per shunt infection for AIS and SS catheters as $46,640 and $49,397, respectively. At the primary author s institution (P.K.), the average cost in 2010 was $51,741. The cost to treat a shunt infection is dependent on 2 primary variables: how the shunt infection is treated and the health care system under which the patient is treated. There is no uniform method or duration of treatment for shunt infection, 6,39 but one of the more common procedures involves the removal of all hardware at the time of diagnosis, placement of an EVD for a period of several days or weeks while the patient is treated with intravenous (and possibly intraventricular) antibiotics until the infection clears, and reimplantation of a new shunt. 35,39,57 Other authors have reported successful outcomes with externalization of the shunt and treatment with systemic and intraventricular antibiotics followed by implantation of a new shunt 5 or in situ treatment with systemic and intraventricular antibiotics in patients with coagulase-negative staphylococci without externalization or replacement of the hardware. 12 As an example of the difference in hospital costs between markedly different health care systems, Eymann et al. 24 reported a much lower average cost of $17,300 and $13,000 for children and adults, respectively, under the socialized government-run German health care system. Even with a much lower cost in treating a shunt infection, they still saved approximately $50,000 with nearly 200 shunt operations. We chose $50,000 as the total cost because we believe it most likely reflects the cost of a shunt infection in the US. Annual cost savings were calculated based on 200 shunt operations being performed during that year, a number we thought was reasonable for a typical high-volume children s hospital where the impact of savings would be the greatest. Conclusions Shunt infections can have long-term consequences to the patient and impose significant burdens on the family, neurosurgeon, and health care system. The prevention of shunt infection is therefore critically important. The AIS system has been a welcome addition to the treatment of patients with hydrocephalus, but its effectiveness remains unclear. While recognizing the variable quality of existing literature, the lack of a uniform definition of shunt infection, and a possible publication bias, we have nonetheless shown in our meta-analysis the protective effect of AISs. The infection rate decreased from 7.0% with SSs to 3.5% with AISs. When all 14 studies were included in our analysis, the risk of developing a shunt infection with an SS was 2.18 times greater than that with an AIS. The protective effect of the AIS translated into a significant per annum cost savings, ranging from $90,000 to over $1.3 million. Economically, the decision to convert to an AIS system must be institution-based and is dependent on the baseline shunt infection rate, the estimated change with conversion to an AIS, the average hospital costs for the treatment of a shunt infection, and the number of shunt operations performed at the institution. Unless an institution s shunt infection rate is already well below 5%, we believe that an AIS should strongly be considered in all patients, especially in those who have the highest risk of shunt infection. Disclosure The authors report no conflict of interest concerning the materials or methods used in this study or the findings specified in this paper. Author contributions to the study and manuscript preparation include the following. Conception and design: Klimo, Thompson. Acquisition of data: Klimo, Ragel. Analysis and interpretation of data: Klimo, Thompson. Drafting the article: Klimo. Critically revising the article: all authors. Reviewed submitted version of manuscript: all authors. Approved the final version of the manuscript on behalf of all authors: Klimo. Statistical analysis: Thompson. Study supervision: Boop. References 1. Abed WT, Alavijeh MS, Bayston R, Shorvon SD, Patsalos PN: An evaluation of the epileptogenic properties of a rifampicin/ clindamycin-impregnated shunt catheter. Br J Neurosurg 8: , Albanese A, De Bonis P, Sabatino G, Capone G, Marchese E, Vignati A, et al: Antibiotic-impregnated ventriculo-peritoneal shunts in patients at high risk of infection. Acta Neurochir (Wien) 151: , Ammirati M, Raimondi AJ: Cerebrospinal fluid shunt infections in children. A study on the relationship between the etiology of hydrocephalus, age at the time of shunt placement, and infection rate. Childs Nerv Syst 3: , Armon C, Evans RW: Addendum to assessment: Prevention of post-lumbar puncture headaches: report of the Therapeutics and Technology Assessment Subcommittee of the American Academy of Neurology. Neurology 65: , Arnell K, Enblad P, Wester T, Sjölin J: Treatment of cerebrospinal fluid shunt infections in children using systemic and intraventricular antibiotic therapy in combination with externalization of the ventricular catheter: efficacy in 34 consecutively treated infections. J Neurosurg 107 (3 Suppl): , Arthur AS, Whitehead WE, Kestle JR: Duration of antibiotic therapy for the treatment of shunt infection: a surgeon and patient survey. Pediatr Neurosurg 36: , Aryan HE, Meltzer HS, Park MS, Bennett RL, Jandial R, Levy ML: Initial experience with antibiotic-impregnated sili- 610 J Neurosurg: Pediatrics / Volume 8 / December 2011