Author's response to reviews Title: Pneumococcal vaccination and otitis media in Australian Aboriginal infants: comparison of two birth cohorts before and after introduction of vaccination Authors: Grant A Mackenzie (gmackenzie@mrc.gm) Jonathan R Carapetis (jonathan.carapetis@menzies.edu.au) Amanda J Leach (amanda.leach@menzies.edu.au) Peter S Morris (peter.morris@menzies.edu.au) Version: 3 Date: 18 December 2008 Author's response to reviews: see over
17 th December 2008 The Editor BMC Paediatrics To the Editor, Re: Response to reviewers of manuscript submitted for publication I thank the journal for its consideration of the manuscript. I thank all of the reviewers for their time and comments. In summary, the primary concerns related to appropriate interpretation of potential bias and confounding and the size of effects that have been observed. I emphasise the strengths of the study being the first report of pneumococcal vaccine effectiveness against all cause tympanic membrane perforation and the use of reliable and consistent measurement of ear disease status throughout the study. I will address the comments of the reviewers point by point. Reviewer 1: Major Compulsory Revisions 1. I share the reviewer s concerns that the limitations of the study are properly conveyed. I have revised the conclusion section of the abstract including the sentences; Results of this study should be interpreted with caution due to potential bias and confounding. It appears that... I have revised the conclusions including the sentences; Due to potential confounding and bias, the results of this study should be interpreted with caution. and Of note, the lack of vaccine effectiveness to substantially reduce proportions experiencing different OM outcomes and delay onset of OM does not imply evidence of no effect. (p.16, final paragraph, 1 st and 3 rd sentence). I agree there is potential confounding due to differences in background incidences in the two cohorts that are being compared. In order to understand the significance of this potential confounding I have highlighted the following: a. Inclusion of an ecological comparison of census data (p.10, 2 nd paragraph) contextualising some of the circumstances (income, employment, proportion of population aged 0 5 years and household occupancy) in which the Tiwi people have lived during the period of observation. b. Inclusion of partial adjustment for temporal trends. The a priori inclusion in multivariate analyses of rate of non respiratory illness (p. 9, lines 3 5; p.11, final line; Table 4) addresses, albeit incompletely, potential confounding due to temporal trends in factors associated with otitis media and general paediatric illness. Adjusting for rate of non respiratory illness
adjusts for measured levels of common paediatric illness, using a variable which would not be expected to change due to pneumococcal vaccination, but which may be expected to change due to temporal changes in factors affecting background incidences of illness. Thus, concern about using historical controls is lessened as analysis for the effect of pneumococcal vaccination has held steady the effect of background incidences of measured illnesses. c. Results suggest that temporal changes in factors affecting otitis media and general child health are not substantial. On p.16, 1 st sentence, I have discussed historical controls, emphasising the fact that if temporal changes were substantial it would have been expected that changes in clinical characteristics may have been evident in Table 2, but this was not the case. Values in Table 2 suggest that the vaccinated group may have experienced less illhealth than the comparison group. Thus, concerns about potentially erroneous, confounded, conclusions would be valid if positive conclusions about vaccine effectiveness were made, however the conclusions made in this study are of a non significant effect of pneumococcal vaccination on rates and prevalence of otitis media (p.16, line 4). Of course, it is still possible that unmeasured factors may still have exerted an effect. d. Concerning sample size, I agree with the reviewer that there are issues of sample size concerning this study. Combining our expectation of substantial vaccine effects against perforation with logistic and funding limitations, our study was designed to detect large effects. A larger study would have been needed to detect with certainty the observed maximum difference in TMP prevalence of 9% (Table 4). Power of 77%, to detect a significant difference in time to first OME, is not unreasonable and I have included a power calculation for detection of a reduced proportion of participants experiencing TMP (p. 9, 1 st paragraph, final sentence). The baseline proportion is derived from the presented data and the expected 50% reduction is based on documented efficacy of 100% against TMP associated with vaccine serotype pneumococci excluding 19F, and 64% against TMP associated with any vaccine serotype pneumococci (Black et al 2000). According to these sample size calculations, the sample included in this study is acceptable for detection of large differences in time to first OME and the prevalence of TMP, and so the conclusion that vaccination was not associated with large effects is reasonable. 2. The reviewer has identified a lack of clarity concerning enrolment. Enrolment of comparison participants was not at the time of occurrence of first OME and enrolment of vaccinated participants was not at the time of vaccination. I have added a sentence that provides clarity concerning enrolment of the vaccinated group (p.6, 2 nd paragraph, line 5). It should now be clear that both groups were enrolled as soon as possible after birth, and examined every two weeks until OME was detected. Minor Essential Revisions 3. To improve the readability of the results section I have included additional, more explicit headings for the subsections, for example, Enrolment and Comparability of Groups, Prevalence of Otitis Media Outcomes in the First Year of Life, and Time to First Otitis Media Outcomes. This gives guidance to the reader concerning the information they are about to
read. I have revised the title and column headings of Table 4 which should more simply and better explain the content of the table. I have also revised the column headings of Table 5 and given further explanation in the table footnote. 4. The observation of the reviewer concerning Fig 3 is well noted and appropriate changes have been made to the text on p.13 and 14. 5. The reviewer has noted the difference between the groups in rates of lower respiratory tract infection (Table 2) and suggests that this should be discussed. I have included the following sentence on p.10, 1 st paragraph, final sentence; As the reduced rate of lower respiratory illness among vaccinees may be attributable to pneumococcal vaccination, and not potential confounding factors, the other characteristics in Table 2 may still be used for comparison of the two groups. 6. A high rate of sick clinic visits is noted with a request for verification. These values had been recorded as sums of sick and well clinic visits and I can verify that they are correct, children do visit the health clinic 10 20 times in the first 6 months of life (Table 2). Indeed, other studies in the NT have documented even rates of sick clinic visits. 7. A definition of Absolute Rate Reduction has been included as a footnote in Table 5. 8. The highlighted value in Table 5 has been revised to include the /yr indication of rate. 9. The numbers of mixed H. influenzae and S. pneumoniae infections in Table 6 are included in the numbers for single pathogen infections. This is now described in a footnote to Table 6. Discretional revisions 10. As suggested, I have made more explicit the duration of therapy in the OM RCT; p.5, 2 nd paragraph, line 2, now describes the therapy in the OM RCT as long term while the duration of follow up is specified on p.10, 1 st paragraph. 11. A justification for inclusion of all OM RCT participants in the time to first OME analysis and only placebo participants for other outcomes has been added on p.5, 2 nd paragraph, As subjects enrolled in the OM RCT began randomised therapy after detection of the first OME episode, the time to first OME outcome included all subjects enrolled in the OM RCT. 12. Regarding replacement disease with H. influenzae, the statement on p.15, line 5 7, is as follows, Although our findings do not support substantial replacement OM disease due to H. influenzae, they are consistent with US reports documenting potential H. influenzae replacement disease following 7PCV. I thank Reviewer 2 for their time and comments. Reviewer 2: Major Compulsory Revisions 1. I thank the reviewer for highlighting the inconsistency between the Cox model results and the legends describing the K M curves. The legends for the K M curves for time to first event (Figure
2) have been revised and now correctly reflect the results reported by Cox proportional hazards models in Table 4. 2. The reviewer has requested modification of the abstract and conclusions to acknowledge a limited power to conclude that there is no effect. I have included the following sentend in the Conclusions (p.16, final sentence), Of note, the lack of vaccine effectiveness to substantially reduce proportions experiencing different OM outcomes and delay onset of OM does not imply evidence of no effect. I have also revised the abstract, including the following, Results of this study should be interpreted with caution due to potential bias and confounding. It appears that introduction of pneumococcal vaccination among Aboriginal infants was not associated with significant changes in prevalence or age of onset of different OM outcomes or the incidence of AOM or TMP. 3. The reviewer has raised an issue of appropriate inclusion of variables in statistical modelling and controlling for events that occur after beginning of follow up or limiting variables in the model to those present at baseline. Although the issue is noted, it is a valid alternative and accepted epidemiological practice in cohort studies, that events occurring during follow up which may influence the outcome are measured and included in analyses as potentially confounding factors. An important example in our study would be that of antibiotic prescriptions during follow up. Importantly, variables included in analyses were based on an a priori decision making. Other suggested revisions 1. The reviewer has suggested discussing the issue of including the 23 valent polysaccharide vaccine as a booster in this study as opposed to a fourth dose of 7PCV. I have added a sentence to the discussion on this topic (p.15, 1st paragraph, penultimate and final sentence). 2. I note the other comments made by the reviewer. I thank Reviewer 3 for their time and comments. Reviewer 3: 1. The reviewer highlights the inappropriate inclusion in results of participants from the comparison group who were excluded from the OM RCT as they had a diagnosis of CSOM at initial examination. The sentence reporting this data has been modified as follows, Seven children excluded from the OM RCT due to a diagnosis of CSOM at initial assessment[16] were not included in this study, although their exclusion may have reduced the proportion of comparison participants at risk of developing CSOM. 2. The reviewer has noted the different proportions of participants in the two groups that are included in some analyses. The reviewer has suggested that the different proportions may be due to differential drop out between the two groups. In fact, these differences are due to
analysis inclusion criteria, and in particular the criteria that data would only be included if the first examination was performed at less than a maximum age. This was done in survival analyses to increase the precision of the time to first event estimates and as age at first examination predicted the age at first event, it was included in modelling to control for age at first examination. A similar justification for this approach was also used for incidence analyses, and sensitivity analyses were conducted to explore the effect of relaxing these inclusion criteria (p.12 and Table 5). Sensitivity analyses produced altered estimates and so the a priori analyses strategy was maintained and reported. 3. a. The reviewer notes possible differences in AOM, TMP, and CSOM that appear to dissipate over the first 18 months of life (Table 3) and asks if proof can be provided about the significance of these differences. I have included in Table 3, a marker for the one comparison that reached statistical significance. The study was designed to detect large differences and not powered to detect the observed 8% and 9% differences in TMP observed at 6 and 9 months of age. However, the confidence intervals around our estimates are consistent with small or modest effects. b. The reviewers next comment concerning possible waning of protection or lack of effect of 23PPV is pertinent, however the data available do not allow interpretations concerning waning of protection or the effect of 23PPV. The number of data points in the K M curves for TMP and CSOM (Figure 2) describing the latter period of follow up are too few to conclude whether the curves suggest waning protection. c. A further comment notes results from Table 5 questioning whether observed rate reductions were not statistically significant due to inadequate power. Again, the study was designed to detected large differences which were expected and a larger study would have been needed to detect smaller, or more modest effects. 4. The reviewer s final comment regarding direct and indirect effects of 7PCV are relevant and the request to examine the detection of vaccine serotype pneumococci in the early and late periods of follow up for the vaccinated group is noted. Seven vaccine serotype pneumococci were isolated from perforation discharge in the vaccinated group. These events occurred on 29 Jan 02, 11 July 02, 10 Oct 02, 28 Jan 03, 21 Jul 03, 17 Nov 03, and 10 Feb 04. Follow up ceased in March 2004. I have included the following sentence; In the vaccinated group, vaccine type pneumococci were associated with 7 episodes of perforation which were distributed evenly throughout the duration of the study (data not shown). 5. I agree that our data are consistent with reductions of CSOM but given the sample size, these reductions did not reach statistical significance. Given the primary outcomes did not include CSOM, small numbers of CSOM events and wide confidence intervals I was not prepared to made statements about reductions in CSOM. Thank you for your further consideration of this manuscript.
Yours sincerely, Grant Mackenzie