Author's response to reviews Title: Pharmacy sales data versus ward stock accounting for the surveillance of broad-spectrum antibiotic use in hospitals Authors: Jon B. Haug (jobhau57@gmail.com) Randi Myhr (randi.myhr@relis.no) Åsmund Reikvam (asmund.reikvam@medisin.uio.no) Version: 2 Date: 16 September 2011 Author's response to reviews: see over
Pharmacy sales data versus ward stock accounting for the surveillance of broad-spectrum antibiotic use in hospitals We would like to thank the reviewers for valuable comments, which we respond to below. In particular, the advice to undertake more statistical analyses has been helpful, and the findings from these have given additional support to our conclusions. As a consequence of these analyses, one table and one figure have been added in the revised manuscript (Table 3 and Figure 3). We are aware that BMC Medical Research Methodology has an Additional materials section; if transfer of any illustration to this section is needed we think that maybe the Figure 3 would be best suited. Reviewer Evangelos I Kritsotakis - Discretionary revisions: 1. Statistical analysis, 2. Paragraph: some more details on the specific form of the ICC that was used. We used the statistical software SPSS in the original article, and since it appeared difficult to explore the details of the ICC version from the accompanying documentation we have rerun the ICC analyses, now using the Stata program. Here, the command loneway is based on a variance component or mixed model, one-way ANOVA. The method is now specified in the revised article. The ICC results obtained by use of Stata are the same as those originally obtained by use of SPSS (see revised Table 2). 2. Bland and Altman criticism of ICC method, discuss in the limitations section. We are aware of the criticism of the intraclass correlation and did inspect our data with manually generated Bland-Altman (BA) plots in the initial stage. However, our study includes not only one, but 12 different comparisons (parenteral, oral, and all antibiotics for one to four weeks registrations). Because the interpretation of BA analyses is mainly visual, it appeared problematic to report these findings. More explicitly, the lack of a single measure and the challenge of a rather subjective interpretation made us leave out this statistics in the first submitted version. The Stata statistical program includes a Bland-Altman module, allowing for more 1
extensive analyses than we initially achieved with our manual BA-plots. However, use of this method did not change our conclusions, rather they were strengthened. For example this was definitely the case for pharmacy sales measures of oral ciprofloxacin even at 4-week intervals. In conclusion, we think we in the revised manuscript have complied with the referee s comments in an appropriate way. Text has been added to all sections of the paper (Abstract/Results - last sentence page 2; Statistical analyses - last paragraph, page 6; Reliability analysis - last paragraph, page 7, Discussion - first paragraph, page 8), as well as a new Table 3 and a new Figure 3. 3. Figure 1: Constant values for both ward stock and pharmacy sales data, weeks 10-12 and 25-26. We found it most correct to apply an average also for the pharmacy sales data during these weeks. By doing so, the shortcoming related to the missing data would result in a higher rather than a lower level of agreement, that is to say introducing a Type I or alfa error. Also, for the analyses of the 3-weeks and 4-weeks intervals we chose to disregard the weeks 12 and 26 leaving 24 weeks with less error for these accumulated registration periods. Accordingly, we have added text (or reformulated) in Methods page 5 - Ward stock accounting two last sentences, Pharmacy sales data second last sentence; Discussion, page 9-4th paragraph, sentence 3 and 4. Also see a footnote to Figure 3. 4. Figure 1: The authors may consider replacing the bar chart with a time-series sequence chart We see the point and have in the revision replaced the bar chart with a time series line-chart (see revised Figure 1). Reviewer Ria Benko - Major points: 1. I do not understand why the authors choose these agents and why they stick 2
to study the same 7 agents on all the different units The results on oral agents are based on one single antibacterial agent, oral ciprofloxacin. The differences in the relative use of agents within the BSA group clearly show the different AB use profile of the different wards. All these suggest that the inclusion of these 7 antibacterials is insufficient. I would suggest studying the use of antibacterials in the DU90% segment in each unit. 1. Our purpose has been to evaluate the reliability of pharmacy sales data as a measure of antibiotic consumption for short registration intervals. In this context, the different utilization profiles of antibiotics between wards are not criteria that have determined the design of this study. Intervention studies, often using the interrupted time-series design with antibiotic use registrations of one month intervals, are focusing on the antibiotics that have been intervened upon (e.g., broad-spectrum antibiotics). The purpose, both of scientific studies and routine surveillance is to reduce antibiotic resistance, for which broad-spectrum antibiotics (BSAs) are main promoters. We therefore chose a one-hospital setting with focus on wards that that had high levels of use of these agents. We agree that an approach to evaluate the 90% DU segment of antibiotic use for each ward would have given more data in this study. On the other hand, in many hospitals, including our hospital, this would have left out much of the BSAs, as the largest proportion of use is penicillins. This would be unfortunate, as a main focus should be on BSAs. We have, based on the referee s comments, again scrutinized our formulations in the Introduction and think the aim of the study is focused and could not be misinterpreted. 2. I think the ward stock size mainly depends (none of these information are provided in the manuscript) a) on the availability of antibiotics (can you obtain it within one hour in case of emergency is there an on-call pharmacy service outside working hours/weekends?) b) on the price of the antibiotics Drugs that are relatively cheap (oral forms!) and used often may be stored in larger quantities. From expensive broad spectra agents used in the empiric therapy of severe infections mostly the starting dose is stored in the unit. To study the influencing role of price, frequency of use would be a good idea. c) policy of the institution/head of department 2. Routines and practices may differ from country to country, and we have added some clarifications in the Methods section: a) Availability of antibiotics and pharmacy routines are now described (Methods - page 4 Study population second paragraph). 3
b) We have added lesser cost ( cheaper ) to the possible reason for larger variation for oral ciprofloxacin (Discussion - page 8, second paragraph, last sentence) c) Regarding institutional policies: In Norway, these do not differ in any major ways, and none that would limit the availability of antibiotics, or their use, in hospitals. (No changes made). 3) Another methodological drawback that the authors focused on five units of one hospital instead of one type of unit in several hospitals. Stock size could be determined by institutional policy. 3. We would not agree that this is a better approach. The representativeness and the possibility for generalization (external validity) are best taken care of through the design we have chosen. See also above, point 1 and 2c. 4) Why are there two urogenital wards? They have similar AB use..does not cause duplication in the correlation? (What) are the differences in their profile? 4. Both urological wards were included since they both had a historical high level of BSA use, this basis being stated in the article as a prerequisite for inclusion. To include two wards of the same specialty does not interact with the relationship - for the study population at large - between pharmacy sales data and ward stock accounting. The only characteristic of urology wards as compared to other (medical) wards is that they have a higher relative consumption of oral ciprofloxacin. We hope our background for choice of wards has been explained in enough detail in Methods - page 4 Study population, second paragraph. - Minor points: 1) The way of ward stock counting is not easy to follow. Please write it more precisely. How drug returns to pharmacy is registered (manually by nurses or in the computer database. 1. We have improved the text on this issue, thus hopefully making it better to understand. ( Ward stock accounting - page 4, second paragraph). 2) Did the pharmacy computer system use the same ATC/DDD version (which year)? 4
2. The pharmacy computer system used the ATC/DDD version 2007 as stated in the reference 9 in the text, see Ward stock accounting - page 4, third sentence. 3) How many beds are on these units? 3. The number of beds varies from 20 (haematology/endocrinology) to 28 (a urology ward) but we have not included this information as it does not seem to bear any relevance to the subject under scrutiny. 4) How explanation can you give that for urological units the total pharmacy BSA was markedly higher than ward BSA (6.4% 6.9%), while in the other three units the opposite was found. 4. As explained under major point 4, the urological wards had a higher total use of oral ciprofloxacin, for which the pharmacy sales data tended to be higher than the ward stock data, supporting a tendency to order larger bulks of tablet formulations than what is used on a weekly basis. This issue is discussed we hope sufficiently in Discussion page 9, second paragraph. 5) What explanation can you give that (considering the whole study period) the total ward BSA DDD was higher than the pharmacy sales DDDs for three units? 5. The most probable explanation is that this is due to random variations. Alternatively, although only smaller amounts of antibiotics were discarded or lent out to other wards, this may explain these relatively small differences. However, we have not deemed this point to be important enough to merit further discussion in the article. No changes made. 6) Cefuroxime has not been used orally? - and - 7) Was ciprofloxacin the only fluoroquinolone used in the units? Would be nice to provide a more detailed overview of antibiotic use. 6. and 7. Some specific aspects of our national drug formulary are: 1) the total number of registered BSAs was ten, of which seven were used in our institution during the study period; 2) ciprofloxacin was the only fluoroquinolones in use in our institution (ofloxacin is also registered in Norway) and is also the only BSA that comes in oral formulation (i.e. cefuroxime does not); 3) parenteral cefuroxime is our only registered second generation cephalosporin. These facts have now been added in the revised manuscript: Methods - page 4 Study population second paragraph, last sentence. 5
8) How do you explain the weak correlations with oral ciprofloxacin? 8. Oral ciprofloxacin, as discussed in the article (Discussion - page 8 second paragraph, sentences 2 and 3) tended to be ordered in a more arbitrary manner and with less attention to the actual need, because of lower cost and lesser package volume, i.e. it is easier to stock over longer periods than parenteral formulations. No changes made. Figure 1 9) All five units are included? 9. All five units are included. This information has now been added to the legend to Figure 1. 10) 17-18 weeks: were there any holidays? 10. No holidays during week 17-18, only in the weeks 10-12 and 25-26 (as already described in the Methods section). Figure 2 11) All five units are included? 11. All five units are included. This information has now been added to the legend to Figure 2. 12) In the table 1 you wrote antibacterial classes, now you wrote antibacterial agents and one antibiotic group (carbapenems). It is not didactic. 12. We see the point. Figure 2 show the single ATC 5-level substances and Table 1 only the antibiotic subclasses, to save space. Carbapenems were referred to as a group in Figure 2 because one ward used a few doses imipenem on one occasion whereas meropenem was the predominant carbapenem used. In the revised manuscript, we have added a footnote (2) to the figure legend specifying which two substances belong to the carbapenems group. 13) For the outliers "n=2" values are not interpretable. What do they mean? What n= values are displayed twice (ceftazidime boxplot n=1; cefuroxime boxplot n=2)? 13. n in Figure 2 means the number of outliers with the particular number of DDDs (yaxis). For ceftazidime the n=1 is displayed twice to show one order of DDD=10 and one of DDD=15; correspondingly for cefuroxime the two n s refer to DDD=25 and DDD=30. For ceftriaxone and parenteral ciprofloxacin no inter-quartile range could be displayed because 6
only three sizes of packages are used. Since n is a commonly accepted abbreviation for number we don t think further explanation is necessary in the legend. Table1 14) It is also not clear whether cefuroxime was the only second generation agent or not. If yes, why you use the plural form? 14. Cefuroxime is only registered in parenteral formulation, see point 6 above. The plural form 2 nd generation cephalosporins is used out of convention. 15) Strange those differences are displayed for 26 weeks. Why? 15. The minor DDD difference between the two measurements methods for the study as a whole (26 weeks) is considered by the authors to be of interest. 16) Is there any explanation that for carbapenems the difference is always positive (3 wards) so the pharmacy DDD is higher that the Ward DDD?. 16. The numbers for carbapenems are considered so small that trends could not be revealed. (No changes made). Table2 17) week instead of "w". 17. Week is now fully written in the revised Table 2. 18) The abstract is misty. Aim/Background: The aim is nor well defined in the abstract. I think the concrete aim of this study was to assess how well the pharmacy based dispensing data approximate/reflects the real antibiotic use in different time-intervals. 18. We have considered this comment and find that the aim of the study is defined clearly in our original abstract; containing the same message as the one phrased by the referee. 19) Methods: It is not clear in the methods of the abstracts that which and how many agents are included in the study. 19. The text have been revised slightly to During 26 weeks, we performed a weekly ward stock count of use of broad-spectrum antibiotics that is second- and third-generation cephalosporins, carbapenems, and quinolones in five hospital wards We do not think BSAs need to be listed on a substance level in the Abstract since this is accounted for in detail in the Methods section. 7
20) Results: It is not defined before that what you mean under broad spectrum agents. ATC classification should be part of the method section 20. See our comment to point 19 for broad-spectrum antibiotics. The ATC classification is included in the abstract, and also cited (reference 9) in the original text under Methods - page 5 Ward stock accounting, line 6. 21) For the pulmonary ward the not the full measurement unit is written: not only DDD but DDD per occupied bed-days. 21. We agree that the sentence in the original manuscript was not exact, and have changed it to BSA use during the 26 study weeks ranged from 12.8 DDD/100 bed days in one urological ward to 24.5 DDD/100 bed days in the pulmonary diseases ward. 22) I have found some misunderstanding of the referred articles In a recent survey of hospital pharmacy practice in Europe it was found that 70% of hospitals based their practice on ward held stocks [5]. It is not precise, in the original manuscript it says that on average in 70% of EU hospitals hold antibiotic stocks at the wards. to have antibiotic stock#base practise on ward stock. 22. We are grateful that this imprecision have been brought to our attention. We have changed the sentence (Background page 3, second paragraph) so that it refers correctly to the cited paper: In a recent survey of hospital pharmacy practice in Europe it was found that 70% of hospitals held antibiotic stocks at the wards [5]. 23) The best method to assess the effect of drug intervention is to have several data points before and after policy implementation (interrupted time-serial analysis), for example have ~30 points before and after. It can be ~30 weeks or ~30 months, what is important to have longitudinal data, so the observation period has to be long (only the the used intervals can be weeks)! You have to incorporate this in this sentence, otherwise it is misleading. - and 24) However, the appropriateness of using short observation periods has not been assessed. How the duration of the registration period impacts the recording of antibiotic use has not been scrutinized. See my suggestions above. These sentences are not precise. Duration of observation/registration period#duration of the registration intervals. 8
23 and 24. We see that the distinction between observation period/interval and registration interval was not clear enough, and in the revision we have used the term registration interval consequently throughout the text. A change has been made in paragraph referred to by the referee: Background - page 3, third paragraph. 9