Jose Merino (Chair), Georg Roeggla, Tiago Villaneuva, John Fletcher. Amy Price, Elisabeth Loder. Jamie Kirhham (statisticians), Rubin Minhas

Transcription

1 BMJ "Efficacy of drugs used in preventive chemotherapy against soil-transmitted helminths: systematic review and network meta-analysis" ********************************************************************************************************* Basel, 25 July 2017 Dear Dr. Rubin Minhas We were delighted to receive your dated 21 June 2017 and we would like to thank you, the members of the committee and the external referees very much indeed for having studied our manuscript so carefully and we are grateful to the series of useful and constructive comments. Please, find below the point-by-point response, indicating how and where the manuscript was adjusted or changed (line numbers). Furthermore, we will attach the manuscript were all the changes are highlighted in yellow. We look forward to your further disposition. Yours sincerely, Jennifer Keiser and Wendelin Moser (on behalf of all the authors) Decision: Put points Detailed comments from the meeting: Jose Merino (Chair), Georg Roeggla, Tiago Villaneuva, John Fletcher. Amy Price, Elisabeth Loder. Jamie Kirhham (statisticians), Rubin Minhas JK: some issues with reporting of NMA, probably due to following wrong reporting guideline, their is a specific reporting guideline for NMA, and PRISMA was not the appropriate one here. Stats report would be required. Update is relevant. Comments addressable though significant. RQ is warranted, drugs may lose efficacy. May add to practice, focus on egg reduction rates may be interesting. Can they also discuss harms, too? 1

2 Decision: Put points + stats report We are grateful to these comments. As the editor rightly points out all comments were addressable we have updated our manuscript accordingly. We also follow the new guidelines. ** Comments from the external peer reviewers** Reviewer: 1 Recommendation: Comments: BMJ This is a very interesting update to previous reviews evaluating the efficacy of available drugs for STH. These results have been widely cited in the field previously and it is likely that the current paper will also be used for decision making at the policy and program level. The authors have done a very nice job of presenting a simple, easy to interpret analysis of their findings As a systematic review and meta-analysis, there are a few details that would be worth including in the paper. It would be nice to see results stratified by some baseline demography (age, gender). In addition, it would be useful to clarify when treatment was school based vs. not (community based). We added seven stratifications for CRs and ERRs, as requested by several reviewers (appendix table S5- S11). A stratification according to gender and age was not possible because we did not have enough studies for a meaningful stratification with enough observations, i.e. only one study reported results from boys or girls only. We faced the same problem with age, as the categories were mixed, however the stratification school versus community based might reflect best age differences (appendix table S6). We extended also the description of the included studies and added the information about the study settings (school or community; appendix table S2) In addition, the assessment of study quality is critical to be able to evaluate these data. The quality of such an analysis is only as good as the data going in. I would like to see a GRADE analysis applied to all included studies so that a study quality can be ascertained in a more standardized manner. This is really quite important, as if there are a few higher quality studies included, it may be worth summarizing those separately. We agree with the reviewer and added a stratification of CRs and ERRs according to the quality criteria of Jadad and colleagues (appendix table S8) It is also quite important to discuss differences in drug efficacy by time in more details. Is it possible that the decline in efficacy observed can be completely explained by the overall decline in intensity of infection observed over this period (with the limitations of the included diagnostics)? Was albendazole more likely to be used in more recent studies? Was diagnostic test used different by time period? 2

3 In addition to the stratification (before and after the year 2000, appendix table S10), we now present estimates of CRs and ERRs rates from an interaction analysis with two different time points, 1995 and 2015 in the supplementary file (appendix table S13). These estimates were derived from regression models with drug-specific time trend variables. Potential period effects which might confound the comparisons between the different drugs can be controlled in this way. Additional stratifications include baseline infection intensity, diagnostic methods and follow-up time (appendix table S7-S11). Important results from the stratification and interaction analysis are discussed in strength and limitations (line , page 10/11) and clinical implication (line , page 12) Was there a difference in the choice of diagnostic used in the studies by drug being compared? A stratification according to the diagnostic method was added in the Appendix (Table S7; discussion line , page 11) While the methods section was very clear and well written, the introduction, abstract and conclusion need significant editing. The language is awkward and difficult to understand at points. In addition, some sections could be better organized. I did not include line by line changes as I believe the authors are capable of editing this to a reasonable standard. In the meanwhile we carefully read the manuscript and hopefully improved the language according to the reviewer s expectation The last sentence of the abstract RCTs with anhelminthic drugs. Is certainly true but comes out of the blue as no mention of study design is made in the results of the abstract. I would remove this. We agree with the reviewer and removed this sentence in the abstract For ascaris, a test of significance for the difference between drugs is provided for the comparison of CRs but not for ERRs? The result part was carefully revised and p-values were added for significant results only The conclusions ramble a little bit and are unfocused. At the very end, tribendimidine is brought in without much context. The final paragraph really needs to summarize and conclude. Again, careful editing of this section is needed. We agree with the reviewer and carefully revised the discussion including the final paragraph. Reviewer: 2 Recommendation: Comments: This review is an update of the two important systematic reviews such as Efficacy of current drugs against soil-transmitted helminth infections: systematic review and meta-analysis, in which new evidence and new analysis methodology were applied. Before publishing, the following issues should be considered: 3

4 2.1. Please rethink the title of this paper and use the term preventative chemotherapy with caution. Are you sure that the term preventative chemotherapy fits with the targeted RCTs studies? The term preventive chemotherapy (PC) was introduced by the World Health Organization (WHO), to cover the approach of treating populations at risk of human helminth diseases, to prevent transmission or morbidity of those diseases, with drugs either alone or in combination. We agree with the reviewer and changed the title to Efficacy of recommended drugs against soiltransmitted helminths: systematic review and network meta-analysis 2.2. Could you make sure that there was no language limitation in this review? Had related individual studies published in Chinese, Thai, Korean, or Arabic been included? We could identify some related English abstracts of non-english articles in the database such as Pubmed. Did you try to find and read the fulltexts in order to make a further decision? We specified the procedure for non-english written studies (line , page 5). In case an abstract met the inclusion criteria, the full text was read and evaluated for inclusion. Abstracts and full-text in other languages were translated by native speakers (i.e. Chinese) Please supply lists of included and excluded studies as supplement materials. In the meanwhile we extended the major characteristics of the included (appendix table S2) and added a table with the excluded studies (appendix table S3) Please supply the table to describe the characteristics of included studies. Please, see answer above Others: spelling mistakes. We carefully revised the manuscript for spelling mistakes and made major English correction and hope we have improved our manuscript. Reviewer: 3 Recommendation: Comments: - I am not a native speaker but I think that there are some typos in the manuscript that the authors have to check. We apologize for the typos and carefully revised the manuscript. - The paper is a very updated review. I consider that its main input is that the authors estimated the efficacy in terms of cure rates (CRs) and also egg-reduction rates (ERR) of albendazole, for each 4

5 helminth. Additionally, the use of network meta-analysis methodology is an excellent and original approach to the problem. - I consider that the paper is very important for police-makers in order to choose the best available alternatives to control soil-transmitted helminths. - The question is well defined and the authors followed very well the PRISMA statement. - The methodology is also very appropriate but I have some comments about it: The authors have to specify that they included studies with AT LEAST one of the eligible treatments (versus other treatment or placebo). We specified the inclusion of only one eligible treatment arm (line , page 5) and explained the procedure of pooling for studies including only one eligible treatment arm (line , page 6) Inclusion and exclusion criteria should be better described in the manuscript even if there was not any particular restrictions (age, minimum number of arms, minimum information needed, etc...). We have extended the inclusion and exclusion criteria s (line , page 5) and the excluded studies have been listed (appendix table S3) as requested by reviewer #2 (2.3) Please describe better how EER was obtained considering that it is actually a discrete variable. Although, I its not very clear for me how the authors got the "percentage" described. In general, the management of this variable should be better explained in methods. The Paragraph Data synthesis and analysis was extended (line , page 6/7) and hopefully meets the reviewer s expectation. ERRs was considered as continuous variable as it represents a ratio, and we therefore used mixed linear regression models to analyse this outcome The use of network analysis must be better explained in this study (methods section). Also, this results could be interpreted more clearly. An explanation about the advantage of a network meta-analysis over a common meta-analysis was added (line , page 6) and the result for the network explained in more detail (line , page 8). In the supplementary file, we also added a table comparing the results obtained by network metaanalysis with the corresponding results from traditional meta-analyses (appendix table S4) I consider that the authors have to present the chi-squared tests for heterogeneity between and intra-study, the Cochran s Q and also the I squared, for EACH meta-analysis that they did. Please include them in methods and in the respective graphs or tables. Also, please discuss about them. Unfortunately, the traditional measures of heterogeneity cannot be computed in network metaanalysis. But as we now also performed classical meta-analyses comparing pairs of treatments separately, we could derive I 2 and tau 2 for these comparisons (line , page 7). They are given in the new table comparing results from network and from traditional meta-analysis (appendix table S4). 5

6 3.5. The use of random mixed models must be better justified with statistical arguments not just because they are following the approach of other authors. Also, the authors said that they used logistic mixed models. Certainly, It is appropriate for binary responses (as such as "cure": yes versus no) but I am not sure that it is a good alternative for the changes of ERR. Actually the eggs count is discrete variable and other models should be used, instead. We have now extended the description in Data synthesis and statistical analysis and also explain the rationales. As regards the analysis of ERRs, we have used linear mixed models. This was already written in the previous version of the manuscript but may not have become clear enough Please describe the exact models specification for each response. Maybe, the other problem is that they have to explain better how they work with the variable: ERR. We carefully revised the method and now described the method in more detail The authors dichotomized the infection intensity using the median of baseline egg counts, what did not they use the well recognized WHO classification? (see: Eliminating soil-transmitted helminthiases as a public health problem in children: progress report and strategic plan ). Please explain and consider this alternative. We considered the classification of WHO and tried to dichotomize according to the official cut-offs into light and moderate/heavy baseline infection intensity. However, because we did not have enough studies with moderate/heavy baseline infection intensities for a balanced stratification for each treatment, we had to stratify using the median Kessels and colleagues approach could be explained in few sentences. In general terms, analysis sections could be better shown with a better redactions. The approach by Kessels and colleagues are now explained in detail (line , page 6) and we did some major changes in the statistical analysis paragraph I would like to know of the authors considered to use meta-regression to estimate the changes of the estimators according to the intensity of infection, continent, publication year and size of study? I think that It could be a good alternative. We added seven different stratification for CRs and ERRs, as requested by several reviewers (appendix table S5-S11). Stratification analyses are an alternative to meta-regression. We prefer this approach for its better understandability. However, we also performed two analyses mimicking classical metaregression in the framework of our mixed model based network meta-analysis. We considered models with interaction terms between treatments and time and between treatments and study size. These analyses provided estimates of cure and ERRs for 1995 and 2015 (appendix table S13) and for small (n=30) and large (n=200) studies (appendix table S12) The management of the Average egg-reduction rates should be well clarified. (the variables as such as the statistical management). 6

7 - the results are well presented. But they have to complete them with my suggestions. The time period of the evaluation of the outcomes should be specified for each study We have added a stratification according to the length of the follow-up (appendix table S9). - I think that they can improve the visualization of the graphs and also include the estimators suggested above. We present our data as OR for CR and difference for ERR as it is common for meta-analysis. To keep the figures more simple and easier to read, as requested by reviewer #4 (4.6), we did not present the comparison of the treatments versus placebo. As the respective odds ratios are much larger than the ones between treatments, a figure also including the comparisons with placebo would have taken the focus away from the comparisons between treatments. However, we added a figure including all comparisons with placebo in the Appendix (Figure S1). The management of the ERRs are now further explained (line ). - They could discuss how this results do not include more than a single-dosage treatments which could be important for some helminths. This is an excellent suggestion. However, as according to WHO guidelines single doses are the standard of care we focus on single doses in this paper. Of note there are unfortunately very few studies evaluating multiple doses, hence we doubt that we could have calculated summary estimates. - They must discuss how we can expect that the reduction obtained with this treatment could remain. Is there any way to assess the time that reductions in prevalence or parasitic loads would remain? We now additionally stratified analyses according to follow-up time (<= 3 weeks vs. > 3 weeks, appendix table S9). They didn t provide evidence of decreasing effects with increasing time of follow-up. However, as follow-up times were limited to 6 weeks in all studies, we have now possibility to address longer term sustainability of effects. We added also an analysis including an estimation of CRs and ERRs for the years 1995 and 2015 to investigate the time trend in the appendix (Table S13). - They did not discuss enough the results obtained from the network analysis. The discussion about the network results was expanded (line , line , page 10/11). - The conclusions are very clear and appropriate. - References are relevant. Reviewer: 4 7

8 Recommendation: Comments: BMJ Moser et al. A comprehensive review of the efficacy of the main drugs used for treating and preventing infections caused by soil-transmitted nematodes (STNs). It updates and complements previous similar work (Keiser & Utzinger, JAMA, 2008.) Main questions: 4.1. Please explain how come, if only RCTs were included, that there are 19 studies with one eligible treatment arm only (line 204). We extended the explanation of the inclusion of RCTs with only one eligible treatment arm (line , page 6) It is very difficult to figure out how many studies with how many patients contribute to the analysis of each drug. Some suggestions: o Verify numbers in the circles in figure 1: something wrong with mebendazole it cannot be 1. Also, the placebo circle is empty. We adapted the figure and the figure description (see figure 1 and line , page 21). o Add N studies and N patients in Table 1 for both CR and ERR, as well as the Supplementary tables. All study and patient N were added for Table 1 and for the stratifications in the appendix (table S5-S11). Additionally we present the participant, cured participants and cumulative CRs in the appendix table S The authors should be more explicit about the type of ERRs considered. It looks as if they took any (arithmetic, geometric mean, unknown) as understood from line 179 in the M&M s, when talking about the mixed linear model. If such is the case, this is confusing. How were the average rates (ERR in particular) calculated? Is it the crude average of the ERRs of all studies with a given treatment, irrespective of which calculation method was applied, and not weighted for the sample size? And how are the 95%CIs calculated? We extended the description of the analysis in the Data Synthesis and Statistical Analysis (line , page 7). Study-specific ERRs were used as outcomes of the mixed linear regression models. Estimates of average ERRs were derived from these models by taking the sum of the intercept term and the regression coefficient of the respective drug. We have added a discussion about the ERRs (line , page 11) Does the above explain the difference between the ERRs in Table 1 ( average ERRs ) and Table S3 ( estimated ERRs )? Indeed this is the case because of the involvement of random effects in the mixed regression models Figure 2 (CRs). Is it correct to assume it is based on the CRs presented in Table 1? Or please make clear if otherwise, at that one can make sense of all the figures presented. Also please: 8

9 Yes, the reviewer is correct. We added an explanation (line , page 7). o add n/n for each arm; We have added the number of included participants and studies by treatment arm and helminth species in table 1. o add comparisons vs. placebo; We have added the comparison with placebo in the figure. However, we would like to put the figure with placebo in the Appendix (figure S1) and keep the figure without placebo in the manuscript, for the reason the reviewer stated in the point below (4.6). o indicate which side the comparison favours which arm. We have added an indication The figures are small and difficult to read for people of a certain age the colour coding helps. All figures were revised The Discussion is consistent with the findings presented. Only minor comments: o Lines There is a sudden shift from mebendazole to pyrantel pamoate when talking about N. americanus. Also the former is expected to be less effective. o Mebendazole misspelled (line 299) o The discussion about the effects of sample size on comparisons (lines ) is rather convoluted. Results should be weighted for the sample size. The message should be simplified: we might have under- or over-estimated treatment effects because studies have been traditionally small. Larger studies (which we are more confident with) on hookworms show less additional benefit of albendazole and mebendazole over placebo and a larger effect of albendazole over mebendazole; and larger studies on T. trichiura show a smaller difference between albendazole and mebendazole. All corrections and adjustments have been done. Reviewer: 5 Recommendation: Comments: 5.1. Can the authors please follow the PRISMA extension reporting guideline for Network Meta-Analysis studies as opposed to the standard PRISMA Statement. We have added all points from the extension for reporting Network Meta-Analysis as requested by the reviewer. We uploaded the extended PRISMA statement. 9

10 5.2. In the methods section Only studies with data including the exact number of patients at baseline and follow-up were included to assess CRs. Were study authors contacted to obtain any missing information in order to make the necessary calculations? Please justify if this was not done. The entire method section was carefully revised and this sentence deleted. We did not contact any author. One study was only reporting ERR and no CR, therefore excluded for the CR analysis (figure 1) The authors quote The ERR were extracted as percentage and categorized as arithmetic and geometric or unspecified if no information about the mean was given. Consider moving this section as this is not part of study selection. For the general reader, what is the importance of the geometric or arithmetic mean in terms of the analysis? We have carefully revised the method section. Arithmetic and geometric means are systematically different. Failure to adjust for such differences might result in confounded treatment comparisons if the two different types of means are not equally distributed across the treatments compared. We have added an extended description in the method (line , page 7) and discussed out ERR model (line , page 11) Can the authors justify why the older Jadad risk of bias assessment tool was used in preference to newer tools such as the Cochrane Risk of Bias tool. Can the authors also provide a brief sentence on how risk of bias assessment was undertaken, e.g. was this done by more than one authors and how were discrepancies resolved. The risk assessment was explained in more detail (line , page 6). We added the Cochrane Risk of Bias tool (appendix table S2) and presented its result (line , page 8; appendix table S14). However, we kept the Jadad score in order to stratify according to the study quality (appendix table S8). 5.5 The data synthesis and analysis section reads a little disjoint for the general BMJ reader and there are some important omissions in the methods for network meta-analysis which should be included. Can this section be amended. Here are some suggestions although the PRISMA NMA should also be consulted: a) Discuss the principals of network meta-analysis (i.e. it is the combination of both indirect and indirect evidence incorporated into one analysis). Describe briefly how an indirect estimate is informed by direct evidence. A brief description has been added (line , page 6). b) Briefly describe the mixed logistic regression network meta-analysis models used (already described) We have added a more detailed description of the mixed logistic regression model of Kessels and colleagues in the Data Synthesis and Statistical Analysis section ( , page 6). c) Where direct evidence is available, pairwise meta-analysis should be performed; the methods for conducting this analysis should be described. 10

11 We now also conducted classical meta-analyses comparing pairs of treatments separately and described the procedures in the Data synthesis and statistical analysis paragraph (line , page 7). Their results are compared with the corresponding estimates from the network meta-analysis in the appendix (appendix table S4). d) The agreement between the direct and indirect evidence should be assessed. This is the inconsistency assumption used in network meta-analysis. Can you describe if/how this was explored (e.g. inconsistency plot) and how this would be accounted for if inconsistencies were found. We have added the comparison of network and pairwise meta-analysis (appendix table S4). Additionally an inconsistency plot was constructed (appendix figure S3). The detailed method is described in the Data synthesis and statistical analysis paragraph ( , page 7) and results (line , page 8). e) Specify the assessment of risk of bias that may affect the cumulative evidence (mostly described already), inclusive of small study effects (e.g. funnel plots), where appropriate. To address potential publication bias, we additionally conducted a mixed regression analysis (model described line , page 7) including interactions between treatments and study size and estimated CRs and ERRs for small (n=30) and large (n=200) studies (appendix table S13; discussion line , page 12). f) Was heterogeneity assessed how was this done? We did not quantify heterogeneity between studies in our network meta-analyses. However, we now also conducted classical meta-analyses. The respective results, including the I 2 - and tau 2 -statistics (line , page 7), are presented in the appendix (table S5). g) Could the treatments considered be ranked, e.g. using an appropriate analysis. In helminthology a combination out of CRs and ERRs reflect the true efficacy. We presented the results of both in a direct comparison of the drugs (figure 2 and appendix figure S1). However, as both measurement have to be considered in parallel, a simple ranking of the treatments in terms of efficacy is not possible without defining a specific rule for combining the two criteria. As this does not exist up to date, we just discuss and ranked the drugs qualitatively in the discussion (line , page 12) In the results section it is quoted that 55 studies were included, however Figure 1 and the abstract mentions 56. Can this be clarified throughout. We have done a major revision of figure 1 and presented now the search and included studies in more detail and for ERRs and CRs separately. Briefly, out of the 56 studies one study reported only ERRs and not CRs, which was confusing Results: Figure 2 seems to show the pairwise comparisons between the different interventions (according to Figure title). Crucially, where is the Network meta-analysis of treatment options. Perhaps labelling of figures/tables needs improving. A suggestive approach would be to present the network meta-analysis as a graphic and then present as a table the pairwise results (where possible) and the NMA results for the outcome measures. This way consistency can be easily demonstrated. 11

12 The pairwise comparisons in Figure 2 derive from NMA and not from separate pairwise comparisons. We now emphasize this in the legend of the figure. Moreover, we have added a comparison of the results from NMA with those from classical meta-analysis in the supplementary appendix (Table S5 and appendix figure S2). 12