May 31, Re: Submission of revised manuscript (BMJ R1) Dear Drs. Villanueva and Godlee

Transcription

1 May 31, 2016 Re: Submission of revised manuscript (BMJ R1) Dear Drs. Villanueva and Godlee We greatly appreciate the positive critique of the reviewers' and the Editors in response to our revised manuscript titled "Chemoprevention of Colorectal Cancer in Individuals with Prior Colorectal Neoplasia: A Systematic Review and Network Metaanalysis" (BMJ R1). On behalf of my co-investigators, I wish to submit a thoroughly revised manuscript addressing all concerns raised by the Reviewers. In particular, we have appropriately interpreted SUCRA, added details of small study effect assessment, details of statistical approach to network meta-analysis. Please find attached the detailed point-by-point response. This manuscript is not being considered for publication elsewhere and none of its contents have been previously published. This work was presented at Digestive Disease Week held from May 21-24, All authors have read and approved the revised manuscript and the responses to the editor and reviewers. We have verified and confirmed that all author disclosures listed in the Acknowledgment section of the manuscript are accurate and up-to-date. I hope this revised version of the manuscript is found worthy of consideration for publication in your journal, and I look forward to receiving your comments. Best Regards Siddharth Singh, MD, MS Assistant Professor of Medicine, Division of Gastroenterology University of California San Diego RESPONSE TO REVIEWERS COMMENTS RESPONSE TO REVIEWER #1 Comment 1. There is a misconception that funnel plot is used exclusively to detect publication bias. In a strict sense, funnel plot was invented for the visual inspection of small-study effects which might result from heterogeneity, publication or sampling error. In the light of existing small-study effects (either as indicated by the funnel plot or the Egger s test), publication bias should be considered as one of many possible explanations of such evidence. Please, rephrase as befits in the Methods section and Results section. Response: We thank the reviewer for highlighting this important distinction. We have rephrased our statements within the Methods and Results section to reflect that the funnel plot was used to assess for small-study effects. To allow for the readership to appropriately interpret our assessment, the funnel plot is accompanied by a table of comparisons (active vs. control), as recommended by Reviewer 4. Moreover, as detailed in our response to comments #3, #4 and #5 by Reviewer 4, we have excluded the use of Egger s test to assess for small study effects, given some of the limitations due to the small number of studies per comparison, and the confounding effect of directionality of comparisons in its use for the overall study. Changes in manuscript: Methods, page 11, para 1; Results, page 18, para 2; Online supplement, efigure 6 Comment 2. I would advise the authors to rephrase the interpretation of SUCRAs as follows: the percentage of efficacy/ safety achieved by an agent compared to an imaginary agent that is always the best without uncertainty (i.e., SUCRA = 100%). A note: in the statistical analysis in the Supplementary material it seems that the authors wrote twice the same sentence about SUCRA. Response: We appreciate the reviewer s comment, and have rephrased the interpretation of SUCRA as suggested, and as also suggested by Reviewer 4, comment #1. We have also deleted the duplicate sentence regarding SUCRA in the Online Supplement. Changes in manuscript: Methods, page 11, para 2 Comment 3. It is important to accompany the results on agent-hierarchy with rankograms as these plots illustrate the uncertainty in the ranking for each agent. Response: We appreciate the reviewer s suggestion. In the revised manuscript, we have included the rankograms in the Online Supplement (efigures 5A-C). Additionally, in the SUCRA plot of efficacy and safety, we have added a table representing the median (with 95% confidence interval) safety and efficacy ranking of each agent. Changes in manuscript: Figure 2; Online supplement, efigure 5A-C Comment 4. Since the authors employed a Bayesian random-effects NMA, results on the heterogeneity standard deviation and its 95% CrI should be presented. What assumption was made for the heterogeneity parameter; common within-network, common within-loop or comparison specific? (Then first one is usually assumed). Then the authors can compare the magnitude of the heterogeneity variance with the respective empirical distribution as obtained by Turner et al (2015, Stat. Med. doi: /sim.6381). Results on the heterogeneity parameter (under tau-square) are also omitted in the pairwise meta-analysis results. Response: For our network meta-analysis, the heterogeneity assumption was that heterogeneity is common within-network. This is now stated explicitly in the Methods section. We also present a table for tau-squared and its 95% confidence interval

2 which is quite comparable to the empirical distribution for between-study heterogeneity in studies of surgical/device-related success assessing semi-objective outcomes [which was 0.12 (0.08 to 0.16) in table III in the study by Turner et al]. Other heterogeneity indicators are presented in the same table (I2 and Q-test) to give readers an overall picture of heterogeneity, which in general was very minimal. However, it is important to note that with a small number of trials per analysis, tests for heterogeneity may be underpowered. Changes in manuscript: Online supplement, emethods, page 3, para 2; page 4, para 1; etable 3 Comment 5. No numerical results on the inconsistency evaluation are provided, apart from textual references solely. A very informative presentation of the results from the node-splitting method could be in a table that illustrates the direct, indirect and mixed effects (mean value, standard deviation and measure of conflict P) for each observed comparison. Node-splitting is a local investigation of inconsistency. A global evaluation is also needed to get a complete and better understanding of possible sources of inconsistency. The design-by-treatment model by Higgins et al. (2012, Res. Synth. Method. doi: /jrsm.1045) and the Q statistic for inconsistency by König et al. (2013, Stat. Med. doi: /sim.6001) and Krahn et al. (2014, BMC Med Res Methodol. doi: / ) have the advantage to be robust on the reparameterization of the model in the presence of multi-arm trials. In the description of the statistical analysis in the Supplementary material, please, include the relevant references. Response: We agree with reviewer about presenting a more detailed evaluation of network inconsistency. As suggested, in the revised manuscript, we provide the direct, indirect and mixed effects, along with P value as a measure of conflict or inconsistency. However, we do not believe that a global measure of inconsistency would be very helpful and may be potentially misleading. Following GRADE guidance for network meta-analysis (Puhan MA, Schünemann HJ, Murad MH; GRADE Working Group. A GRADE Working Group approach for rating the quality of treatment effect estimates from network meta-analysis. BMJ Sep 24;349:g5630), we evaluated inconsistency by looking at every comparison and do not think that a single number (representing tens of possible permutations of comparisons) is appropriate. The quality of evidence and strength of inference should be assessed per comparison and evidence users (patients, physicians and policy makers) should consider a certainty or confidence level per comparison (e.g., per drugs they are contemplating to use in a particular setting). Changes in manuscript: Online supplement, etable 12A-C Comment 6. No numerical results on the model fit are provided. It suffices to mention them directly in the text. Response: We appreciate the reviewer s comment. As suggested, we have added results on model fit and convergence in the revised manuscript. The model fit was appropriate with total residual deviance approximating the number of unconstrained data points; respectively for the outcomes of any adenoma (37.78, 36), advanced adenoma (33.96, 33) and serious adverse effects (30.04, 33). Deviance information criterion was for the random-effects model and for the fixed-effects model (a difference less than 5 suggests that both models are reasonable without a significant difference). Convergence of chains was verified visually by looking at trace plots and inspecting the Brooks-Gelman-Rubin diagnostic statistic with values around 1. Changes in manuscript: Results, page 18, para 2 Comment 7. The description of the set-up of the Bayesian analysis is not complete. It is not enough to state only which distributions were used for the priors. The type of prior (vague or informative) is dictated by the values used on the parameters of the respective distribution. To achieve transparency, the authors should fully specify the prior used for each parameter of the model. Furthermore, specify how many chains were used. Response: In the revised manuscript, we have provided more detailed description of priors. We used vague priors for all variables. For trial baselines and for treatment effects, we used prior distributions with a mean of 0 and very low precision (0.0001). For the prior distribution of between-trial standard deviation, we used a uniform distribution with a mean of 0 and large variance (5). We changed the precision/variance of the priors in sensitivity analyses with no or trivial change to the estimates suggesting robust approach. We tested 3 chains with different initial values and ascertained convergence (see response to comment #6 above, regarding convergence). Changes in manuscript: Online supplement, emethods, page 4 Comment 8. Regarding the sensitivity analyses, I am confused with the term per-protocol completer analysis. The way the authors define this term, I understand it as modified ITT. In addition, it is not clear to me what assumption the authors used for the missing outcome data in the primary analyses. Response: We apologize for the confusion. The per-protocol completer sensitivity analysis included only those individuals who underwent colonoscopy surveillance at the pre-specified time period per protocol, and excluded individuals who underwent a colonoscopic surveillance assessment prior to, or after, the anticipated main surveillance interval. On the other hand, a modified ITT analysis (primary analysis) included all individuals who received at least 1 dose of the candidate agent and had at least one colonoscopy post-randomization [for efficacy] or had one post-randomization follow-up [for safety]. Since the primary mitt analysis included only patients who received at least one dose of the medication AND underwent postrandomization colonoscopy (for assessment of occurrence of outcome metachronous advanced neoplase), we did not have to impute any missing outcome data. To account for inherent biases of this mitt analysis, we performed a worst-case scenario sensitivity analysis, in which patients who did not undergo a post-randomization colonoscopy were assumed to have developed advanced metachronous neoplasia. Changes in manuscript: Online supplement, emethods, page 5 Comment 9. The authors should link the sensitivity analyses with the evaluation of the transitivity assumption in terms of important effect-modifiers (also possible sources of heterogeneity). It is important to include some text on the transitivity assumption (in the supplementary material, for instance) to persuade the reader that you accounted for this assumption which is the cornerstone of NMA. For instance, for the set of studied agents does the assumption of jointly randomized applies? Are the agents similarly defined across the trials (for instance, in terms of doses)? Please, refer to Salanti (2012, Res. Synth. Method. doi: /jrsm.1037) for more details on the subject. Further in the sensitivity analyses, I suggest the authors to put the name of the treatments across the first row and the first column in order to include in the diagonals the SUCRA values

3 (and the probability of being the best treatment in a parenthesis). In that way the reader will be able to evaluate the robustness of the results both in terms of pairwise treatment effects and ranking. Response: We agree with the reviewer s comment regarding evaluation of the transitivity assumptions, in terms of important effect modifiers. Based on a priori protocol, we used standard doses of potential chemopreventive agents, and in case of aspirin, separated out all analyses in terms of low- and high-dose aspirin, consistent with clinical practice. In the revised manuscript, we have evaluated the distribution of potential effect modifiers (prior advanced colorectal neoplasia, family history of colorectal cancer and body mass index), for the outcome of metachronous advanced neoplasia in included trials; these have been reported in etable 2. In included studies, the median and interquartile range proportion of patients with prior advanced colorectal neoplasia and family history of colorectal cancer in 1st degree relatives was 44% (IQR, 23-68) and 21% (IQR, 19-32), respectively. The median of average BMI (and range of average BMI) across trials was 27.6kg/m2 (range, kg/m2). Hence, the trials were generally similar in terms of distribution of important effect modifiers, suggesting that the transitivity assumption is valid. Changes in manuscript: Results, page 19, para 1 Comment 10. I have some concerns for the Risk of Bias tool and particularly for the item pertaining to attrition bias. The attrition rate is variable across the studies and between arms, apart from the study of Chu (2011) where the information is not available. However the authors rated attrition bias as low for the great majority of the studies including also the study by Chu (2011). Response: We recognize that the rates of attrition varied considerably across trials, and between arms. In our a priori risk of bias assessment protocol for this meta-analysis, we deemed studies to be at a high risk for attrition if they had >5% differential attrition between arms. We have clarified this in the figure legend of the supplementary material (efigure 3). Additionally, for our primary modified ITT analysis, only patients with a post-randomization colonoscopy (where primary outcome could be assessed) were included, and hence, in the context of our primary outcome of metachronous advanced neoplasia, the risk of bias due to high or variable attrition is low. Hence, though the study by Chu et al does not provide data to estimate attrition rates, with our modified ITT analysis, the study was deemed to be at low risk of bias. Changes in manuscript: Methods, page 14, para 3; Online supplement, page 37, Figure legends Comment 11. The GRADE evaluation would be definitely more informative if the results could be presented as the authors in the cited reference have indicated. The authors could create the indicated table in order to supplement the already provided table. Response: We appreciate the reviewer s suggestion. We have created a table reporting the risk estimates and quality of the evidence for pairwise comparisons, indirect comparisons (from first-order loops) and mixed effects from network metaanalysis. This has been reported in etable 14. Changes in manuscript: Online supplement, etable 14 Comment 12. Please, include 3 or 4 relevant references the very first time you mention NMA. Optional: Increased precision of the results and ability to rank the studied interventions are also very important strengths of NMA that are worth being mentioned. The term direct is used to refer to comparisons, evidence and treatment effect. When it comes to meta-analysis you might say either classical or pairwise meta-analysis. Response: As suggested, we have added references regarding network meta-analysis in the Introduction. Additionally, we have replaced the term direct meta-analysis with pairwise meta-analysis. Changes in manuscript: Introduction, page 6, para 2; page 7, para 1; throughout the manuscript RESPONSE TO REVIEWER #2 Comment 1. The authors abstracted data from modified ITT analyses. I was wondering whether the requirement that participants had to have at least one post-randomization colonoscopy for inclusion has any potential for bias. This reminds me of a classical, didactical examples on ITT: a trial comparing surgery with medical treatment to prevent stroke (where more patients in the surgery group died but were excluded from analysis and therefore did not count as events...). What do the authors think? Response: We thank the reviewer for highlighting this point and for providing the illustrative example. The requirement for having at least one post-randomization colonoscopy for inclusion in the analysis does have the potential for bias as it assumes that those individuals who did not undergo a surveillance colonoscopy did not have a higher rate of metachronous neoplasia(s). This has the potential to artificially increase the efficacy of chemoprevention agents. However, we believe the impact of this on our results would be very small, if any. First, the rate of attrition across trials was small (median, 10.5%; IQR, ), and non-differential between active agent and placebo in individual trials (<5% difference in rates of attrition). Second, to account for any potential risk of bias due to use of mitt analysis, we performed a worst-case scenario assumption sensitivity analysis wherein all individuals who did not undergo a post-randomization colonoscopy were assumed to have developed advanced metachronous neoplasia (etable 8). The results of this sensitivity analysis were similar to the primary mitt analysis. Changes in manuscript: Online supplement, etable 8. Comment 2. What is the rationale for restricting the follow-up to 40 months (especially in the trials with extension? or does extension in this case mean follow-up without chemoprevention?)? Also, was this cut-off only used for trials that allowed for re-randomization and extension or all trials? Response: The development of metachronous neoplasia is time-dependent and increases over time. The rationale for restricting follow-up to 40 months was to create a more uniform time interval for assessment. Most societal guidelines recommend a surveillance colonoscopy in 1-3 years for individuals with baseline advanced neoplasia. Thus, we felt a 40-month time interval for assessment was most in keeping with guidelines and allowed for a more uniform assessment across studies to allow for a comparison of treatment efficacy. Additionally, extension trials were more heterogeneous, with some involving treatment-free follow-up (Arber, et al. Five-Year Analysis of the Prevention of Colorectal Sporadic Adenomatous Polyps Trial. Am J Gastroenterol 2011; 106: ) and others leaving the option of continuing or

4 stopping medication to patients (Bertagnolli, et al. Five-Year Efficacy and Safety Analysis of the Adenoma Prevention with Celecoxib Trial. Cancer Prev Res 2009;2: ). We therefore applied the 40-month cut-off to all trials, including those who allowed for re-randomization or extension after 3 years. Changes in manuscript: Methods, page 10, para 1 Comment 3. The definition of SAE is different to the regulatory definition. This is just a comment and not a critique. But it might worth mentioning this explicitly. Response: We thank the reviewer for highlighting this, and recognize variability in reporting SAE in individual studies. We used our best judgment to standardize this definition across trials, including obtaining data from individual study authors. Changes in manuscript: Discussion, page 24, para 1 Comment 4. The authors used the DerSimonian-Laird model to pool direct comparisons. Given the low number of trial per comparison I do not think this is a valid/useful approach. Why not use a Bayesian approach also for the direct comparisons (maybe using informative priors for the between-trial heterogeneity; see the work by Turner RM et al. Int J Epidemiol 2012; 41: 818. and Stat Med 2015; 34: 984.)? Response: We appreciate the reviewer s comment. We feel there is merit in following the traditional frequentist approach for pairwise meta-analysis, including widely understood methodology and easy reproducibility. We acknowledge that the DerSimonian-Laird model may overestimate significance when the number of studies is small, but pairwise meta-analysis is only a small part of this larger Bayesian network meta-analysis, where effect sizes are primarily drawn from mixed estimates. In the revised manuscript, we have reported sensitivity analysis using the Hartung-Knapp method to overcome potential type I error associated with the conventional DL method (etable 4). Using this approach, the summary estimates were consistent with the primary method, though with wide 95% CI, comparisons were not statistically significant. With the use of network meta-analysis, the precision of summary estimates were increased. Changes in manuscript: Methods, page 11, para 1; Results, page 16, para 1; Online supplement, etable 4 Comment 5. Can you provide details on how you pooled rates/proportions from single arms (placebo)? Maybe in the webappendix. Response: Rates/proportions from single arms (placebo) were weighted for each study, and numerators and denominators were added up, to estimate unadjusted pooled rates of serious adverse events in placebo arms. This information has been added to the emethods section of the Online Supplement. Changes in manuscript: Online supplement, emethods, page 5, para 2 Comment 6. Please explain what you mean by "Two studies limited recruitment to individuals with prior CRC"? I thought this was an inclusion criterion. Or does this mean that the other trials allowed for neoplasia and CRC? Response: We apologize for the confusion. We included trials in which participants had previously had any resected colorectal neoplasia (adenoma, advanced adenoma and/or colorectal cancer). The median rate of advanced colorectal neoplasia in included trials was 44% (IQR, 23-68). Two trials included only patients with prior colorectal cancer these were included in the primary analysis. However, since these patients may inherently be at higher risk of metachronous advanced adenomas and may have adopted lifestyle measures that may modify risk of metachronous neoplasia, we performed sensitivity analysis after excluding these two studies. We have clarified our statement to better reflect this difference. Changes in manuscript: Results, page 14, para 2 Comment 7. Please be consistent in your wording (advanced metachronous neoplasia or high-risk adenoma) Response: We apologize for the confusion. We have revised the manuscript and high-risk adenoma has been replaced with high-risk neoplasia (neoplasia incorporates both advanced and non-advanced adenomas as well as colorectal cancer). Advanced metachronous neoplasia and high-risk neoplasia are not synonymous within our study. High-risk neoplasia at baseline implies participants had resected advanced adenomas or colorectal cancer prior to study entry, and were at potentially higher risk of developing metachronous neoplasia on follow-up [which could be advanced neoplasia, or non-advanced adenomas]. Thus, we have left the remainder of the wording as is throughout the manuscript. Changes in manuscript: Results, page 16, para 2 Comment 8. Funnel plots are used to investigate small-study effects not necessarily publication bias Response: We thank the reviewer for highlighting this important distinction, similar to comment #1 by Reviewer 1. As suggested, we have rephrased our statements within the Methods and Results section to reflect that the funnel plot was used to assess for small-study effects. Changes in manuscript: Methods, page 11, para 1; Results, page 18, para 2; Online supplement, efigure 6 Comment 9. It would be good to also graph uncertainty in the estimates provided in Figure 2. Response: We appreciate the reviewer s comment. Besides the SUCRA plots for efficacy and safety, to better highlight uncertainty, we have added median (and 95% CrI) for ranking of agents (in Figure 2), and have provided rankograms in the Online Supplement (efigure 5A-C) Changes in manuscript: Figure 2; Online supplement, efigure 5A-C Comment 10. The author state "We observed a strong association between non-aspirin NSAID use and a reduced risk for advanced...". This wording sounds like observational evidence whereas there were 3 trials i.e. there is experimental evidence. The authors need to mention the transitivity assumption in their discussion. Response: Thank you for your comment. As suggested, we have re-worded the sentence to reflect experimental evidence. Additionally, as suggested by Reviewer 1, comment #9, we have detailed the transitivity assumption in the Discussion. Changes in manuscript: Results, page 23, para 2; page 19, para 2 RESPONSE TO REVIEWER #3

5 Comment 1. The PPi involvement in the research has been limited to practioners views of what PPI May input, and potentially the research may have been richer or altered slightly if PPi had been involved. For instance the view of what would be acceptable in terms of risk and side effects. Having said that, I think that patients would be very interested in the information in the study, particularly those who were considered high risk of further neoplasia and cancer. Response: We appreciate the reviewer s comment. We agree that shared decision making with a thorough understanding of patients values and preferences in the context of risks and benefits, would be most suited to this topic of chemoprevention. Future research in this area, by our group and others, is underway to inform ways to apply of our observations. Changes in manuscript: Discussion, page 25, para 2 Comment 2. The interventions all appeared to be feasible and achievable. It could be useful to understand more about the adverse effects and risks, particularly of the NSAIDs detailed in the RCTs reviewed. This would compliment the information in the article and would help with informed choice. Response: Serious adverse events in this meta-analysis were defined as those resulting in: (a) death, (b) adverse eventrelated hospitalization, (c) severe gastrointestinal bleeding, (d) vascular (cardiac or non-cardiac) complications (including myocardial infarction cerebrovascular accidents, etc), (e) discontinuation of therapy due to an adverse event, or (f) those that were graded as serious or severe by original study authors. We observed a significantly higher risk of SAEs with non-aspirin NSAIDs and calcium. Beyond their toxicity reported in the chemoprevention trial setting, extensive indirect evidence from other disciplines have been informative regarding type and severity of side effects with NSAIDs. Changes in manuscript: None RESPONSE TO REVIEWER #4 Comment 1. There is a consistent misunderstanding in the text, where the authors seem to equate SUCRA = probability of being best, while these two correspond to different things. The probability of a treatment being the best for a certain outcome is generally thought to be misleading and its use should be avoided. For example, it is possible to envisage a (not so unlikely) scenario where a treatment has a high probability of being the best, but also a high probability of being the worse. Focusing on the former will give the erroneous impression of the treatment being highly effective. For this reason ranking measures like SUCRAs (or median ranks) should be used instead. SUCRAs do not correspond to the probability of a treatment being the best. Rather, they provide a ranking of the treatments, and they are calculated using the probabilities of each treatment being found in all possible ranks (not just the 1st). So SUCRAs are not probabilities (even though they go from 0 to 1), they are a ranking tool. In fact, they are a transformation of the mean rank of each intervention, brought to 0 to 1 scale. Thus, in page 4, line 48, the authors should rephrase NSAIDs had the highest probability of being ranked best to NSAIDs ranked as the best treatment. Same thing a couple of lines down. Also in page 11 line 14 it writes cumulative ranking (SUCRA) probabilities, which represent their likelihood of being ranked best. This is not true, please rephrase. Also see pages 15 (line 33), 16 (line 22, 50 and 57), page 19 (line 55), page 23 (line 38). In figure 2 please amend the title of the axis, write SUCRA ranking of instead SUCRA probability of. Also rephrase the legend of the figure, where it writes SUCRA rankings between 0 and 1 represent the probability of being ranked highest. Maybe write something like The higher the SUCRA values the better the treatment Response: We thank the reviewer for highlighting this important distinction and have rephrased the interpretation of SUCRA throughout the manuscript. Changes in manuscript: Throughout the manuscript Comment 2. One very important aspect that is missing from this paper is the assessment of transitivity. Transitivity is one of the fundamental assumptions of network meta-analysis, and authors would first need to assess its validity before using NMA. If transitivity does not hold, then NMA results will be invalid. The literature offers guidance on how to assess conceptually and epidemiologically this assumption, see for example Conceptual and Technical Challenges in Network Meta-analysis by Cipriani et al., Is network meta-analysis as valid as standard pairwise meta-analysis? It all depends on the distribution of effect modifiers by Jansen and Naci, etc. For example, in order to assess this assumption the authors can inspect the included trials in terms of important patient characteristics that may modify treatment effects, the inclusion-exclusion criteria etc. If for example all studies performing a treatment comparison AvB have older patients, while all other studies have young patients, and if age is a relative effect modifier, then NMA results will be biased. Response: We completely agree with the reviewer s comments (as also brought up by Reviewer 1, comment #9) that transitivity must be addressed in any network meta-analysis, and had carefully addressed these issues during the study design phase with a strict protocol for inclusion and exclusion criteria (limiting only to patients with prior colorectal neoplasia, and excluding use of chemopreventive agents for primary prevention or their use in patients with familial colorectal cancer syndromes), use of standard doses of putative chemopreventive agents, without co-interventions, and standard assessment of outcomes at predefined time periods. Additionally, in the revised manuscript, we have evaluated the distribution of key potential effect modifiers in chemoprevention trials of colorectal cancer, namely, prior advanced colorectal neoplasia, family history of colorectal cancer and body mass index; these have been reported in etable 2. In included studies, the median and interquartile range proportion of patients with prior advanced colorectal neoplasia and family history of colorectal cancer in 1st degree relatives was 44% (IQR, 23-68) and 21% (IQR, 19-32), respectively. The median of average BMI (and range of average BMI) across trials was 27.6kg/m2 (range, kg/m2). Hence, the trials were generally similar in terms of distribution of important effect modifiers, suggesting that the transitivity assumption is valid. We have explicitly stated this in GRADE assessment of results of network meta-analysis, and none of the comparisons were rated down for intransitivity. Changes in manuscript: Results, page 19, para 1; Online supplement, etable 2 Comment 3. The authors write in page 10 that Publication bias was assessed using funnel plot symmetry. I am not sure that the authors used valid methods here, the description of what they did is rather minimal. Using funnel plot for NMA is not as straightforward as in simple, pairwise meta-analysis. In a pairwise meta-analysis you have AvB from all studies. But in NMA you may have AvB, AvC and BvC. So if you decide to put everything in a funnelplot you first need to decide on the direction (e.g. will you put the relative effects for AvB or BvA?). This is called comparison-adjusted funnelplot. In this case, for example

6 you have low-dose vs high dose aspirin. In what direction are studies with this comparison included in the funnel plot of figure 5? For an example of a funnel plot you can see here, where the direction is defined as older vs. newer treatment. Without some sort of directionality the funnel plot does not make sense. In any case, I would like to see a more detailed description of the actual methods that the authors used. Response: Thank you for the insightful comment. We apologize for missing key information that is required to appropriately interpret the funnel plots we presented. As suggested by the reviewer, we had presented a comparison-adjusted funnel plot using the network graphics package by Chaimani et al. (Chaimani A et al. Graphical tools for network meta-analysis in STATA. PLoS One Oct 3;8(10):e PMID: ). In this presentation, all studies are centered on the summary effect estimate of their respective comparisons [μxy (logor for present study)] which is represented by the vertical red line. Individual study-level effect size is represented by yixy [where X and Y are two study agents]. The green line represents linear regression of the comparison specific differences yi - μxy on the standard error of yi. Outer dotted lines indicate the triangular region within which 95% of studies are expected to lie in the absence of both biases and heterogeneity (logor ± 1.96*standard error). However, we realize that several of our treatment comparisons have only 1 study, hence a comparisoncentered approach would be meaningless for these comparisons (i.e. the estimate in the funnel plot will always be at the center). Hence, we have presented a comparison adjusted funnel plot for comparisons with at least 2 studies, so that a comparison pooled effect is meaningful and we can assess asymmetry around this. Moreover, as the reviewers suggests, to allow the readership to appropriately interpret the data we present, the funnel plot is accompanied by a tabulated list of pairs of comparisons (active vs. control) that explicitly specify the directionality of comparisons included in the funnel plot. As we detail in response to the comments #4 and #5 below, we have excluded the Egger s test due to the limitations with its use, specifically, due to the small number of studies for each comparison and the effect of their directionality while performing this test for the overall study. Changes in manuscript: Methods, page 12, para 1; Results, page 19, para 2; Online supplement, efigure 6 Comment 4. Later in the same paragraph, the authors refer to Egger s test, but they fail to give any detail. Is the Egger test to be applied per each comparison separately? If not, how did they apply it for the whole network? See also previous comment. Response: We appreciate the reviewer s comment. As acknowledged above, in the revised manuscript, we have excluded the Egger s test due to the limitations with its use, specifically, due to the small number of studies for each comparison and the effect of their directionality while performing this test for the overall study. Changes in manuscript: Deleted Egger s test; Methods, page 12, para 1; Results, page 19, para 2; Online supplement, efigure 6 Comment 5. In Page 17, it writes that We did not find any evidence of publication bias, both qualitatively based on funnel plot asymmetry (efigure 5) or quantitatively (Egger s regression test >0.05 for all comparisons), although the number of studies included in each comparison were very small. Does this mean that the Egger test was performed per comparison? But you only have very few studies per comparison, it doesn t make much sense to apply the test for 2 or 3 studies. Response: We agree with the reviewer, and wish to clarify that we did not use Egger s test for individual comparisons due to the small number of studies for each comparison. Specifically, all comparisons have <5 studies, and none of the currently used statistical tests would be appropriate in this scenario. We had used Egger s for the overall study with the directionality of comparisons similar to the funnel plot we presented. However, while responding to the comment #3 by the reviewer, we realize that this approach is dependent upon the directionality of comparisons, and hence may not be appropriate. We have expanded the discussion on small study effects, including publication bias, as suggested, and present a more detailed interpretation of the evidence from the funnel plot while endorsing limitations due to small number of included studies. Changes in manuscript: Deleted Egger s test; Methods, page 12, para 1; Results, page 19, para 2; Online supplement, efigure 6 Comment 6. Abstract, page 4, line 12: here the authors use the abbreviation NSAID for the first time, without explaining what it stands for. I think it would be better to introduce the abbreviation here, as they do in the next paragraph, for RCT. Regarding terminology: the authors use the word direct for meta-analyses and network meta-analyses in a non-standard way. For example, first line of page 7, it writes In this systematic review, we performed direct and Bayesian network metaanalyses. I suggest removing the word direct, as there are no indirect network meta-analyses. Page 10, line 43, it writes: Direct meta-analysis was performed. I suggest rephrasing to Pairwise meta-analyses were performed, as there is no indirect meta-analysis. Page 14, I suggest renaming the paragraph Pairwise meta-analysis instead of Direct metaanalysis. Paper 20, line 17, the same. Page 16, it writes: When assessing comparative efficacy, non-aspirin NSAIDs were superior to all other agents for the prevention of advanced metachronous neoplasia, except low-dose where the association did not reach statistical significance (OR, 1.91; 95% CrI, ) (Table 2). But, according to Table 2, NSAID vs (ASA+Calcium+Vitamin D) is also nonsignificant (OR 1.92, 0.48 to 7.26). Maybe I have misunderstood something? There is a typo in the legend of Figure 1 (it writes Figure 1. Figure 2: ) Response: Thank you for your comment. We have revised the abstract, introducing the NSAID abbreviation. We have also replaced the term direct meta-analysis with pairwise meta-analysis. We agree that the comparison of NSAID vs. ASA+Calcium+Vitamin D is also non-significant, and have revised the statement accordingly. We have also corrected the typos. Changes in manuscript: Throughout the manuscript