Author s response to reviews Title: Intention-to-treat and transparency of related practices in randomized, controlled trials of anti-infectives Authors: Robert Beckett (rdbeckett@manchester.edu) Kathryn Loeser (kcloeser2016@spartans.manchester.edu) Kathryn Bowman (krsnyder2016@spartans.manchester.edu) Trent Towne (tgtowne@manchester.edu) Version: 1 Date: 08 Aug 2016 Author s response to reviews: August 8, 2016 Dear Editorial Board: Please find the attached manuscript, Intention-to-treat and transparency of related practices in randomized, controlled trials of anti-infectives. We are excited to submit this revised research report based on the helpful and insightful suggestions from peer reviewers. Please find the response to each comment below and alterations highlighted in track changes in the manuscript. I appreciate your consideration of our work for publication in BMC Medical Research Methodology. We welcome any feedback or advice the editors might see fit to provide. Please feel free to contact me with any questions. Thank you for this opportunity. Best Regards, Robert D. Beckett, PharmD, BCPS* Assistant Professor of Pharmacy Practice Director of the Drug Information Center Manchester University College of Pharmacy 10627 Diebold Road, Fort Wayne, Indiana rdbeckett@manchester.edu Tel. 260.470.2663 Fax. 260.470.4403 *Corresponding Author
Comment Response: Reviewer #1 1. Your definition of intention to treat (ITT) is not uncontroversial and partly contradicts your reference 4. True ITT occurs when there are no missing measurements, even if some patients have failed to take medication. Such an analysis is often possible where survival is the outcome but is not generally possible. If this is what you meant to say in Table 1, you should say so. If you did not apply this standard in your research, I have no objection to your maintaining the current classification you have provided that you add a brief note covering the point. We have acknowledged this definition in a footnote to Table 1. As you note, it did not appear most of our sample met this standard. 2. I was surprised that another common problem as regards ITT was not mentioned. In some indications it is not possible to establish whether or not a given patient is infected with the organism treatment is designed to combat. It is necessary to initiate anti-infective treatment based on symptoms. Ant-fungal treatment is an example and this approach is sometimes referred to as 'empiric'. An issue then arises as to whether those individuals who are established postrandomisation on the basis of a baseline measurement not to have been infected should be removed from analysis. The argument in favour of removal is that removal is independent of treatment given, and so the usual randomisation theory is intact, and including these individuals in the analysis simply adds irrelevant noise. The counter-argument is that if the treatment has to be given like this in practice, the appropriate denominator for measuring the consequences in terms of difference in cures is patients treated not patients infected. Very often both analyses would be attempted but either can be defended in terms of randomisation theory. This is quite a different matter to removing patients due to non-compliance, since in that case the noncompliance could be a consequence of treatment and there has to be a model-based assumption for doing this, for example, missing completely at random. The authors agree this was an oversight in the discussion and have corrected this by adding a section specifically tied to this problem. While we did not specifically address this issue as part of our data collection it is nonetheless an important concept. The additions have been made in pages 11 and 12. 3. P9 L8 50% were double blind and 46% were open-label. Does that mean that 4% were single blind? Whatever the explanation, please be explicit. Full description of blinding is provided in Table 2. In general we used text descriptions to describe the most common results only. 4. P10 LL 4-9. BOCF and LOCF (and, one could argue, assuming treatment failure) are single-imputation methods. Your description seems to imply that single imputation methods were an alternative to these approaches.
Thank you for the catch. We have clarified that this referred to other single and multiple imputation methods. 5. Words seem to be missing here: a. P2 L 9 "Results of this analysis will help guide end users the primary drug literature" "users to", "users of", "users through"? b. P4 LL 14-15 "For this reason, it has been proposed to be less in non inferiority trials" "Less appropriate"? c. First "sentence" under method lacks a verb a. Removed the work the and changed it to of b. Added the word appropriate as suggested c. Modified to say This was a 6. References that might have been included: a. Committee for Medicinal Products for Human Use (CHMP) (2010). Guideline on Missing Data in Confirmatory Clinical Trials, European Medicines Agency. b. Little, R. J., R. D'Agostino, et al. (2012). "The prevention and treatment of missing data in clinical trials." N Engl J Med 367(14): 1355-60. Thank you for the suggestion. We have incorporated reference to these papers in the discussion addressing treatment of missing data. They were very helpful in providing additional support for our conclusions. Reviewer #2 1. The title of the study and the background and early part of the paper refer primarily to descriptive statistics rather than the real purpose of the paper, well covered in the Discussion section, to relate the findings to published norms such as CONSORT. One simple way to improve the title is simply to eliminate the phrase "Descriptive study of". That is, the title could be simply "Intention-to-treat and transparency of related practices in randomized controlled trials of anti-infectives" or something similar. Also, the background should emphasize that you are not just describing the practices but comparing them to current best practices - as you do nicely in the discussion. Thank you for the suggestion. We have changed the title and alluded to the comparison to best practices in the Introduction.
2. In the text, put the number of studies along with percentages wherever these are mentioned. Number of studies has been added wherever percentages appear in the Results section. 3. A list of the 105 articles reviewed should be made available as supplementary materials. The appendix has been added with appropriate reference. 4. The issue of modified ITT needs further development. It would be helpful to have some type of tabulation of the types of modification you found. It appears that the most common modification was simply to require that the assigned treatment was started in order to include the patient in the analysis. But exactly how often did this occur and what about other types? What are they? Also, you mention in the discussion that "it was anecdotally observed that mitt was frequently mischaracterized as ITT". This needs further exploration. How often did this occur? Is it always possible to determine when this occurred? Mischaracterization of mitt as ITT could be a fairly serious matter if it is a deliberate intent to deceive - something that likely it is not possible to know in any particular paper. In any case, it would be good to know how often this occurs. To your first point, we have elaborated in the fourth Results paragraph to describe all modifications, not just the most common. To your second point, unfortunately we did not anticipate the high rate of mischaracterization and did not formally collect this data in our structured data collection phase. 5. There are some typos and incomplete sentences. Please review the text carefully in this regard. We have carefully proofread the final version of the manuscript. Reviewer #3 1. At least in the discussion the authors should provide a brief overview of the potential bias of mitt trials as compared with ITT ones (Deviation from intention to treat analysis in randomised trials and treatment effect estimates: meta-epidemiological study Abraha et al. BMJ 2015). We have now addressed on this on pg 11-12, and added a section on how multiple changes to a mitt can lead to an almost per-protocol-type of analysis. We cautioned authors and readers alike to review with care. 2. I don't like the idea of selecting the journals "based on anecdotal experience". I would prefer something more accurate such as basing the selection on journal impact factor or other criteria.
We have deleted the reference to anecdotal experience, and added information demonstrating the high impact factors of the selected journals. 3. I think that the authors missed important points in designing the study. By increasing the time span they could investigate also the trend of prevalence of ITT across years and see the modification of the other information. I strongly suggest the authors to consider this study as a starting point and undertake a meta-epidemiologic study in the future by investigating also the potential modification of effect size according to the different approaches used in the RCTs. The authors agree and have added this to page 14 as part of the future directions of this research 4. pg 3 line 10: despite the background of the authors too much emphasis is given to pharmacist figure. "pharmacists and other medical professionals" should be replaced with "medical professionals" The change has been made. 5. pg 5 lines 2-4: another important thing to consider when imputing missing data is to perform a sensitivity analysis. The addition has been made. 6. pg 7 line 4: I don't understand what is the added value of the "PubMed search conducted in February 2015" that has "identified relevant articles published in the 5 nine selected journals" We agree that there is minimal value in indicating the year. We have retained the clarification that we used PubMed to search the journals in case there are questions regarding the search tool (databases vs. journal). 7. pg 8 line 18: "Median and interquartile range were used to describe results for ordinal and non-parametric continuous data." What about discrete values and parametric continuous data? We noted use of number and percentage for discrete values. No parametric data were gathered. 8. pg 8 line 24: Literature screening process should be reported with a flow-diagram. Flow diagram was created and added as Figure 1. Editorial 1. In accordance with BioMed Central editorial policies and formatting guidelines, all submissions to BMC Medical Research Methodology must have a Declarations section which includes the mandatory sub-sections listed below:
- Ethics approval and consent to participate - Consent to publish - Availability of data and materials - Competing interests - Funding - Authors' Contributions - Acknowledgements - Authors' Information We reviewed each section for completeness and made several minor changes.