Reviewer Assessment Open Access P. Probst, K. Grummich, U. Klaiber, P. Knebel, A.Ulrich, M. W. Büchler, and M. K. Diener* Conflict of interest in randomised controlled surgical trials: Systematic review, qualitative and quantitative analysis DOI 10.1515/iss-2016-0001 Original submission Feb 2, 2016; revised submission Mar 16, 2016; accepted Apr 1, 2016 *Corresponding Author: Markus K. Diener University of Heidelberg, Germany E-mail: markus.diener@med.uni-heidelberg.de Reviewers Comments to Original Submission Reviewer 1: anonymous Feb 29, 2016 Overall Reviewer Manuscript Rating: 75 Is the subject area appropriate for you? Does the title clearly reflect the paper s content? 3 Does the abstract clearly reflect the paper s content? Do the keywords clearly reflect the paper s content? 3 Does the introduction present the problem clearly? Are the results/conclusions justified? 3 How comprehensive and up-to-date is the subject matter How adequate is the data presentation? Are units and terminology used correctly? Is the number of cases adequate? Are the experimental methods/clinical studies adequate? Is the length appropriate in relation to the content? Does the reader get new insights from the article? Please rate the practical significance. 2 Please rate the accuracy of methods. Please rate the statistical evaluation and quality control. Please rate the appropriateness of the figures and tables. Please rate the appropriateness of the references. Please evaluate the writing style and use of language. Revise with Major Modifications N/A 1
The paper deals with a relevant topic regarding transparency in scientific publishing. The question if any conflict of interest is in place is important in order to allow for an adequate assessment of the results by the readership. Therefore, the topic is interesting for the surgical scientific community. However, some questions arise by reviewing the manuscript, which should be answered or elaborated in more detail before publication. 1. The authors refer to their initial publication presenting the primary results of the systematic review project; however, this manuscript is not yet available in order to compare the findings with the given paper. Thus, the authors should try to elaborate in more detail the differences between "industry sponsorhip" and "conflict of interest" in the given paper. On page 3, line 1, it should read "...recently submitted article..." instead of "...recently published article..." 2. The authors should discuss in more detail, how the Instructions for Authors and the request of the journals to state conflicts of interest have changed over time. 3. Did the authors check if funding or potential sources for conflict of interest have been published in the registration of some of the studies, e.g. in the clinicaltrials.gov database?. In the discussion some words regarding the situation of conflict of interest statements in the pharmaceutical study framework compared to the surgical research field should be included. 5. It remains unclear, how the authors evaluated if a conflict of interest statement "helps the reader to draw a conclusion". Therefore, the "qualitative analysis" is lacking some depths as presented. What exactly makes a "statement adequate to judge potential influence on trial validity"? 6. In the "Methods" section, subheading "Inclusion and exclusion criteria" it remains unclear, why studies which "examined the effectiveness of a drug" or "a perioperative nutritional intervention" are included. The authors should clearly elaborate if all studies in the field of surgery, or only studies dealing with medical devices are of interest. To compare the pharmaceutical field with the medical device area might be of special interest; thus, probably a subgroup analysis of both article groups (devices 66,2% versus nutrition or perioperative medication 33,8%) might be considered. 7. In the "Discussion" section, page 10, second paragraph, the authors refer to one model of cooperation - the investigator-initiated study. It would be interesting, how the authors rate a fullysponsored industry trial, as long as all funding and support is published in a transparent fashion. 5. In the "Discussion" section, page 10, last paragraph, the authors are a little bit sloppy by putting together "industry participation", "funding" and "conflict of interest". Clear financial support of a given clinical study project and other sources of conflict of interest are different kettles of fish. The conflict of interest of participating physicians are also consulting fees, travel fees, etc. And - talking about "key opinion leaders" might come from a variety of industry sources, not only from the particular company funding the given study. Thus, conflicts of interest is beyond direct financial funding of studies and is not only covered by "industry participation", but by relationships between industry and academia. This should be elaborated in more clarity and details in the discussion section. 2
Reviewer 2: Florian Lordick Mar 06, 2016 Accept with Minor Revision Overall Reviewer Manuscript Rating: 75 Is the subject area appropriate for you? 3 Does the title clearly reflect the paper s content? Does the abstract clearly reflect the paper s content? 3 Do the keywords clearly reflect the paper s content? Does the introduction present the problem clearly? Are the results/conclusions justified? How comprehensive and up-to-date is the subject matter How adequate is the data presentation? Are units and terminology used correctly? Is the number of cases adequate? Are the experimental methods/clinical studies adequate? 3 Is the length appropriate in relation to the content? Does the reader get new insights from the article? Please rate the practical significance. 3 Please rate the accuracy of methods. 3 Please rate the statistical evaluation and quality control. 3 Please rate the appropriateness of the figures and tables. Please rate the appropriateness of the references. Please evaluate the writing style and use of language. 3 This interesting study investigated the number and quality of potential conflict of interest statements in publications of randomized controlled surgical trials over the last three decades. Some changes should be considered: In general: the article would benefit from some linguistic improvements. Abstract: The first two sentences of the abstract are not very straight forward. They are difficult to understand and not really a good introduction to the purpose of this study. Please try to reword. Why not easy like: "Conflicts of interest may lead to biased trial designs and unbalanced interpretation of study results. We aimed to evaluate reporting of potential conflicts of interest in full publications of surgical randomized controlled trials. Abstract conclusions: as the study focused on surgical trials, I recommend to focus the conclusions on the segment where the authors gained data. The final statements are a bit too broad: E.g. "in medical publishing". This could be more focused. Please consider rewording and focus on publication of surgical trials. You don't really know if it is the same situation in medical oncology, neurology, psychiatry etc. Background: "well-being of the company" is not a good expression. Do you mean "for-profit orientation"? Methods: it should be explained and clearly defined how quantitative and qualitative investigation of conflicts of interest was actually done. 3
Results: "was furnished" is an uncommon expression in this context. Better "was given" or "was disclosed". Author s to Reviewers Comments Mar 16, 2016 Dear Editors, We hereby submit our revised manuscript Conflicts of interest in randomised controlled surgical trials - Systematic review, qualitative and quantitative analysis to Innovative Surgical Sciences. Point-by-point answer to comments: REVIEWER 1 1. The authors refer to their initial publication presenting the primary results of the systematic review project; however, this manuscript is not yet available in order to compare the findings with the given paper. Thus, the authors should try to elaborate in more detail the differences between "industry sponsorship" and "conflict of interest" in the given paper. On page 3, line 1, it should read "...recently submitted article..." instead of "...recently published article..." : Thank you for this important hint. The article was stated to be in press but it is available online as full article. However, we understand the importance to delimit the present study from the other We made changes to the first paragraph of the discussion: Therefore, for this systematic review all information on potential conflicts of interest and not only industrial participation in surgical trials were analysed quantitatively and qualitatively. Further changes were made to the third paragraph of the discussion. For this please see response 8. 2. The authors should discuss in more detail, how the Instructions for Authors and the request of the journals to state conflicts of interest have changed over time. : We agree that this is important information. We included now some of the results of our study about author s information in surgical journals [26]: In 2015 among 6 journals with scope on general and abdominal surgery 22% of journals did not define what constitutes a conflict of interest. Another 36% used standardized definitions mostly the definition of the ICMJE and 2% of journals defined conflict of interest individually in their information for authors. Among the same journals 12% did not ask authors to provide information on potential conflict of interest. In 27% of journals the information was necessary for acceptance of an article and in 61% of journals the information was necessary and was printed with the article. Furthermore, among these 6 journals particularly journals from medical societies had lower editorial demands with regard to disclosure of conflict of interest [26]. 3. Did the authors check if funding or potential sources for conflict of interest have been published in the registration of some of the studies, e.g. in the clinicaltrials.gov database? : For all evaluations the publications of results were the source of information. Protocols or databases were not searched. We wanted to create a picture of reality were in most cases the reader will also not search himself for this specific information if not provided to him in the publication of results.. In the discussion some words regarding the situation of conflict of interest statements in the pharmaceutical study framework compared to the surgical research field should be included. : We agree with the reviewer and added a sentence in the discussion section: The situation in surgery is similar to other medical fields. Also in pharmacological trials potential conflict of interest is underreported and financial but also non-material ties lead to overestimation of treatment effects [11,15]. 5. It remains unclear, how the authors evaluated if a conflict of interest statement "helps the reader to draw a conclusion". Therefore, the "qualitative analysis" is lacking some depths as presented. What exactly makes a "statement adequate to judge potential influence on trial validity"? : We made a major revision of the methods section. Please see response 13. 6. In the "Methods" section, subheading "Inclusion and exclusion criteria" it remains unclear, why studies which "examined the effectiveness of a drug" or "a perioperative nutritional intervention" are included. The authors should clearly elaborate if all studies in the field of surgery, or only studies dealing with medical devices are of interest. To compare the pharmaceutical field with the medical device area might be of special interest; thus, probably a subgroup analysis of both article groups (devices 66,2% versus
nutrition or perioperative medication 33,8%) might be considered. : We agree that the reason for inclusion of some drugs and nutrition trials is described to short in the text and changed it accordingly. The aim was to create a cohort of surgical trials investigating products with inherent industrial interest and therefore mirroring the conflict of interest arising from this collaboration. Please note that this is also described extensively in the published protocol. Moreover, we absolutely agree that such a subgroup analysis is important. We divided into three groups (medical devices, drugs and nutrition) an added the text to the methods, results (text and table 1) and discussion section. We found that trials investigating nutrition published significantly less disclosure statements (9 of 120 trials; 7.5%) than trials investigating drugs (7 of 30 trials; 23.3%) or trials investigating medical devices 77 of 29 trials (26.2%). 7. In the "Discussion" section, page 10, second paragraph, the authors refer to one model of cooperation - the investigator-initiated study. It would be interesting, how the authors rate a fully-sponsored industry trial, as long as all funding and support is published in a transparent fashion. : This is a very important point. At the moment, because of lack of transparency the question cannot be resolved but has surely to be investigated in a few years. We added a statement to the text: [ ] However, the question whether or not investigator-initiated trials are more robust to industry bias than industry-initiated trials cannot be answered yet because a sufficient amount of transparent data on trial funding is lacking. 8. In the "Discussion" section, page 10, last paragraph, the authors are a little bit sloppy by putting together "industry participation", "funding" and "conflict of interest". Clear financial support of a given clinical study project and other sources of conflict of interest are different kettles of fish. The conflict of interest of participating physicians are also consulting fees, travel fees, etc. And - talking about "key opinion leaders" might come from a variety of industry sources, not only from the particular company funding the given study. Thus, conflicts of interest is beyond direct financial funding of studies and is not only covered by "industry participation", but by relationships between industry and academia. This should be elaborated in more clarity and details in the discussion section. : We agree that this point needed clarification in the We reworded and added some sentences with your apt arguments: An industry involvement always generates a potential conflict of interest. However, the proportion of trials with industry funding disclosing potential conflict of interest was not higher. Besides industry funding, there are other financial or ideal bindings which may create a conflict of interest. Researchers receiving consulting fees, travel fees or are tied in non-material ways have also a potential conflict of interest. Furthermore, there are "key opinion leaders" which perform professional lobbying for different industrial partners which should be transparently disclosed. Therefore, both information i.e. the funding source and conflict of interest in general are crucial to transparency. This was also the main reason to assess them in two separate studies [15]. REVIEWER 2 9. In general: the article would benefit from some linguistic improvements. : Thank you for this important advice. The manuscript was double-checked by a professional native speaking language editor. We are of course open for further tangible suggestions by reviewers. 10. Abstract: The first two sentences of the abstract are not very straight forward. They are difficult to understand and not really a good introduction to the purpose of this study. Please try to reword. Why not easy like: "Conflicts of interest may lead to biased trial designs and unbalanced interpretation of study results. We aimed to evaluate reporting of potential conflicts of interest in full publications of surgical randomized controlled trials. : We agree with reviewer 2 and changed the sentence according to his suggestion. 11. Abstract conclusions: as the study focused on surgical trials, I recommend to focus the conclusions on the segment where the authors gained data. The final statements are a bit too broad: E.g. "in medical publishing". This could be more focused. Please consider rewording and focus on publication of surgical trials. You don't really know if it is the same situation in medical oncology, neurology, psychiatry etc. : We changed medical to surgical which concisely describes the purpose of our study. 12. Background: "well-being of the company" is not a good expression. Do you mean "for-profit orientation"? : Thanks for this suggestion which was implemented in the 5
13. Methods: it should be explained and clearly defined how quantitative and qualitative investigation of conflicts of interest was actually done. : We made a major revision to the methods section. The second paragraph was split up to the first and the last paragraph. The last paragraph is now called data synthesis and lists explicitly the quantitative and qualitative analysis and how they were defined and made: Data synthesis Quantitative analysis The proportion of trials disclosing conflicts was analysed over time. The absolute numbers and proportions of studies with conflicts of interest were recorded for the entire period and for the three periods of 1985-199, 1995-200 and 2005-201. In a chi-squared test at a level of significance of 5% it was tested whether the proportion of trials disclosing a potential conflict of interest increased with the passage of time. Further, it was investigated how many industry-funded trials disclosed a potential conflict of interest. This information is given in given in absolute and relative terms. Moreover, it was examined whether or not trials published by ICMJE-associated journals more frequently provided information on potential conflicts of interest. The association of conflicts of interest with industry funding and ICMJE-association was checked in a chi-squared test. Furthermore, the proportion of reported conflicts of interest in trials with medical devices was compared to those investigating drugs and nutrition in a chi-squared test. Qualitative analysis For qualitative analysis it was judged whether or not the presented information helped the reader in critical appraisal of the trial. Simple terms like none or no conflict of interest without a clear statement of how conflict of interest was defined, were considered as unsupportive. Specifically, in presence of industry funding the statement had to explicitly define how this may have affected the trial. Statements were considered as helpful when they clearly defined involved secondary interests in study planning, conduct and analysis. The statistical analysis was performed using R (Version 3.1.2) [20]. 1. Results: "was furnished" is an uncommon expression in this context. Better "was given" or "was disclosed". : We changed "was furnished into was disclosed. ADDITIONAL CHANGES Please not that an acknowledgements section has been added at the end of the main body of the text: The first author received the Müller-Osten Price 2016 at the annual congress of the German Society of Surgery for his contribution to literature about the history and influence of conflict of interest in surgery. We thank the reviewers for their constructive criticism which improved our manuscript! Thank you for the opportunity to revise our manuscript and we appreciate your consideration for publishing our article in Innovative Surgical Sciences. Reviewers Comments to Revision Reviewer 1: anonymous Apr 01, 2016 Overall Reviewer Manuscript Rating: 80 Is the subject area appropriate for you? Does the title clearly reflect the paper s content? Does the abstract clearly reflect the paper s content? Do the keywords clearly reflect the paper s content? Does the introduction present the problem clearly? Are the results/conclusions justified? How comprehensive and up-to-date is the subject matter How adequate is the data presentation? Accept 6
Are units and terminology used correctly? Is the number of cases adequate? Are the experimental methods/clinical studies adequate? 3 Is the length appropriate in relation to the content? Does the reader get new insights from the article? Please rate the practical significance. 2 Please rate the accuracy of methods. 3 Please rate the statistical evaluation and quality control. 3 Please rate the appropriateness of the figures and tables. Please rate the appropriateness of the references. Please evaluate the writing style and use of language. N/A Thank you for submission of the revised version. From my point of view, all questions have been answered sufficiently and all open points are now addressed in the revised version. Reviewer 2: Florian Lordick Mar 16, 2016 Accept Overall Reviewer Manuscript Rating: 90 Is the subject area appropriate for you? Does the title clearly reflect the paper s content? Does the abstract clearly reflect the paper s content? Do the keywords clearly reflect the paper s content? Does the introduction present the problem clearly? Are the results/conclusions justified? How comprehensive and up-to-date is the subject matter How adequate is the data presentation? Are units and terminology used correctly? Is the number of cases adequate? Are the experimental methods/clinical studies adequate? Is the length appropriate in relation to the content? Does the reader get new insights from the article? Please rate the practical significance. 3 Please rate the accuracy of methods. Please rate the statistical evaluation and quality control. Please rate the appropriateness of the figures and tables. Please rate the appropriateness of the references. Please evaluate the writing style and use of language. no further comments 7