PEER REVIEW HISTORY BMJ Open publishes all reviews undertaken for accepted manuscripts. Reviewers are asked to complete a checklist review form (http://bmjopen.bmj.com/site/about/resources/checklist.pdf) and are provided with free text boxes to elaborate on their assessment. These free text comments are reproduced below. TITLE (PROVISIONAL) AUTHORS REVIEWER GENERAL COMMENTS ARTICLE DETAILS Full-text publication rate of abstracts presented at the Japan Primary Care Association Annual Meetings (2010-2012): a retrospective observational study Komagamine, Junpei; Yabuki, Taku VERSION 1 REVIEW Katherine Harding Eastern Health / La Trobe University, Australia 14-Feb-2018 This paper describes publication rates from one conference in Japan. The data raises some interesting issues about high discrepancy between low publication rates for this conference compared to data presented elsewhere, but findings about the factors associated with publication appear to replicate findings of previous studies. One of my main concerns about this paper is whether the findings have enough interest for an international audience. I think that this needs to be considered in the introduction and in the discussion. For example, the authors are encouraged to consider the questions: Why is this of interest to anyone other than Japanese Primary Care Practitioners? To what extent might the findings be useful to people working in other settings? How might the findings be used to inform strategies to increase publication rates from conferences? Currently, the paper sheds little light on whether the findings are an indication of an issue with this conference alone, or whether they reflect a broader issue. Introduction The authors begin by explaining why primary care is important, and the need for better primary care systems. They then go on to explain that only about half of papers presented at scientific conferences are published, and then state that Therefore, it is important to improve the abstract-to-publication ratio of these scientific meetings in order to apply the research results to patient care. Currently this seems to be a bit of a leap what evidence is there that publishing papers leads to better application to care? What proportion would be considered a good rate? Translation of research to practice is well known to be a huge challenge facing healthcare. I am not suggesting that there aren t some good reasons why publication is a good idea, but I am not convinced that a general need to increase publication rates is the main purpose or justification for this study. Another important issue to cover in the introduction is to briefly 1
summarise current evidence and to highlight gaps in knowledge. The Cochrane review cited in the paper provides good evidence about rates of publication from conferences from a variety of studies/clinical areas, and features of the studies that are associated with publication. However it doesn t tell us about the specific experience of primary care in Japan or anything about personal factors related to publication rates. Other studies may partially answer the latter question (there is some suggestion from literature included in the discussion that suggests this is the case) but are there gaps remaining? I think the introduction could be rewritten to provide a more logical argument for the current study. That is, identify the strengths and gaps in current knowledge, and clearly state how this study addresses those gaps. My other concern with the introduction is that it sets up the paper with a significant focus on one particular conference, in one cultural (Japan) and clinical (Primary Care) setting. This is likely to be of interest to primary care practitioners in Japan, but maybe less so for an international and more general audience. However, the paper is more interesting from a broader perspective, particularly in terms of the personal factors that are associated with publication. These findings are likely to apply to other conferences and translated across different settings. Therefore a stronger emphasis on this part of the research question in the introduction would help to ensure that this paper has broader appeal. Throughout the paper the conference name is written in full this may be a journal style preference, but given the frequency of use I would have expected that the abbreviation JPCAM would be justified in this case. Methods Inclusion criteria: It is not clear to me what the authors mean by the following statement: abstracts published before the deadline of abstract submission of the Japan Primary Care Association Annual Meeting were excluded. Does that mean peer reviewed articles describing the same work as abstracts submitted to the conference that were published in advance of submission of abstracts to the conference were excluded? If so, I would like to see the authors justify this decision. Sometimes people do present work at conferences that has already been published, and I am not sure why this is a problem. Surely the important thing is whether or not the work presented at the conference is being disseminated more broadly as an academic publication. The same applies to publications between submission of abstract and the conference. Also, why make a distinction between publication before the conference and before abstract submission deadline? I think this is unnecessarily confusing. Why not just say articles published before the conference were excluded? Why was Medline the only database that was searched? I would have expected that work presented at a Primary Care Conference could be published in nursing or allied health journals, or in the mental health literature. There are many journals in these fields that are indexed on databases such as PsychInfo and CINAHL. I note 2
REVIEWER that this is acknowledged as a limitation, but wonder why this decision was made. Even searching specific databases for abstracts that fit certain criteria (eg searching for those addressing mental health conditions in Psychinfo) on additional databases would appear to be a worthwhile strategy for this study. Results Results are clearly presented. Discussion The discussion raises some interesting issues but I think could be reorganised to focus on those that are most interesting to a broad audience. Currently the first part of the discussion is largely a repeat of the key points made in the results. The later section what do the results mean is much more interesting - I suggest removing the subheading and moving this content up into the body of the discussion, before the limitations. This is the most interesting part of the discussion and therefore should be given a more prominent place earlier in this section. Other things that would be interesting to include are some discussion of how the findings could be used to inform strategies to increase publication rates. For example: if university affiliated authors are more likely to publish, does this mean that partnering universities with primary care practices might be one strategy that could be helpful? Is there any evidence from the literature that this is the case? Why do oral presentations have a higher rate of publication? Is it simply that higher quality work tends to be accepted for oral presentation? Or maybe oral presenters get positive reinforcement and encouragement from the peers that that poster presenters don t, and that leads to increase likelihood of publication. I have some concerns about the statement This finding implies that the low publications rate of abstracts presented at the Japan Primary Care Association Annual Meetings is one of the causes of the low proportion of Japanese articles in the five high-impact international primary care journals.[24] Although it is clear that both of these things are happening at the same time, I don t think that it is possible to say that one is the cause of the other. Perhaps it would be better to argue that these findings support previous research showing that publication rates in general are low from primary care practitioners in Japan, and suggest some reasons why this might be the case. This has been touched on in the meaning of the results section, but I suggest it warrants a stand-alone paragraph earlier in the discussion. The discussion raises several points pointing to previous evidence that people are more likely to publish if the have given oral presentations or are university affiliated. I think some of these knowns need to be better summarised in the introduction so that a case can clearly be made for the gap that this study is addressing. See comments in introduction. Ronald Rousseau KU Leuven, Belgium 17-Feb-2018 3
GENERAL COMMENTS REVIEWER GENERAL COMMENTS REVIEWER GENERAL COMMENTS This is a rather simple, but nevertheless interesting study with some valid practical conclusions. The title of this submission is misleading in my opinion. I thought that it was a study of publishing abstracts, but it was not. It was a study of full-text publications derived from presented abstracts. It turned out further (p.9) that also brief reports and research letters were included. Is a brief report in Medline the same as a meeting abstract in the Web of Science? I assume not, as otherwise authors could have published their abstracts without any change. I see no reason why they would not do so, if this opportunity would exist. Note though that I am not in favour of publishing meeting abstracts as they are rarely cited (and hence have no influence on the field). See: XJ. Hu and R. Rousseau (2013). Meeting abstracts: a waste of space? Current Science, 105(2), 25 July 2013, 150-151. As to the question of why such a low percentage of abstracts are published as full articles I offer the following suggestions: 1) a language barrier 2) many studies are replication/confirmation studies (are they?). These have great value, but are sometimes difficult to publish 3) colleagues not working at universities (the majority) are not interested in publishing. They only care about their reputation among their immediate peers (by presenting at the conference). 4) Maybe there are in Japan local journals for practitioners (there are in my country) in which such findings are published. Such journals are not covered by international databases, but nevertheless reach the colleagues that should be reached. Some of the points mentioned as limitations are just of no importance in this investigation. Examples are: the possibility that some abstracts were presented at multiple meetings. If these other meetings are not organized by the Japan Primary Care Association then it does not matter; if they were organized by the JPCA then the authors have searched for them more than once, and found them. If authors publish and do not present at the annual meeting is not the topic of this investigation. Murat Şakir Ekşi Acıbadem University School of Medicine, Turkey 27-Feb-2018 The authors presented a well-structured study in a detailed fashion. Sarah Lensen University College London, UK 07-Mar-2018 Thank you for the opportunity to review this manuscript which describes a retrospective cohort study of conference abstracts, and which factors influenced the likelihood of subsequent full-text publication. I have a few comments: 1. The choice of predictive factors to investigate is limited and not rationalised. A number of them relate to the abstract authors, however the authors did not investigate the research design used or the significance of the abstract results. It would have been 4
interesting to see if the RCT design was predictive of publication, or at least research involving human consent as lack of publication here introduces more risk in terms of research waste and ethics. Further, a major issue of publication deficit is publication bias whereby abstracts with positive results may be more likely to reach full-text publication than abstracts which report (boring) negative or null results. Why weren t these factors investigated? This leads to a bigger issue than simply a publication deficit. 2. The second issue is the calculation of median time to publication; it does not appear this has been done as a time-to-event analysis. Calculation of a simple median here is not statistically valid. 3. The measure of inter-person reliability in searching for the abstracts is fine, but the subsequent decision to only search one database and to reduce the search terms to the first two author names only is not explained and a limitation of the study. Further, why weren t the title or key-words of the abstract used in this search? It must be rather difficult to find related full-text papers from only an author s name???? 4. This sentence is a bit misleading and ambiguous Abstracts were considered unpublished if no positive result was obtained from this search strategy 5. This sentence doesn t seem to be quite accurate Before beginning this study, we conducted a preliminary study to evaluate the interobserver reliability for the outcome assessment. It wasn t before the study, it was the first part of the study? 6. Lines 202-211 you present the univariate results as actual OR and 95% CI but then you don t present those for the (arguably more important) multivariate analyses? VERSION 1 AUTHOR RESPONSE Response to Reviewer #1 (Katherine Harding) Comment: This paper describes publication rates from one conference in Japan. The data raises some interesting issues about high discrepancy between low publication rates for this conference compared to data presented elsewhere, but findings about the factors associated with publication appear to replicate findings of previous studies. One of my main concerns about this paper is whether the findings have enough interest for an international audience. I think that this needs to be considered in the introduction and in the discussion. For example, the authors are encouraged to consider the questions: Why is this of interest to anyone other than Japanese Primary Care Practitioners? To what extent might the findings be useful to people working in other settings? How might the findings be used to inform strategies to increase publication rates from conferences? Response: Thank you for your valuable comments. According to your suggestions, we modified the Introduction and Discussion thoroughly. In the present study, the publication rate of abstracts was much lower than that of past studies conducted outside Japan regarding similar specialties. Furthermore, known factors positively associated with publication are unlikely to account for the extremely low publication rate in this study. This means that unknown factors can strongly affect the abstract publication. Therefore, we think that our findings will be interesting to an international audience other than Japanese primary care practitioners. In addition, the strategies to increase the publication rate of abstracts presented at conferences were discussed in the revised manuscript (Pages 17-18, lines 268-292). Comment: Currently, the paper sheds little light on whether the findings are an indication of an issue with this conference alone, or whether they reflect a broader issue. 5
Response: Thank you for your comment. We agree with you. According to your suggestion, we revised the manuscript to make it more relevant to a broader audience. Introduction: Comment: The authors begin by explaining why primary care is important, and the need for better primary care systems. They then go on to explain that only about half of papers presented at scientific conferences are published, and then state that Therefore, it is important to improve the abstract-topublication ratio of these scientific meetings in order to apply the research results to patient care. Currently this seems to be a bit of a leap what evidence is there that publishing papers leads to better application to care? What proportion would be considered a good rate? Translation of research to practice is well known to be a huge challenge facing healthcare. I am not suggesting that there aren t some good reasons why publication is a good idea, but I am not convinced that a general need to increase publication rates is the main purpose or justification for this study. Response: Thank you for your comment. According to your suggestion, we rewrote the Introduction to justify the present study (Pages 6-7, lines 62-91). Comment: Another important issue to cover in the introduction is to briefly summarise current evidence and to highlight gaps in knowledge. The Cochrane review cited in the paper provides good evidence about rates of publication from conferences from a variety of studies/clinical areas, and features of the studies that are associated with publication. However it doesn t tell us about the specific experience of primary care in Japan or anything about personal factors related to publication rates. Other studies may partially answer the latter question (there is some suggestion from literature included in the discussion that suggests this is the case) but are there gaps remaining? Response: Thank you for your comments. We also think that known factors associated with publication, which the Cochrane review reported, do not explain our findings. According to your suggestions, we summarized the current evidence and highlighted gaps in knowledge in the Introduction (Pages 6-7, lines 62-91). Comment: I think the introduction could be rewritten to provide a more logical argument for the current study. That is, identify the strengths and gaps in current knowledge, and clearly state how this study addresses those gaps. Response: Thank you for your comments. According to your suggestions, we rewrote the Introduction to provide a more logical justification for the current study (Pages 6-7, lines 62-91). Comment: My other concern with the introduction is that it sets up the paper with a significant focus on one particular conference, in one cultural (Japan) and clinical (Primary Care) setting. This is likely to be of interest to primary care practitioners in Japan, but maybe less so for an international and more general audience. However, the paper is more interesting from a broader perspective, particularly in terms of the personal factors that are associated with publication. These findings are likely to apply to other conferences and translated across different settings. Therefore a stronger emphasis on this part of the research question in the introduction would help to ensure that this paper has broader appeal. Response: Thank you for your comments. According to your suggestions, we modified the Introduction (Pages 6-7, lines 62-91). Comment: Throughout the paper the conference name is written in full this may be a journal style preference, but given the frequency of use I would have expected that the abbreviation JPCAM would be justified in this case. Response: Thank you for your comment. According to your suggestion, we used the abbreviation JPCAM in the revised manuscript. Methods: 6
Comment: Inclusion criteria: It is not clear to me what the authors mean by the following statement: abstracts published before the deadline of abstract submission of the Japan Primary Care Association Annual Meeting were excluded. Does that mean peer reviewed articles describing the same work as abstracts submitted to the conference that were published in advance of submission of abstracts to the conference were excluded? If so, I would like to see the authors justify this decision. Sometimes people do present work at conferences that has already been published, and I am not sure why this is a problem. Surely the important thing is whether or not the work presented at the conference is being disseminated more broadly as an academic publication. The same applies to publications between submission of abstract and the conference. Also, why make a distinction between publication before the conference and before abstract submission deadline? I think this is unnecessarily confusing. Why not just say articles published before the conference were excluded? Response: We apologize for confusing you. In similar past studies investigating the publication rate of abstracts presented at conferences, different exclusion criteria were used. Most past studies fell into the following three categories: 1) studies that did not report whether abstracts published before the conference were included; 2) studies that excluded abstracts published before the abstract submission deadline of the conference; and 3) studies that excluded abstracts published before the conference. Therefore, we stated the reasons why we chose the third exclusion criteria in the Methods section. However, as you pointed out, our explanation was confusing for readers. Therefore, we modified the Methods section in the revised manuscript (Page 8, lines 103-104). Comment: Why was Medline the only database that was searched? I would have expected that work presented at a Primary Care Conference could be published in nursing or allied health journals, or in the mental health literature. There are many journals in these fields that are indexed on databases such as PsychInfo and CINAHL. I note that this is acknowledged as a limitation, but wonder why this decision was made. Even searching specific databases for abstracts that fit certain criteria (eg searching for those addressing mental health conditions in Psychinfo) on additional databases would appear to be a worthwhile strategy for this study. Response: Thank you for your comments. A recent study regarding conferences for general internal medicine also used Medline as the only search database (J Gen Intern Med 2017;32(6):673-8). Another recent study regarding the Nordic Congress of General Practice (Scand J Prim Health Care 2017;35(1):84-88) used Medline and Google scholar. Therefore, we used Medline and Google scholar as search databases for the initial 100 abstracts in the preliminary study. However, we could identify all full-text publications corresponding to presented abstracts by using Medline only. Therefore, we decided to use Medline as the only database to search publications. Nonetheless, as you noted, we think that using a single database to search for publications was one of limitations in this study (Page 19, lines 300-301). Discussion: Comment: The discussion raises some interesting issues but I think could be reorganised to focus on those that are most interesting to a broad audience. Currently the first part of the discussion is largely a repeat of the key points made in the results. The later section what do the results mean is much more interesting - I suggest removing the subheading and moving this content up into the body of the discussion, before the limitations. This is the most interesting part of the discussion and therefore should be given a more prominent place earlier in this section. Response: Thank you for your comment. According to your suggestion, we removed the subheading and modified the Discussion in the revised manuscript. Comment: Other things that would be interesting to include are some discussion of how the findings could be used to inform strategies to increase publication rates. For example: if university affiliated authors are more likely to publish, does this mean that partnering universities with primary care practices might be one strategy that could be helpful? Is there any evidence from the literature that this is the case? 7
Why do oral presentations have a higher rate of publication? Is it simply that higher quality work tends to be accepted for oral presentation? Or maybe oral presenters get positive reinforcement and encouragement from the peers that that poster presenters don t, and that leads to increase likelihood of publication. Response: Thank you for your comments. According to your suggestions, we discuss how the findings could be used to inform strategies to increase publication rates in the revised manuscript (Pages 17-18, lines 268-292). Comment: I have some concerns about the statement This finding implies that the low publications rate of abstracts presented at the Japan Primary Care Association Annual Meetings is one of the causes of the low proportion of Japanese articles in the five high-impact international primary care journals.[24] Although it is clear that both of these things are happening at the same time, I don t think that it is possible to say that one is the cause of the other. Perhaps it would be better to argue that these findings support previous research showing that publication rates in general are low from primary care practitioners in Japan, and suggest some reasons why this might be the case. This has been touched on in the meaning of the results section, but I suggest it warrants a stand-alone paragraph earlier in the discussion. Response: Thank you for your comments. We agree with you. The statement This finding implies that the low publications rate of abstracts presented at the Japan Primary Care Association Annual Meetings is one of the causes of the low proportion of Japanese articles in the five high-impact international primary care journals was removed, and the Discussion was rewritten. Comment: The discussion raises several points pointing to previous evidence that people are more likely to publish if the have given oral presentations or are university affiliated. I think some of these knowns need to be better summarised in the introduction so that a case can clearly be made for the gap that this study is addressing. See comments in introduction. Response: Thank you for your comment. According to your suggestion, we summarized some of known factors positively associated with publication in the Introduction in the revised manuscript (Page 6, lines 70-79). Response to Reviewer #2 (Ronald Rousseau) Comment: The title of this submission is misleading in my opinion. I thought that it was a study of publishing abstracts, but it was not. It was a study of full-text publications derived from presented abstracts. It turned out further (p.9) that also brief reports and research letters were included. Is a brief report in Medline the same as a meeting abstract in the Web of Science? I assume not, as otherwise authors could have published their abstracts without any change. I see no reason why they would not do so, if this opportunity would exist. Note though that I am not in favour of publishing meeting abstracts as they are rarely cited (and hence have no influence on the field). See: XJ. Hu and R. Rousseau (2013). Meeting abstracts: a waste of space? Current Science, 105(2), 25 July 2013, 150-151. Response: Thank you for your comment. We agree with you. According to your suggestion, we modified the title in the revised manuscript (Page 1, line 1). Comment: As to the question of why such a low percentage of abstracts are published as full articles I offer the following suggestions: 1) a language barrier 2) many studies are replication/confirmation studies (are they?). These have great value, but are sometimes difficult to publish 3) colleagues not working at universities (the majority) are not interested in publishing. They only care about their reputation among their immediate peers (by presenting at the conference). 8
4) Maybe there are in Japan local journals for practitioners (there are in my country) in which such findings are published. Such journals are not covered by international databases, but nevertheless reach the colleagues that should be reached. Response: Thank you for your comments. We agree with you regarding a language barrier. Regarding whether many studies were replication/confirmation studies in this study, abstracts describing results of observational studies reporting new findings were probably the most common in this study. In March 2018, we reevaluated 100 consecutive abstracts presented as oral presentations at the 2012 Japan Primary Care Association Annual Meetings (JPCAM). Of those, 52 abstracts gave us difficulty in evaluating whether they reported new findings, due to a lack of information within the abstracts. Of 48 evaluable abstracts, only seven abstracts reported replication studies. We agree with you regarding a lack of interest in publishing. However, even among abstracts of first authors from university-associated institutions, only 6.4% were subsequently published in peer-reviewed journals. Therefore, we think that lack of interest or presentation of abstracts at the conference as a last goal may be most influential factors for abstract publication, regardless of affiliations associated with university-associated institutions. Regarding the fourth suggestion, the Japan Primary Care Association (JPCA) is one of the major Japanese medical academic organizations, to which more than ten thousand members of physicians belong (Page 7, lines 96-97). Furthermore, JPCA has two official journals ( Journal of the Japan Primary Care Association [in Japanese] and Journal of General and Family Medicine [in English] ). The full-text articles of the former journal are not searchable in the MEDLINE database but are in Google scholar. In a pilot study evaluating 100 abstracts presented at the 2012 JPCAM, no full-text publications went unidentified in MEDLINE but identified in Google scholar. The full-text articles of the latter journal are searchable in MEDLINE. Therefore, we think that the fourth suggestion was unlikely to be a major reason. Comment: Some of the points mentioned as limitations are just of no importance in this investigation. Examples are: the possibility that some abstracts were presented at multiple meetings. If these other meetings are not organized by the Japan Primary Care Association then it does not matter; if they were organized by the JPCA then the authors have searched for them more than once, and found them. If authors publish and do not present at the annual meeting is not the topic of this investigation. Response: Thank you for your comment. We agree with you. We removed the two limitations ( the possibility that some abstracts were presented at multiple meetings and the frequency with which studies were published in peer-reviewed journals without submission to an annual meeting ). Response to Reviewer #3 (Murat Şakir Ekşi) Comment: The authors presented a well-structured study in a detailed fashion. Response: Thank you. Response to Reviewer #4 (Sarah Lensen) Comment 1: The choice of predictive factors to investigate is limited and not rationalised. A number of them relate to the abstract authors, however the authors did not investigate the research design used or the significance of the abstract results. It would have been interesting to see if the RCT design was predictive of publication, or at least research involving human consent as lack of publication here introduces more risk in terms of research waste and ethics. Further, a major issue of publication deficit is publication bias whereby abstracts with positive results may be more likely to reach full-text publication than abstracts which report (boring) negative or null results. Why weren t these factors investigated? This leads to a bigger issue than simply a publication deficit. Response: Thank you for your comments. Only two abstracts described results of randomized controlled trials (RCTs) (0.2%) out of the 1003 included abstracts. Most of them were probably abstracts describing results of quantitative observational research. (In March 2018, we reevaluated 100 consecutive abstracts presented as oral presentations at the 2012 Japan Primary Care 9
Association Annual Meetings (JPCAM). Of those, 76.8% were abstracts describing results of quantitative observational research.) Furthermore, it was difficult to retrieve detailed information on the contents of presented abstracts due to their poor descriptions (Page 10, lines 149-151). Therefore, we decided not to evaluate the research design used or the significance of the abstract results as predictive factors associated with publication. As you pointed out, we think that a lack of investigation into these factors was one of the limitations in this study (Page 19, lines 309-311). Comment 2: The second issue is the calculation of median time to publication; it does not appear this has been done as a time-to-event analysis. Calculation of a simple median here is not statistically valid. Response: According to your suggestion, the median time to publication was recalculated using a time-to-event analysis (Page 11, lines 162-163). Comment 3: The measure of inter-person reliability in searching for the abstracts is fine, but the subsequent decision to only search one database and to reduce the search terms to the first two author names only is not explained and a limitation of the study. Further, why weren t the title or keywords of the abstract used in this search? It must be rather difficult to find related full-text papers from only an author s name???? Response: Thank you for your comments. In the pilot study evaluating the inter-observer reliability for the outcome assessment, we used two databases (MEDLINE and Google Scholar) and the name of the first, second, and last authors as keywords in the search. All identified publications in the pilot study were searchable without using Google Scholar or searching by the last author s name as a keyword (Page 10, lines 140-142). Therefore, we used only the MEDLINE database and searched the first and second authors names as keywords in the present study (Page 10, lines 142-144). In addition, without a searching error, all identified publications in the pilot study were searchable by using the first author s name only as a keyword. One publication was not identified in the search using the first author s name but was identified in search using the second author s name. However, when this search was retried retrospectively, the publication was identified by using the first author s name as a keyword. Considering the possibility of this searching error, we decided to use the names of the first and second authors as keywords. In another study using a similar search strategy, the publication rate of abstracts presented at the Japan Geriatrics Society Annual Meetings was 23.6% (BMC Res Notes 2018;11:36). Therefore, we do not think that the low publication rate in this study was caused due to the defect of this search strategy. The use of the title as a keyword was difficult because titles in the Japanese language needed to be translated to English. Comment 4: This sentence is a bit misleading and ambiguous Abstracts were considered unpublished if no positive result was obtained from this search strategy Response: We apologize for this oversight. We modified this sentence in the revised manuscript (Page 9, line 125). Comment 5: This sentence doesn t seem to be quite accurate Before beginning this study, we conducted a preliminary study to evaluate the interobserver reliability for the outcome assessment. It wasn t before the study, it was the first part of the study? Response: We apologize for this oversight. As you pointed out, it was the first part of the study. We modified this sentence in the revised manuscript (Page 9, lines 128-129). Comment 6: Lines 202-211 you present the univariate results as actual OR and 95% CI but then you don t present those for the (arguably more important) multivariate analyses? Response: According to your suggestion, we also present the multivariate results in the revised manuscript (Page 14, line 209). 10
REVIEWER GENERAL COMMENTS REVIEWER GENERAL COMMENTS REVIEWER GENERAL COMMENTS VERSION 2 REVIEW Ronald Rousseau KU Leuven, Belgium 06-Apr-2018 Response to Reviewer #1 (Katherine Harding) The authors have modified their original submission in an acceptable way. Sarah Lensen University College London, UK 07-Apr-2018 Thank you for the opportuunity to re-view this manuscript. 1. A sentence has been added (line 149 "Due to poor descriptions of abstracts, we did not retrieve information on whether the results described in the abstracts were positive or negative findings." I suggest removing this sentence as it is confusing. In the previous review I queried why publication bias was not investigated, but if the nature of the conference made it difficult to investigate this (as findings may not easily be categorized as positive or interesting etc) then publication bias may not be relevant. But this sentence does not explain this, so I suggest to remove it 2. The authors commented they have recalculated the median using "time to event" analysis but with no description of the statistical test used. Please add this, and I suggest statistical review. 3. Re-writing of the discussion etc. has placed a lot of emphasis on 'personal factors' which may be affecting publication rate. This is speculative and I think too much emphasis has been placed on these 'personal factors'. The conclusion should read that more like "more research is needed to understand why the abstracts don't reach full-text" rather than assuming is personal factors. Katherine Harding 1. Eastern Health Australia 2. La Trobe University, Australia 24-Apr-2018 The authors have provided a well thought out and thorough response, and addressed the concerns raised in my original review. VERSION 2 AUTHOR RESPONSE Comment: The authors have provided a well thought out and thorough response, and addressed the concerns raised in my original review. Response: Thank you very much for your helpful comments. 11
Response to Reviewer #2 (Ronald Rousseau) Comment: The authors have modified their original submission in an acceptable way. Response: Thank you very much for your helpful comments. Response to Reviewer #4 (Sarah Lensen) Comment 1: A sentence has been added (line 149 "Due to poor descriptions of abstracts, we did not retrieve information on whether the results described in the abstracts were positive or negative findings." I suggest removing this sentence as it is confusing. In the previous review I queried why publication bias was not investigated, but if the nature of the conference made it difficult to investigate this (as findings may not easily be categorized as positive or interesting etc) then publication bias may not be relevant. But this sentence does not explain this, so I suggest to remove it. Response: Thank you for your comment. According to your suggestion, this sentence was removed from the revised manuscript. Comment 2: The authors commented they have recalculated the median using "time to event" analysis but with no description of the statistical test used. Please add this, and I suggest statistical review. Response: Thank you for your comment. The median time (months) from the annual meeting to publication was calculated using Kaplan-Meier survival analysis in the former revised version of the manuscript. The month of publication was used as the cut-off point for censoring. However, we feel that the calculation of the time from the annual meeting to the time of publication is not important in this study. Even if this information is absent, our discussion and conclusion remain unchanged. Furthermore, most recent studies (11 of 15) investigating the publication rate of abstracts presented at scientific meetings also reported a simple median or mean time from the annual meeting to publication (Am J Obstet Gynecol 2015;213:405.e1-6, BMC Res Notes 2015;8:492, Digestion 2016;94:215-21, J Exp Orthop 2016;3:21, Can J Ophthalmol 2017;52:343-8, Foot Ankle Int 2017;38:558-63, J Gen Intern Med 2017;32:673-8, Rheumatol Int 2017;37:949-53, Spine 2017;42:1723-9, SICOT J 2017;3:36, J Oncol Sci 2017 [in press] doi:10.1016/j.jons.2017.11.001). Three studies did not calculate the time from the annual meeting to publication (J Gastrointestin Liver Dis 2016;25:533-6, ANZ J Surg 2017 [in press] doi:10.1111/ans.14103, Scan J Prim Health Care 2017;35:84-8). Only one study calculated the median time from the annual meeting to publication using a Kaplan-Meier survival analysis (Am J Surg 2016;211:166-71). Thus, we respectfully disagree with your suggestion. Based on these studies, we present a simple median time (months) from the annual meeting to publication in the revised manuscript (Page 11, lines 162-163), similar to in the original version of the manuscript. Comment 3: Re-writing of the discussion etc. has placed a lot of emphasis on 'personal factors' which may be affecting publication rate. This is speculative and I think too much emphasis has been placed 12
on these 'personal factors'. The conclusion should read that more like "more research is needed to understand why the abstracts don't reach full-text" rather than assuming is personal factors. Response: Thank you for your comment. We agree with you. According to your suggestion, the conclusion was modified in the revised manuscript (Page 20, lines 329-332). Furthermore, the discussion about the effects of personal factors on abstract publication was also modified in the revised manuscript (Pages 16-17, lines 256-270). BMJ Open: first published as 10.1136/bmjopen-2018-021585 on 22 June 2018. Downloaded from http://bmjopen.bmj.com/ on 5 January 2019 by guest. Protected by copyright. 13