BMJ Open. For peer review only -

Transcription

1 BMJ Open Assessment of the quality of reporting in randomized controlled trials of acupuncture in the Korean literature using the CONSORT and STRICTA statements Journal: BMJ Open Manuscript ID: bmjopen--000 Article Type: Research Date Submitted by the Author: -Feb- Complete List of Authors: Kim, Kun Hyung; School of Korean Medicine, Pusan National University, Department of Acupuncture and Moxibustion Kang, Jung Won; College of Korean Medicine, Kyung Hee University, Department of Acupuncture and Moxibustion Lee, Myeong Soo; Korea Institute of Oriental Medicine, Lee, Jae Dong; College of Korean Medicine, Kyung Hee University, Department of Acupuncture and Moxibustion <b>primary Subject Heading</b>: Secondary Subject Heading: Keywords: Complementary medicine Complementary medicine, Medical publishing and peer review, Research methods COMPLEMENTARY MEDICINE, EPIDEMIOLOGY, EDUCATION & TRAINING (see Medical Education & Training) BMJ Open: first published as 0./bmjopen--000 on July. Downloaded from on December by guest. Protected by copyright.

2 Page of BMJ Open Assessment of the quality of reporting in randomized controlled trials of acupuncture in the Korean literature using the CONSORT and STRICTA statements Kun Hyung Kim, Jung Won Kang, Myeong Soo Lee, Jae-Dong Lee School of Korean Medicne, Pusan National University, Yangsan, South Korea Department of Acupuncture & Moxibustion, College of Korean Medicine, Kyung Hee University, Seoul, South Korea Korea Institute of Oriental Medicine Running Title: CONSORT and STRICTA for Korean acupuncture trials Source of support: None Financial Disclosure: None Keywords: acupuncture, randomized controlled trial, CONSORT, STRICTA, reporting guideline Corresponding author : Jae Dong Lee, KMD., PhD. Department of Acupuncture & Moxibustion, College of Korean Medicine, Kyung Hee University, Seoul, 0-, South Korea Tel : -(0) -- Fax: -(0) -- ljdacu@gmail.com BMJ Open: first published as 0./bmjopen--000 on July. Downloaded from on December by guest. Protected by copyright.

3 BMJ Open Page of Abstract Objectives: Adequate reporting quality of randomized controlled trials (RCTs) is a prerequisite for comprehensive understanding and optimal utilization of research results. This study aims to assess the reporting quality of RCT of acupuncture in the Korean literature. Design: Systematic review Methods: Twelve Korean databases and seven Korean journals were searched to identify eligible RCTs of acupuncture published from to July. The Consolidated Standards of Reporting Trials (CONSORT) checklist for parallel RCTs and the revised STandards for Reporting Interventions in Clinical Trials of Acupuncture (STRICTA) were used to assess the quality of reporting in Korean RCTs. The completeness of reporting of CONSORT and STRICTA items in papers published in two time periods (-0 as pre-consort and 0- as post-consort) were compared. Results: In total, eligible RCTs were identified and analysed using CONSORT (n=) and STRICTA (n=0). Information related to randomisation sequence generation, allocation concealment and implementation, participant flow, recruitment and reporting of adverse events were significantly more completely reported in the post-consort trials; however, the absolute reporting rate was suboptimal (ranged from. to. % in 0 post- CONSORT trials). The acupuncture rationale, number of needles, depth of insertion, response to needle stimulation, needle stimulation methods and description of acupuncture practitioners were significantly more completely reported in the post-consort trials. Although most of the STRICTA items were reported in 0% of post-stricta trials, the reporting of setting/context (.%) and practitioner background (.%) items remained incomplete. Conclusion: The reporting quality of Korean acupuncture RCTs was suboptimal with regard to the CONSORT and revised STRICTA statements. The inclusion of the CONSORT and BMJ Open: first published as 0./bmjopen--000 on July. Downloaded from on December by guest. Protected by copyright.

4 Page of BMJ Open revised STRICTA statements in author instructions and the use of those statements in writing as well as editorial and peer review processes are necessary to enhance reporting quality in Korean acupuncture RCTs. Strengths and limitations of this study This is the first study that investigated the incompleteness of quality of reporting in Korean RCTs of acupuncture with regard to the CONSORT and STRICTA recommendations and their changes over time. Our findings demonstrate that most of core components of trials remained substantially under-reported in Korean RCTs of acupuncture. The assessment criteria for completeness of reporting in each item may be different with other relevant studies. Future periodical updates of the results are also needed. Keywords: acupuncture, CONSORT, STRICTA, reporting quality, randomised controlled trial BMJ Open: first published as 0./bmjopen--000 on July. Downloaded from on December by guest. Protected by copyright.

5 BMJ Open Page of Background Rigorous randomised controlled trials (RCTs) can minimise bias and thus contribute to the establishment of gold-standard evidence for medical interventions. Well-designed and properly conducted RCTs can also provide raw data for systematic reviews and meta-analyses to supply reliable information to clinicians, patients and policymakers. A complete, accurate and transparent report of RCTs also facilitates dissemination, interpretation, translation and replicability, whereas poorly conducted or reported RCTs impede the improvement of evidence. Given these concerns, the Consolidated Standards of Reporting Trials (CONSORT) statement was established to improve the reporting of RCTs and thus enable readers to understand the study design, conduct, analysis and interpretation through complete transparency. The CONSORT statement was revised in 0 and 0, and the revised STandards for Reporting Interventions in Clinical Trials of Acupuncture (STRICTA) serves as an official extension of the statement for descriptions of acupuncture treatments. Previous studies revealed that adherence to the CONSORT statement improved the reporting quality of trials. Recent studies assessing the reporting quality of RCTs published in languages other than English identified low adherence to the CONSORT statement and suggested the need to improve the quality of RCT reports in China and Japan. In Korea, RCTs for conventional medicine demonstrated very low adherence to the CONSORT statement. However, a study assessing the quality of the reporting of acupuncture RCTs in Korean databases is lacking despite the fact that acupuncture is regularly practiced in Korea, and many clinical trials assessing the effects of acupuncture have been published in Korean databases. A previous study assessing the reporting quality of acupuncture trials reported in English suggested that future research should investigate the reporting quality of acupuncture trials in languages other than English. Hence, we aimed to evaluate the reporting quality of BMJ Open: first published as 0./bmjopen--000 on July. Downloaded from on December by guest. Protected by copyright.

6 Page of BMJ Open acupuncture RCTs published in Korean databases using the CONSORT and revised STRICTA statements. BMJ Open: first published as 0./bmjopen--000 on July. Downloaded from on December by guest. Protected by copyright.

7 BMJ Open Page of Methods Study design The quality of reporting in eligible Korean RCTs of acupuncture was compared before and after publication of the revised CONSORT 0 statements and the first STRICTA recommendation, which was also published in 0. Before-and-after comparisons were made to investigate whether the reporting quality of Korean acupuncture RCTs was altered after the publication of the CONSORT and STRICTA statements in 0. In this study, we compared Korean RCTs published before 0 with those published after 0, which represents the four-year period spanning the publication of the revised CONSORT and the original STRICTA statements. Type of studies Parallel group RCTs reported in Korean databases were eligible for this study. We decided not to include Korean RCTs reported in English databases given that the primary aim of this study was to assess adherence to the CONSORT statement among trials reported in Korean databases that might be unknown due to language restriction. Studies reported in English but recorded in Korean databases were eligible. Crossover or cluster RCTs were excluded because we employed the CONSORT guidelines for parallel RCTs as the main analysis tool for this study. CONSORT and STRICTA statements The revised version of the CONSORT statement for parallel RCTs published in 0 was used to compare the reporting quality of Korean trials before and after the publication of the BMJ Open: first published as 0./bmjopen--000 on July. Downloaded from on December by guest. Protected by copyright.

8 Page of BMJ Open statement. Although the CONSORT initiatives recommend the use of the most recently released version of CONSORT when reporting and analysing RCTs, the revised CONSORT version published in 0 was not used to avoid potential systematic disadvantages for RCTs that were published prior to CONSORT publication in 0. The revised version of STRICTA published in 0 was used to compare the reporting quality of acupuncture intervention components before and after the publication of the original STRICTA recommendation. We did not use the original STRICTA recommendation for this comparison based on the following reasons. First, we found sufficient consistency among the STRICTA components recommendation for this review between the original and revised STRICTA statements, thereby justifying use of the latest STRICTA version. 0 Second, the ultimate aim of the STRICTA statement in this review is not to simply score studies for each STRICTA item but to identify the extent to which Korean RCTs comprehensively report the essential components of acupuncture treatment that are considered to best represent consensus among international acupuncture experts. 0 STRICTA items were not analysed for non-acupuncture control group interventions because our primary interest is to assess the reporting quality of acupuncture treatment components. The original statements were transformed into data extraction checklists for the CONSORT statement with items and the revised STRICTA statement with items. Type of participants RCTs of patients with any health problems or diseases were eligible. RCTs of healthy individuals were excluded. No study was excluded based on its outcome measures. Type of interventions and comparisons BMJ Open: first published as 0./bmjopen--000 on July. Downloaded from on December by guest. Protected by copyright.

9 BMJ Open Page of For the CONSORT analyses, acupuncture was defined as a stimulation of the body or auricular points regardless of the type of stimulation. Studies using acupuncture-like techniques that stimulate acupuncture points (i.e., acupuncture point injection or acupressure) were also included when classified and reported as a type of acupuncture in an RCT. Studies testing moxibustion as a primary intervention were excluded. Any types of control groups were eligible for this analysis. For the STRICTA analyses, acupuncture was defined as needle penetration of the body or auricular points with manual or/and electrical stimulation given that STRICTA was originally developed to assess the components of these interventions. Studies comparing acupuncture as a control group with non-acupuncture interventions were also eligible for this study, and only acupuncture-related information was extracted for the STRICTA analysis. With regard to studies that compare different types of needle-penetration acupuncture interventions, the most comprehensively described acupuncture intervention was selected for the STRICTA analysis. Searching methods The studies included herein were selected from the dataset of Korean RCTs described as previously published. In the article, Korean databases (i.e., NANET, RISSU, KISS, DBpia, KMbase, KoreaMed, KISTI, NDSL, OASIS, DLlibrary, KoreanTK, and RICHIS) were searched from their inception to July. The search terms included acupuncture and clinical trials (Table ). Theses and dissertations were also included if they met the eligibility criteria. Data extraction and assessment BMJ Open: first published as 0./bmjopen--000 on July. Downloaded from on December by guest. Protected by copyright.

10 Page of BMJ Open General characteristics of the included RCTs were extracted. Two reviewers performed data extraction for the assessment of reporting quality independently, and any disagreements were resolved with discussion. Each item was assessed as yes or no based on whether it was reported in the study. If an item has more than one concept to be assessed, item reporting was considered complete when at least one concept was reported in a given trial. For example, item # on the CONSORT checklist assesses the random sequence generation with two concepts (i.e., method used to generate the random allocation sequence, including details of any restrictions). The reporting of the item was considered complete when the sequence generation method was reported regardless of information of restriction method provided in a given RCT. Explanation and elaboration documents of the CONSORT and revised STRICTA statements were used as assessment references. For item # on the CONSORT 0 checklist (i.e., details of intervention intended for each group and how and when they were actually administered), the rating criteria of Hoffmann et al. were adopted with slight modification. We selected four items (i.e., procedure, materials, intensity and schedule) from the checklist that was developed to assess reports of nonpharmacological interventions in RCTs because we considered these items to be most relevant for the replication of acupuncture interventions. If all four items were assessed as yes, item # on the CONSORT 0 statement was rated as yes. Table presents detailed assessment criteria for item # (Table ). Item # on the CONSORT 0 checklist (i.e., blinding of participants, intervention providers and outcome assessors) was modified to only include outcome assessor blinding. Participant and intervention provider blinding are often not feasible in complex interventions, such as acupuncture, whereas outcome assessors can be blinded without interfering with the acupuncture treatment process. BMJ Open: first published as 0./bmjopen--000 on July. Downloaded from on December by guest. Protected by copyright.

11 BMJ Open Page 0 of Statistical analysis For each time period (i.e., before and after 0), general publication details (e.g., sample size, type of journals, acupuncture/non-acupuncture trials, etc.) were compared using t-tests or Chi-Square tests. For each CONSORT and STRICA item, the number and percentage of trials that successfully reported the item as well as the percentage differences between two time periods with binomial % confidence intervals are reported. Multiple linear regression analysis was used to investigate potential predictors of better reporting. Independent variables included year of publication, type of journal (TKM-related or other), sample size and type of acupuncture stimulation (penetration needling or other), and the dependent variable was the CONSORT index score (i.e., aggregate of checklist items). The same independent variables other than the type of acupuncture stimulation and the STRICTA index score (i.e., aggregate of checklist items) as a predictor variable were analysed in the same manner. STATA version.0 (Stata-Corp, College Station, Texas) was used for statistical analyses. BMJ Open: first published as 0./bmjopen--000 on July. Downloaded from on December by guest. Protected by copyright.

12 Page of BMJ Open Results Search results and characteristics of included RCTs In total, studies were screened during the initial process, of which RCTs met the eligibility criteria and were included in the analyses (Figure ). The primary interventions of needle acupuncture and non-needle acupuncture were used in 0 and RCTs, respectively. Non-needle acupuncture stimulation consisted of pharmacopuncture (i.e., the injection of hernal medicine on acupuncture points), bee venom acupuncture (i.e., the injection of diluted bee venom on acupuncture points), acupressure (by hand or device) or electrostimulation of acupuncture points. Additional general characteristics of the included RCTs are provided in Table. Differences in completeness of reporting for CONSORT items between pre- and post- CONSORT statements Among the pre-consort and 0 post-consort trials, 0 and of items were reported in greater than 0% of trials, respectively. The remainder of the items were reported in less than % of included pre- and post-consort trials regardless of the statistical significance of the differences of reporting quality between two time periods. Items regarding random sequence generation, allocation concealment, implementation of randomisation, flow of participants, dates of participant recruitment and adverse events displayed significantly more complete reporting in post-consort trials; however, only two items (random sequence generation and dates of participant recruitment) achieved greater than 0% reporting rates in post-consort trials. The reporting rates of the remaining five items that showed statistically significant improvement in post-consort trials remained markedly incomplete (less than. %). The most significant improvement in reporting quality was identified in random sequence generation (mean difference:.%, % CI:. to.%), BMJ Open: first published as 0./bmjopen--000 on July. Downloaded from on December by guest. Protected by copyright.

13 BMJ Open Page of although the reporting rate was only.% in the post-consort trials. Detailed information regarding the reporting of each item is provided in Table. The CONSORT Index score was significantly increased in post-consort trials compared with pre- CONSORT trials (p=0.00). (Table ) Differences in completeness of reporting for STRICTA items between pre- and post- STRICTA statements Among the pre-stricta and post-stricta trials, 0 and of items were reported in greater than 0% of trials, respectively. The percentage of trials that showed complete reporting in the remaining items ranged from 0.0 to. % in pre-stricta trials and. to.% in post-stricta trials. Items regarding reasoning of acupuncture treatments (item b), number of needles (item a), depth of insertion (item c), response to needle (item d), needle stimulation methods (item e) and description of acupuncture practitioners (item ) displayed significantly more complete reporting in post-consort trials. Greater than 0% of trials achieved complete reporting of these items with the exception of the item related to the description of acupuncture practitioners, which maintained a low reporting rate (.%) in post-stricta trials. The most significant improvement in reporting quality was identified in needling depth (item c) (mean difference:.%, % CI:. to.%), achieving a.% reporting rate in the post-stricta trials. Although most of STRICTA items were reported in 0% of post-stricta trials, reporting of items for setting/context (.%) and practitioner background (.%) remained incomplete. Detailed information regarding the reporting of each item is provided in Table. The STRICTA Index score was significantly increased in post-stricta trials compared with pre-stricta trials (p<0.000) (Table ). BMJ Open: first published as 0./bmjopen--000 on July. Downloaded from on December by guest. Protected by copyright.

14 Page of BMJ Open Predictors of better reporting for the CONSORT and STRICTA statements The variables we examined (year of publication, type of journal, sample size and type of acupuncture stimulation) accounted for minimal variance in the regression model (0. and 0. adjusted R-squared values in the CONSORT and STRICTA regression models, respectively). Year of publication showed significantly modest positive correlation with increased CONSORT and STRICTA indices (both p<0.000; correlation coefficients: 0. and 0., respectively). The journal type displayed significantly negative correlation with the CONSORT index (correlation coefficient:. point), indicating that RCTs in TKM-related journals displayed CONSORT indices. points lower than RCTs not published in TKMrelated journals. Types of intervention and sample size did not significantly correlate with the CONSORT index scores. In the STRICTA analysis, the sample size demonstrated a statistically significant but minimal positive correlation with the STRICTA index (correlation coefficient: 0.0), indicating that RCTs with 0 more samples display a 0. point increase in STRICTA index score. Type of journals did not significantly correlate with the STRICTA index scores. The regression model showed a good fit. BMJ Open: first published as 0./bmjopen--000 on July. Downloaded from on December by guest. Protected by copyright.

15 BMJ Open Page of Discussion We identified a considerable number of items that were incompletely reported, which may limit the internal validity and applicability of the trial results. Reporting of items that are core components of assessing the risk of bias, such as randomization, concealment of allocation, and outcome assessor blinding, were particularly incomplete. The number of participants at each stage of the trial was not explicitly reported, which suggests high risk of attrition bias of the results of the RCTs. None of the Korean RCTs reported the critical criteria of subgroup analyses including the use of subgroup variables measured at baseline, prespecification of subgroup hypotheses, and statistical significance of interaction tests; inappropriately claimed subgroup effect is not credible and can lead to the misuse of information by researchers, clinicians and policymakers. Harms of study interventions were reported in less than onequarter of the included RCTs, which does not permit the investigation of the safety of acupuncture in the context of trials. Discussions regarding the generalisability of trial findings was lacking in most Korean RCTs, which may interfere with the application of trial results to real clinical situations. Overall, the reporting quality of Korean RCTs was suboptimal as assessed by the CONSORT statement, which could serve as a significant obstacle to the establishment of a sound evidence base. Regarding the reporting quality of intervention details based on the revised STRICTA statement, the theoretical background and some details of the needling processes showed moderate to good quality reporting, whereas items related to contextual factors were markedly under-reported. The inconsistent reporting quality between items may imply that certain items are perceived as more important by investigators or journal editors, although there is no evidence to justify such inference in the Korean context. Practitioner qualifications BMJ Open: first published as 0./bmjopen--000 on July. Downloaded from on December by guest. Protected by copyright.

16 Page of BMJ Open were also incompletely reported, which may increase the uncertainty with regard to treatment quality and safe implementation of interventions. Observed incompleteness of the reporting of practitioner qualifications may have been influenced by the exclusive government registry system for Korean medicine doctors (KMDs) and the possible assumption that trial interventions were conducted by qualified KMDs. Collectively, the reporting quality of intervention details was inconsistent, which may be problematic for the replication of acupuncture treatments in other contexts. Intervention item # (details of intervention) was completely reported in more than two-thirds of the included RCTs when assessed by the criteria in this study. However, practitioner information was not included in the criteria based on the following reasons. First, we found that practitioner information was lacking in most RCTs from our initial search. We considered that including practitioner information as a mandatory component for adequate intervention reporting would result in the exclusion of most RCTs regardless of the completeness of additional components, ultimately causing the assessment to be less meaningful. Second, only KMDs can practice acupuncture in South Korea according to the unique dual healthcare qualification system in South Korea (Western medicine doctors and KMDs). This contextual background may have caused the authors of Korean acupuncture RCTs to assume that the acupuncture practitioner was a KMD and omit the practitioner information. However, this hypothesis may not be supported in other contexts. We conducted a sensitivity analysis by including the practitioner information in the criteria for assessing item #, and found that the complete reporting of the respective item in one (%) and (%) Korean RCTs published before and after 0, respectively. This result is substantially lower than results based on our primary criteria. Because practitioner information can be crucial for the BMJ Open: first published as 0./bmjopen--000 on July. Downloaded from on December by guest. Protected by copyright.

17 BMJ Open Page of replication of trial interventions in research and clinical contexts, this information should be clearly reported in future Korean acupuncture RCTs. In our study, the STRICTA items were generally more completely reported than CONSORT items. This may be because the subject of included studies was acupuncture. Another possible explanation refers to the fact that the Korean translations of the original and revised STRICTA statements were published in a Korean acupuncture-related journal (i.e., Journal of Acupuncture and Moxibustion Society) ; however, an official Korean CONSORT was never published to our knowledge. Although no direct evidence supports the assumption that translated CONSORT statements in published in languages other than English (LOE) are associated with more transparent reporting in trials reported by LOE, attempts have been made to translate and endorse CONSORT statements in domestic journals to assist local authors in comprehensive reporting of the recommended trial components according to international standards (i.e., CONSORT). Consistent with these international efforts, the official Korean translation of the CONSORT statements is expected to be a useful resource for Korean authors and journal editors. Our findings are consistent with previous studies investigating the adherence of RCTs to - CONSORT and STRICTA statements in various clinical fields, including acupuncture. The quality of reporting was relatively incomplete in this study compared with others, especially for items related to randomisation, allocation concealment, outcome assessor blinding and the method of analysis (per protocol or intent-to-treatment analysis). Several factors may be potentially related to the poor reporting quality identified in this study. First, a lack of understanding of the need for comprehensive trial reporting to enable the assessment of internal and external validity in the design and implementation of trials may result in BMJ Open: first published as 0./bmjopen--000 on July. Downloaded from on December by guest. Protected by copyright.

18 Page of BMJ Open incomplete reporting. This factor was the primary motivation for international initiatives to develop the CONSORT statement. However, the uptake of the CONSORT statement at the author level was not investigated in this study due to resource limitations, and this issue should be addressed by future research. Second, the lack of utilisation of the CONSORT and STRICTA checklists by journal editors and peer reviewers may impede the improvement of the quality of trial reporting through the publication process. The utilisation of reporting guidelines by biomedical journals during the peer review process improves the quality of publication in terms of general methodological issues. Hwang et al. found that none of the conventional Korean medical journals endorsed the CONSORT statement as part of the author instructions, which may be responsible for the incomplete reporting quality of Korean RCTs of conventional medicine. Little is known as to whether Korean acupuncture or traditional medicine journals have used reporting guidelines (e.g., CONSORT and STRICTA) for editorial or peer-review processes. To date, only one acupuncture-related journal (i.e., Journal of Acupuncture and Moxibustion Society) endorses the CONSORT and STRICTA guidelines in the author instructions (unpublished communication). To improve the quality of reporting for acupuncture trials in Korean, collective efforts of journal editors, authors and peer-reviewers to utilise the CONSORT and the STRICTA statements during each stage of manuscript preparation and submission as well as the peer-review process are urgently needed. Barriers related to the incompleteness of reporting by Korean authors of RCTs should also be investigated. One strength of this study is that it is the first systematic investigation to assess the reporting quality of Korean RCTs of acupuncture based on the CONSORT statements. Given that the quality of reporting in RCTs of traditional Chinese medicine (TCM), which employ interventions similar to acupuncture, has already been reported, this study complements BMJ Open: first published as 0./bmjopen--000 on July. Downloaded from on December by guest. Protected by copyright.

19 BMJ Open Page of current data regarding the reporting quality of acupuncture RCTs in Korean databases. We also attempted to evaluate the best available dataset of Korean acupuncture RCTs by employing extensive search strategies that targeted Korean databases; however, the possibility of undetected studies could not be completely excluded. Two recent studies analysed study characteristics and bibliographic information using updated search results and database information. The results of this review should also be updated periodically to monitor whether recommended components of trials by the CONSORT and STRICTA statements are transparently and completely reported in Korean RCTs of acupuncture. Several weaknesses of our study should be mentioned. First, the cut-off year of the CONSORT and STRICTA analysis was arbitrarily defined because no standard criteria exist regarding sufficient period for dissemination and implementation of the CONSORT and STRICTA recommendations. Although we assumed that local authors would require at least four years to reflect the CONSORT and STRICTA recommendations into design and writing of trial, no concrete evidence justifies our assumption. Second, we considered the partial reporting of CONSORT and STRICTA items as complete despite the fact that the recent Cochrane review regarded only full descriptions of the required content for a given item as complete. Consequently, our assessment may have reported inflated scores, and this possibility should be recognised when interpreting our results. Third, time may serve as a potential confounder of quality of reporting because the completeness of reporting may have naturally changed over time regardless of CONSORT and STRICTA endorsements. The publication year displayed significant positive correlation with CONSORT and STRICTA index scores in the regression analyses; however, the correlation magnitude was modest. Thus, the potential influence of the natural improvement of completeness of reporting on the observed results remains unclear and should be acknowledged as limitation of this study. BMJ Open: first published as 0./bmjopen--000 on July. Downloaded from on December by guest. Protected by copyright.

20 Page of BMJ Open Implications for future research Endorsement of the CONSORT and STRICTA checklist should be encouraged among editors of relevant Korean medical journals, and potential barriers need to be tackled. Collective efforts to increase compliance with the CONSORT and STRICTA statements during design, implementation and reporting of clinical trials are needed to improve the quality of reporting in Korean RCTs of acupuncture. The need to appropriately address core items relevant to assessments of internal and external trial validity should also be impressed upon trial authors, journal editors and peer reviewers. In particular, items related to randomisation, concealment of allocation, blinding of outcome assessors, flow of participants, subgroup analyses and harms of acupuncture should be more completely reported in future Korean RCTs of acupuncture. Practitioner- and context-related STRICTA items should also be better reported. The authors, editors and peer-reviewers of Korean RCTs of acupuncture should be properly educated to aid in the complete reporting of these items. Conclusions The quality of reporting in Korean RCTs of acupuncture remained suboptimal over time. The CONSORT and STRICTA statements should be thoroughly utilised by trial authors and journal editors to improve the quality of reporting. BMJ Open: first published as 0./bmjopen--000 on July. Downloaded from on December by guest. Protected by copyright.

21 BMJ Open Page of Acknowledgements The study was supported by the Korea Institute of Oriental Medicine. The views expressed are not necessarily those of the funding body. Authors contributions KHK conceived the research. KHK and JWK performed data collection, analysis and wrote the first draft of the paper. KHK, JWK, MSL and JDL were all involved in the development and refinement of subsequent drafts. KHK is the guarantor for the study. All authors read and approved the final manuscript. Competing Interests None Data Sharing Statement Authors adhere to the data sharing statement of BMJ Open. Please send an to the first author (KHK) to obtain additional unpublished data. BMJ Open: first published as 0./bmjopen--000 on July. Downloaded from on December by guest. Protected by copyright.

22 Page of BMJ Open References. MacPherson H, Altman DG, Hammerschlag R, et al. Revised STandards for Reporting Interventions in Clinical Trials of Acupuncture (STRICTA): extending the CONSORT statement. PLoS Med 0;():e Moher D, Jones A, Lepage L, CONSORT Group (Consolitdated Standards for Reporting of Trials). Use of the CONSORT statement and quality of reports of randomized trials: a comparative before-and-after evaluation. JAMA 0;:-.. Turner L, Shamseer L, Altman DG, et al. Consolidated standards of reporting trials (CONSORT) and the completeness of reporting of randomised controlled trials (RCTs) published in medical journals. Cochrane Database Syst Rev ;:MR Uetani K, Nakayama T, Ikai H, et al. Quality of reports on randomized controlled trials conducted in Japan: evaluation of adherence to the CONSORT statement. Intern Med 0;():-.. Wang G, Mao B, Xiong ZY, et al. The quality of reporting of randomized controlled trials of traditional Chinese medicine: a survey of randomly selected journals from mainland China. Clin Ther 0;():-.. Hwang YW, Lee KW, Hwang IH, et al. The quality of reporting of randomized controlled trials in Korean Medical Journals Indexed in KoreaMed: Survey of Items of the revised CONSORT statement. J Korean Acad Fam Med 0;:-.. Kong JC, Lee MS, Shin BC. Randomized clinical trials on acupuncture in Korean literature: a systematic review. Evid Based Complement Alternat Med 0;():-.. Prady SL, Richmond SJ, Morton VM, et al. A systematic evaluation of the impact of STRICTA and CONSORT recommendations on quality of reporting for acupuncture trials. PLoS One 0;():e. BMJ Open: first published as 0./bmjopen--000 on July. Downloaded from on December by guest. Protected by copyright.

23 BMJ Open Page of Kim KH, Kang JW, Lee MS, et al. Assessment of the quality of reporting for treatment components in Cochrane reviews of acupuncture. BMJ Open ;:e00.. Kim KH, Kong JC, Choi JY, et al. Impact of including korean randomized controlled trials in cochrane reviews of acupuncture. PLoS One ;(0):e.. Moher D, Hopewell S, Schulz KF, et al. CONSORT 0 explanation and elaboration: updated guidelines for reporting parallel group randomised trials. BMJ 0;:c.. Hoffmann TC, Erueti C, Glasziou PP. Poor description of nonpharmacological interventions: analysis of consecutive sample of randomised trials. BMJ ;:f.. Paterson C, Dieppe P. Characteristic and incidental (placebo) effects in complex interventions such as acupuncture. BMJ 0;(0):-.. Sun X, Briel M, Busse JW, et al. Credibility of claims of subgroup effects in randomised controlled trials: systematic review. BMJ ;:e.. Sun X, Briel M, Busse JW, et al. The influence of study characteristics on reporting of subgroup analyses in randomised controlled trials: systematic review. BMJ ;:d.. Na SS. East Asian medicine in South Korea. Harvard Asia Quarterly ;():-.. Prady SL, Macpherson H. Assessing the utility of the standards for reporting trials of acupuncture (STRICTA): a survey of authors. J Altern Complement Med 0;():-.. Han C, Kwak KP, Marks DM, et al. The impact of the CONSORT statement on reporting of randomized clinical trials in psychiatry. Contemp Clin Trials 0;():-.. Lu X, Hongcai S, Jiaying W, et al. Assessing the quality of reports about randomized controlled trials of acupuncture treatment on mild cognitive impairment. PLoS One ;():e.. Cobo E, Cortés J, Ribera JM, et al. Effect of using reporting guidelines during peer review on quality of final manuscripts submitted to a biomedical journal: masked randomised trial. BMJ ;:d. BMJ Open: first published as 0./bmjopen--000 on July. Downloaded from on December by guest. Protected by copyright.

24 Page of BMJ Open Lee H-S, Cha S-J, Park H-J, et al. Revised STandards for Reporting Interventions in Clinical Trials of Acupuncture (STRICTA): Extending the CONSORT statement. Korean Journal of Acupuncture 0;():-.[In Korean]. Lee H-S, Park J-B, Seo J-C, et al. Standards for Reporting Interventions in Controlled Trials of Acupuncture: The STRICTA recommendations. Journal of Korean Acupuncture and Moxibustion Society 0;():-.[In Korean]. Costa LO, Maher CG, Moseley AM, et al. Endorsement of trial registration and the CONSORT statement by the Revista Brasileira de Fisioterapia. Rev Bras Fisioter 0;():-.. MacPherson H, Altman DG. Improving the quality of reporting acupuncture interventions: describing the collaboration between STRICTA, CONSORT and the Chinese Cochrane Centre. J Evid Based Med 0;():-0.. Choi J, Lee JA, Yun K-J, et al. Online databases and journals of Traditional Medicine and Complementary and Alternative Medicine in Korea. Eur J Integr Med Published Online First: November. doi: 0.0/j.eujim Kim S, Sagong HS, Kong JC, et al. Randomised clinical trials on acupuncture in the Korean literature: bibliometric analysis and methodological quality. Acupunct Med Published Online First: January. doi: 0./acupmed--000 BMJ Open: first published as 0./bmjopen--000 on July. Downloaded from on December by guest. Protected by copyright.

25 BMJ Open Page of Table. Search terms used in titles and abstracts English search term # acupuncture # clinical # controlled OR random # # AND # AND # # # Korean search term # OR # # AND # AND # BMJ Open: first published as 0./bmjopen--000 on July. Downloaded from on December by guest. Protected by copyright.

26 Page of BMJ Open Table. Assessment criteria of the item # of the CONSORT 0 statement Criteria item Components of respective STRICTA items ) Procedure At least one item should be reported for the yes assessment De-qi response sought (item d) Needle stimulation methods (item e) Names of points (item b) ) Materials Needle specification (item g) ) Intensity Needle retention time (item f) ) Schedules At least one item should be reported for the yes assessment Duration (item b) Frequency (item b) Total or average number of sessions (a) BMJ Open: first published as 0./bmjopen--000 on July. Downloaded from on December by guest. Protected by copyright.

27 BMJ Open Page of Table. General characteristics of included RCTs Before 0 (~0) (n=) After 0 (0-) (n=0) Total number of included studies CONSORT analysis 0 STRICTA analysis Journals Peer-review journals (00.0%) (.%) Unpublished (Master dissertation or PhD Thesis) a 0 (0.0%) (.%) Sample size b. (.). (.) Publication year c 0 (-0) 0 (0-) Type of intervention Needle acupuncture (.%) (.%) Non-needling acupuncture d (.%) (.%) Type of control Active treatment (.%) 0 (.0%) Sham or placebo (.%) (.%) Waitlist (.0%) (.%) Number of arms arms (0.%) (.%) or arms (.%) (0.%) a Unpublished articles were obtained from the NANET database ( b Values are presented as mean (standard deviation). c Values are presented as median (range). d Non-needling acupuncture includes pharmacopuncture, bee venom acupuncture and other acupuncture point stimulation using non-penetrating techniques. BMJ Open: first published as 0./bmjopen--000 on July. Downloaded from on December by guest. Protected by copyright.

28 Page of BMJ Open 0 Table. CONSORT checklist for Korean RCTs of acupuncture Item no Summarized descriptor Before 0 (~0) After 0 (0~) Mean difference [% CI] N= % [% CI] N=0 % [% CI] Title / Abstract How participants were allocated to interventions 0. [.,.] 0.[.,.] -. [-.,.] Introduction / background Scientific background / explanation of rationale 0. [.0,.].[.0,.]. [-.,.] Methods Participants Eligibility criteria for participants 00.0 [00.0, 00.0]. [.0,.] -. [-.0, -0.] Interventions Precise details of the interventions. [.0,.]. [.,.].0 [-.,.] Objectives Specific objectives and hypotheses. [.,.]. [.,.] -. [-.,.] Outcomes Clearly defined primary / secondary outcome measures. [.,.]. [.,.]. [-0.,.] Sample size How sample size was determined. [-.,.]. [-0.,.] 0. [-.0,.] Randomization Sequence generation Method used to generate the random allocation sequence. [.0,.].[.,.0]. [.,.] Allocation Method used to implement the random allocation concealment sequence [0.0, 0.0]. [., 0.]. [., 0.] 0 Implementation Who generated the allocation sequence / enrolled participants / assigned participants to their groups [0.0, 0.0]. [.,.]. [.,.] Blinding Outcome assessor blinding. [.,.]. [.,.]. [-.,.] Statistical Methods Methods used to compare groups for primary outcomes. [., 0.]. [.,.] -. [-.,.] Results Participant flow Flow of participants through each stage. [-., 0.]. [.,.]. [0.,.] Recruitment Dates defining the periods of recruitment/follow-up. [.0,.] 0. [.0,.]. [.,.] Baseline data Baseline demographic/clinical characteristics of each group.0 [., 00.]. [.,.]. [-., 0.0] Numbers Analysed Number of participants (denominator) in each group included in each analysis. [.,.]. [.,.] -. [-.,.] Outcomes / estimation For each primary and secondary outcome, a summary of results for each group 00.0 [00.0, 00.0] 0.0 [., 00.] -.0 [-., 0.] Ancillary analyses Address multiplicity by reporting any other analyses performed 0/ 0.0 [0.0, 0.0] 0/ 0.0 [0.0, 0.0] 0.0 [0.0, 0.0] Adverse events All important adverse events or side effects in each intervention group. [-0.,.]. [.0,.]. [.0,.] Discussion Interpretation Interpretation of the results. [.0,.]. [.,.0].0 [-0.,.0] Generalizability Generalizability (external validity) of the trial findings. [.,.0]. [-0.,.] -. [-.,.] Overall evidence General interpretation of the results in the context of current evidence. [.,.].0 [.0,.] -. [-.,.0] a Values are presented as number of reported RCTs divided by the total number of eligible RCTs assessed for each item and percentage. on December by guest. Protected by copyright. BMJ Open: first published as 0./bmjopen--000 on July. Downloaded from

29 BMJ Open Page of Table. The number of Korean RCTs reporting each STRICTA Item Before STRICTA (~ 0) After STRICTA (0~) Item no N= % [% CI] N= % [% CI] Mean difference [% CI]. Acupuncture Rationale a) Style of acupuncture. [., 00.]. [., 0.]. [-.,.0] b) Reasoning of treatments. [.,.]. [.,.]. [.,.0] c) Acupuncture regimen variation. [.,.] 0. [., 0.]. [-0.,.]. Needling details a) Number of needles.0 [.,.] 0. [.,.]. [.,.] b) Names of points. [., 00.]. [., 0.].0 [-.,.0] c) Depth of insertion.0 [.,.]. [.,.0]. [.,.] d) Response to needle. [.,.] 0. [.,.]. [.,.] e) Needle stimulation methods. [.,.]. [.,.0]. [.,.] f) Retention time. [., 00.]. [.,.]. [-0.,.] g) Type of needles. [.0,.]. [.,.]. [-.,.]. Treatment regimen a) Number of sessions. [.,.]. [.,.0]. [-.,.] b) Frequency / duration. [.,.]. [.,.]. [-.,.]. Treatment context a) Details of other treatments. [.,.]. [.,.0]. [-.,.] b) Setting and context. [.,.]. [.,.] 0. [-.,.]. Practitioner background ) Description of acupuncturists [0.0, 0.0]. [.,.]. [.,.] a Values are presented as number of reported RCTs divided by the total number of eligible RCTs assessed for each item and percentage. BMJ Open: first published as 0./bmjopen--000 on July. Downloaded from on December by guest. Protected by copyright.

30 Page of BMJ Open Table. Comparison of the CONSORT/STRICTA index between pre- and post- CONSORT/STRICTA trials N Mean % CI P-value Pre-CONSORT (-0).. to 0. Post-CONSORT (0-) to. Difference. 0. to Pre-STRICTA (-0).. to. Post-STRICTA (0-). 0. to. Difference.. to. <0.000 BMJ Open: first published as 0./bmjopen--000 on July. Downloaded from on December by guest. Protected by copyright.

31 BMJ Open Page of Figure : Flow chart of the trial selection process RCT; randomized controlled trial. BMJ Open: first published as 0./bmjopen--000 on July. Downloaded from on December by guest. Protected by copyright.

32 Page of BMJ Open Potentially relevant articles identified and retrieved for more detailed evaluation (n=) Reasons for exclusion (n=) uncontrolled clinical trial (n=) case-control study (n=) duplicated article (n=) protocol (n=) review (n=) animal study (n=) non- randomized controlled trial (n=) RCTs but excluded because of not being related to acupuncture (n= ) Full-texts screened for primary analysis (n=) Korean RCTs contributed to the final primary analysis (n=) by analyses: - CONSORT checklist (n=) - STRICTA checklist (n=0) by year: - before 0 (n=) - from 0 (n=0) Exclusions (n=) No acupuncture study (n=) Healthy participants (n=) Non-parallel RCT (n=) Duplication (n=) Not in a domestic journal (n=) BMJ Open: first published as 0./bmjopen--000 on July. Downloaded from on December by guest. Protected by copyright.

33 BMJ Open Assessment of the quality of reporting in randomised controlled trials of acupuncture in the Korean literature using the CONSORT and STRICTA statements Journal: BMJ Open Manuscript ID: bmjopen--000.r Article Type: Research Date Submitted by the Author: 0-Jun- Complete List of Authors: Kim, Kun Hyung; School of Korean Medicine, Pusan National University, Department of Acupuncture and Moxibustion Kang, Jung Won; College of Korean Medicine, Kyung Hee University, Department of Acupuncture and Moxibustion Lee, Myeong Soo; Korea Institute of Oriental Medicine, Lee, Jae Dong; College of Korean Medicine, Kyung Hee University, Department of Acupuncture and Moxibustion <b>primary Subject Heading</b>: Secondary Subject Heading: Keywords: Complementary medicine Complementary medicine, Medical publishing and peer review, Research methods COMPLEMENTARY MEDICINE, EDUCATION & TRAINING (see Medical Education & Training), EPIDEMIOLOGY BMJ Open: first published as 0./bmjopen--000 on July. Downloaded from on December by guest. Protected by copyright.

34 Page of BMJ Open Assessment of the quality of reporting in randomized controlled trials of acupuncture in the Korean literature using the CONSORT statement and the STRICTA guidelines Kun Hyung Kim, Jung Won Kang, Myeong Soo Lee, Jae-Dong Lee School of Korean Medicne, Pusan National University, Yangsan, South Korea Department of Acupuncture & Moxibustion, College of Korean Medicine, Kyung Hee University, Seoul, South Korea Korea Institute of Oriental Medicine Running Title: CONSORT and STRICTA for Korean acupuncture trials Source of support: None Financial Disclosure: None Keywords: acupuncture, randomized controlled trial, CONSORT, STRICTA, reporting guideline Corresponding author : Jae Dong Lee, KMD., PhD. Department of Acupuncture & Moxibustion, College of Korean Medicine, Kyung Hee University, Seoul, 0-, South Korea Tel : -(0) -- Fax: -(0) -- ljdacu@gmail.com BMJ Open: first published as 0./bmjopen--000 on July. Downloaded from on December by guest. Protected by copyright.

35 BMJ Open Page of Abstract Objectives: This study aimed to assess the completeness of reporting of randomised controlled trials (RCTs) of acupuncture in the Korean literature. Design: Systematic review. Methods: We searched Korean databases and Korean journals to identify eligible RCTs of acupuncture published from to July. We used the Consolidated Standards of Reporting Trials (CONSORT) checklist for parallel RCTs and the revised STandards for Reporting Interventions in Clinical Trials of Acupuncture (STRICTA) to assess the quality of reporting in Korean RCTs. We compared the completeness of reporting of CONSORT and STRICTA items in RCTs published in two time periods (-0 referred to as the early period and 0- referred to as the late period). Results: We analysed eligible RCTs using the CONSORT statement concerning RCTs of both needling and non-needling acupuncture and the STRICTA guidelines for 0 trials of needling acupuncture. Among the 0 RCTs in the late period, the proportion of RCTs that completely reported the CONSORT items of outcome definition (.%), sample size calculation (.%), randomisation (%), allocation concealment (.%), implementation of allocation (.%), outcome assessor blinding (.%), flow of participants (.%), number of participants analysed (.%), ancillary analyses (0.0%), adverse events (.%), generalisability of findings (.%) and overall evidence (.0%) remained small. Among the RCTs of needling acupuncture in the late period, the STRICTA items of setting/context (.%) and practitioner background (.%) showed incomplete reporting. The completeness of reporting improved over time in several CONSORT and STRICTA items. Conclusion: The completeness of reporting of Korean RCTs of acupuncture was suboptimal according to the CONSORT and revised STRICTA statements. Trial authors and journal editors should use the CONSORT statement and the STRICTA guidelines for transparent BMJ Open: first published as 0./bmjopen--000 on July. Downloaded from on December by guest. Protected by copyright.