Prospective Registration, Bias Risk and Outcome-Reporting Bias in Randomised Clinical Trials of Traditional Chinese Medicine For peer review only

Transcription

1 Prospective Registration, Bias Risk and Outcome-Reporting Bias in Randomised Clinical Trials of Traditional Chinese Medicine Journal: Manuscript ID: bmjopen-0-00 Article Type: Research Date Submitted by the Author: 0-Apr-0 Complete List of Authors: Liu, Jian-Ping; Beijing University of Chinese Medicine, Centre for Evidence- Based Chinese Medicine Han, Mei; Beijing University of Chinese Medicine, Centre for Evidence- Based Chinese Medicine Li, Xin-Xue; Beijing University of Chinese Medicine, Centre for Evidence- Based Chinese Medicine Mu, Yu_Jie; Beijing University of Chinese Medicine, Centre for Evidence- Based Chinese Medicine Lewith, George; University of Southampton, Centre for Complementary Medicine Research Wang, Yu-Yi; Beijing University of Chinese Medicine, Centre for Evidence- Based Chinese Medicine Witt, Claudia; Charité Universitätsmedizin Berlin, Institute for Social Medicine, Epidemiology and Health Economics Yang, Guo-Yan; Beijing University of Chinese Medicine, Centre for Evidence-Based Chinese Medicine Manheimer, Eric; University of Maryland School of Medicine, Center for Integrative Medicine Snellingen, Torkel; Beijing University of Chinese Medicine, Centre for Evidence-Based Chinese Medicine Berman, Brian; University of Maryland School of Medicine, Center for Integrative Medicine Gluud, Christian; Copenhagen University, Copenhagen Trial Unit <b>primary Subject Heading</b>: Complementary medicine Secondary Subject Heading: Complementary medicine, Epidemiology, Research methods Keywords: COMPLEMENTARY MEDICINE, EPIDEMIOLOGY, STATISTICS & RESEARCH METHODS, Clinical trials < THERAPEUTICS, Herbal medicine < THERAPEUTICS : first published as./bmjopen-0-00 on July 0. Downloaded from on November 0 by guest. Protected by copyright.

2 Page of Prospective Registration, Bias Risk and Outcome-Reporting Bias in Randomised Clinical Trials of Traditional Chinese Medicine Jian-Ping Liu, Mei Han, Xin-Xue Li, Yu-Jie Mu, George Lewith, Yu-Yi Wang, Claudia M. Witt, Guo-Yan Yang, Eric Manheimer, Torkel Snellingen, Brian Berman, Christian Gluud Jian-Ping Liu, MD, PhD, Centre for Evidence-Based Chinese Medicine, Beijing University of Chinese Medicine, Beijing, China ( Liujp@bucm.edu.cn) Mei Han, PhD candidate, Centre for Evidence-Based Chinese Medicine, Beijing University of Chinese Medicine, Beijing, China ( hanmeizoujin@.com) Xin-Xue Li, master student, Centre for Evidence-Based Chinese Medicine, Beijing University of Chinese Medicine, Beijing, China ( lixinxue@.com) Yu-Jie Mu, master student, Centre for Evidence-Based Chinese Medicine, Beijing University of Chinese Medicine, Beijing, China ( muyujie0@.com) George Lewith, MD, Centre for Complementary Medicine Research, University of Southampton, Southampton, UK ( glewith@scmrt.org.uk) Yu-Yi Wang, master student, Centre for Evidence-Based Chinese Medicine, Beijing University of Chinese Medicine, Beijing, China ( wangyuyi@.com) Claudia M. Witt, MD, MBA, Institute for Social Medicine, Epidemiology and Health Economics, Charité Universitätsmedizin Berlin, Germany ( claudia.witt@charite.de) Guo-Yan Yang, master student, Centre for Evidence-Based Chinese Medicine, Beijing University of Chinese Medicine, Beijing, China ( yangguoyanbeijing@.com) Eric Manheimer, MS, Center for Integrative Medicine, University of Maryland School of Medicine, Baltimore, Maryland, 0 USA ( emanheimer@compmed.umm.edu) Torkel Snellingen, MD, Centre for Evidence-Based Chinese Medicine, Beijing University of Chinese Medicine, Beijing, China ( tsnellingen@gmail.com) Brian Berman, MD, Center for Integrative Medicine, University of Maryland School of Medicine, Baltimore, Maryland, 0 USA ( BBerman@compmed.umm.edu) Christian Gluud, MD, Dr Med Sci, Copenhagen Trial Unit, Centre for Clinical Intervention Research, Rigshopitalet, Copenhagen University Hospital, Copenhagen, Denmark ( cgluud@ctu.rh.dk) Contact details for correspondence author: Prof Jian-Ping Liu Centre for Evidence-Based Chinese Medicine, Beijing University of Chinese Medicine, Bei San Huan Dong Lu, Chaoyang District, Beijing 00, China Phone: + 0 Fax: + Liujp@bucm.edu.cn Word count (excluding title page, abstract, references, figures and tables): : first published as./bmjopen-0-00 on July 0. Downloaded from on November 0 by guest. Protected by copyright.

3 Page of ABSTRACT Background Clinical trials on traditional Chinese medicine (TCM) should be registered in a publicly accessible international trial register and report on all outcomes. We systematically assessed and evaluated TCM trial registration in registries with their subsequent publications. Objective To describe the characteristics of TCM trials, estimate bias risk and outcome reporting bias in clinical trials. Data Sources and Study Selection Fifteen trial registries were searched from their inception to July 0 to identify randomised trials on TCM including Chinese herbs, acupuncture and/or moxibustion, cupping, tuina, qigong, and dietary advice. Data Extraction We extracted data from the registries and searched for subsequent publications in PubMed and Chinese databases. We compared information in the registries of completed trials with any publications focusing on study design, bias risk including reporting bias and deviations from the register protocol. Results registered randomised trials were identified evaluating TCM, of which 0 were completed studies (.%). The most frequent conditions were pain (.%), musculoskeletal (.%), nervous (.%), digestive (.%), circulatory (.%), respiratory (.%), mental and behavioral disorders (.%), and cancer (.0%). The trial register data identified parallel, phase II/III randomised trials with sample size estimations and blinding, but limited information about randomization (generation of allocation sequence and allocation concealment). Comparing trial registration data of completed trials (.%) with their subsequent publications, inconsistencies were identified with respect to one or more of the following: sample size (.0%), outcome assessor blinding (.%), primary outcomes (%), and safety (%) reporting. : first published as./bmjopen-0-00 on July 0. Downloaded from on November 0 by guest. Protected by copyright.

4 Page of Conclusions Increasing numbers of clinical trials investigating a variety of TCM interventions have been registered in international trial registries. The study design of TCM trials has improved in estimated sample size, use of blinding and placebo/sham control. However, the quality of reporting of these trials is inconsistent with the registered data and involves selective outcome reporting bias. Key words: clinical trials, traditional Chinese medicine, international trial registry, publication bias Word count: 00 : first published as./bmjopen-0-00 on July 0. Downloaded from on November 0 by guest. Protected by copyright.

5 Page of Article Summary Article focus We wished to evaluate the methodological quality of clinical trials in traditional Chinese medicine; We investigated whether the systematic identification of prior trial registration was associated with an improvement in the methodological quality of the subsequent published studies Key messages: A substantial number of clinical trials in traditional Chinese medicine, covering a broad range of therapies, are now being registered in international trial registries; Registration is associated with more rigorous study methodology and study design (e.g. randomisation protocols, secure blinding and sample size estimates); Outcome reporting bias exists when comparing the registry information and the subsequent publications, and some trials were registered after their publication. Strengths and limitations of this study Systematic searches of all available international trial registries for any clinical trials of traditional Chinese medicine; All interventions involving any traditional Chinese medicine was included as was the diagnosis; The registered information for clinical trials is not uniform across the registries and important methodological information may be missing; Subsequent publications were obtained for those studies recorded as completed in the registry. This may not represent the true situation for trials if the registry data is not updated by the researchers. : first published as./bmjopen-0-00 on July 0. Downloaded from on November 0 by guest. Protected by copyright.

6 Page of Many empirical studies have shown that the methodological quality of randomised clinical trials of traditional Chinese medicine (TCM) is poor with respect to risk of systematic errors (bias) (generation of allocation sequence, allocation concealment, blinding, descriptions of drop-out or losses to follow up, selective outcome reporting) and risk of random errors (play of chance). - Moreover, publications on TCM trials are uniformly positive, raising concerns that trials initiated to investigate TCM are only published if they have positive results. Poor quality trials and risk of publication bias will reduce the strength of evidence when developing clinical practice guidelines or preparing systematic reviews. One of the ways to improve trial quality is to prospectively register clinical trial protocols in international trial registries such as clinicaltrials.gov., The World Health Organization (WHO) established an international clinical trial registry platform (ICTRP) ( in 00, which now links clinical trial registries. Furthermore, several peer reviewed journals such as The Lancet and Trials publish trial protocols to promote transparency and improve trial quality. In order to describe the characteristics of TCM trials, and estimate reporting bias in clinical trials, we systematically searched major international trial registries to identify information about TCM trials, and compared the registered records with subsequent publications regarding outcomes and other data. METHODS Inclusion criteria We included randomised clinical trials for any TCM intervention singly or their combination: Chinese herbs, acupuncture, acupressure, moxibustion, cupping, dietary advice, tuina : first published as./bmjopen-0-00 on July 0. Downloaded from on November 0 by guest. Protected by copyright.

7 Page of (therapeutic massage), taichi, qigong and guasha (scaping massage). We excluded non-randomised studies such as quasi-randomised studies, cohort studies, phase trials, retrospective clinical studies, case series, or case studies. There were no limitations on study type (superiority, noninferiority or equivalence), or study phase. Data source We systematically searched major international trial registries ( linked to WHO ICTRP) from their inception to July 0 (Supplement 0). For trials listed as completed in the registered records, we then searched for published protocols, as well as the full texts of subsequently published articles in PubMed and three Chinese electronic bibliographic databases including Chinese Biomedical Database ( China National Knowledge Infrastructure ( and Chinese VIP Information ( Data extraction Two researchers (from MH, XL, YM, YW or GY) extracted data independently from each trial registry using a standardized, piloted data extraction form. The form was developed by our research group and based on general characteristics of clinical trials, methodology, and the 0 minimum items required for WHO trial registration. The main information collected included the number of trials in each registry, year registered, trial design, methods, sample size, setting, participants and diseases/conditions, interventions, controls, primary and secondary outcomes, inclusions and exclusion criteria, current status (e.g., completed, ongoing), ethical approval, : first published as./bmjopen-0-00 on July 0. Downloaded from on November 0 by guest. Protected by copyright.

8 Page of sponsors, institutions, country of origin, contact details, and funding. The extracted data were cross checked by the authors, and any discrepancy resolved by discussion with a third author (JPL). Conditions were classified according to the WHO international classification of diseases ( (last accessed July 0). We searched for publications in bibliographic databases for those trials listed as completed in the registries, and compared the published trials with the registered records. Data analyses We used The Cochrane Collaboration risk of bias tool to evaluate the registered records. - This tool assesses the following domains: generation of allocation sequence; allocation concealment; blinding; incomplete outcome reporting; and selective outcome reporting (defined as change of primary outcome or new outcome added). We also evaluated the estimated sample size, explicit inclusion and exclusion criteria, and risk of funding bias. Two authors (MH, XL) compared the trial design and methodology from the registered records with the resulting publications to analyze consistency and selective outcome reporting. Selective outcome reporting was defined as when the full paper publications reported different primary outcomes or changed it from the original pre-defined primary outcome in the registered data. Inconsistency was defined as when the items were not the same as described in the registry in the subsequent paper publications. For the sample size estimation a discrepancy of over 0% between the registered and published information was judged as inconsistent. We judged trials as positive results based on () authors conclusion showing the intervention was superior to the control; or () the comparison of between groups showed statistically significant difference (p : first published as./bmjopen-0-00 on July 0. Downloaded from on November 0 by guest. Protected by copyright.

9 Page of <0.0) for primary outcome measures. We compared the dates of registration and full paper publication to assess the proportion of retrospective registrations. RESULTS We identified registered records of randomised clinical trials on TCM. 0 of the (.%) the registered records indicated that the trial had been completed. out of (0.%) were from mainland China (excluding Hong Kong and Macau). The first trial was registered in, in clinicaltrials.gov, and the numbers of registered trials has increased over the last decade (Figure ). Table shows the frequency of the type of TCM intervention included in each registry. The combined therapies studied included trials on integrating Chinese and Western medicine, and trials combining two or more TCM therapies. Using the international disease classification (ICD-), we identified the most frequent conditions: pain (.%) and musculoskeletal (.%), nervous (.%), digestive (.%), circulatory system (.%), respiratory conditions (.%), mental and behavioral disorders (.%), as well as cancer (.0%), endocrine, nutritional and metabolic diseases (.%), and pregnancy and childbirth (.%) (Table ). The required reporting items for trial registration varied across registries (Supplement 0). Our analysis of registered records showed that the majority were phase II/III (/, 0.0%) with (.%) using parallel groups design. (.%) included a sample size estimation, (.%) reported that participants, personnel, and/or outcome assessors were blinded, and : first published as./bmjopen-0-00 on July 0. Downloaded from on November 0 by guest. Protected by copyright.

10 Page of (0.%) were two-armed (Table ). The reporting of control group varied across interventions. For example, none of the trials of tuina or qigong reported use of a placebo/sham control, while a sham control was used in almost half of the acupuncture trials (.%). Other commonly used controls were western medicine (,.%), no intervention (,.%), non-pharmaceutical interventions (,.%), acupuncture (0,.%), and Chinese herbal medicine (,.%). The information about randomization procedures including generation of allocation sequence or concealment, was under-registered and underreported across all intervention types. The estimated sample size of the trials ranged from less than (mostly pilot/feasibility trials) to over 00 participants, with the majority (.%) of sample sizes between 0 and 00 per trial (Supplement 0). 0 (.%) of the registered trials indicated status as completed. of these 0 trials (.%) had protocol publications. Our searches in PubMed and the Chinese databases identified full paper publications for from the 0 completed trials (.%). The trials produced publications reporting study outcomes, among which publications were in English and in Chinese. Among the trials with publications,.% (/) trials specified primary outcomes (Table ). When comparing the registered trials with their publications, inconsistency was identified in: sample size estimation (.0%), outcome assessor blinding (.%), secondary outcome (.%), and safety reporting (.%) (Table ). Selective outcome reporting was found in.% (/) of the subsequent publications when comparing the stated primary outcomes in the registries. Incomplete outcome data were addressed in trial publications (%). 0.% (/) of the trials reported positive results. Among the trials reporting outcomes, six trials showed other deviations from their original : first published as./bmjopen-0-00 on July 0. Downloaded from on November 0 by guest. Protected by copyright.

11 Page of protocols. For example, one study was registered as prospective cohort study but published as randomised trial; one trial was registered as parallel group but published as a cross-over trial; one trial used different intervention from the registered record and three trials showed inconsistent intervention arms from the registered information. To understand adequacy of the trial registration, we compared the date of publications with the date of registration and we found that (.%) trials were registered later than the publication date, suggesting retrospective registration. Furthermore, in trials (.%), the completion date for the trials was earlier than the approval date of the registration, also suggesting inappropriate registration. DISCUSSION The aim of prospective registration of clinical trials is to create transparency about the research methodology, and allow access to the trial protocol before completion of the trial. Researchers and systematic reviewers can then compare the registered information or published protocols and their subsequent publications to assess publication bias or selective outcome reporting., We observed a significant increase in the number of registered TCM trials in international clinical trial registries over the last decade. Our evaluation of trial quality covers a wide variety of TCM interventions from acupuncture and Chinese herbal medicine to some very ancient interventions such as cupping and gua sha therapy. These studies mainly address chronic and long term conditions such as pain, musculoskeletal and neurological problems. TCM trial design in terms of sample size estimation, use of placebo-control and blinding, and the definition of primary and secondary outcome measures is improving according to the information held in the registers. : first published as./bmjopen-0-00 on July 0. Downloaded from on November 0 by guest. Protected by copyright.

12 Page of However, we found that the generation of allocation sequence and concealment were underreported in the registered trials. This might be because these two items were not mandatory for the majority of registries. We found that % of registered TCM trials showed selective outcome reporting bias (discrepancies between the outcomes registered and the outcomes published). The inconsistency between the trial protocols and subsequent publications also relates to sample size, blinding, primary and secondary outcomes as well as safety. This implies that deviations from protocols might be due to the study findings which then resulted in selective outcome reporting. A previous study demonstrated that % of trials in Western medicine showed similar discrepancies between the registered protocols and the outcomes published in high impact general medical journals suggesting that TCM researchers are not alone in reporting outcomes selectively. Our data also suggest that there might be inappropriate registration in about half of the trials. Accordingly, there is still ample space to improve the quality of trial registration. Clinicians and policy makers need unbiased results from clinical trials to make informed clinical decisions. Our study has some limitations. First, due to the lack of standardization of the items required for registration in different registries some important information, such as randomization methods, may be underreported. Second, we only searched for publications for those trials indicating a completed status. We do not know how often this information is updated in different registries. Therefore, there may be trials in the registries that are not listed as completed, but that have : first published as./bmjopen-0-00 on July 0. Downloaded from on November 0 by guest. Protected by copyright.

13 Page of been completed. This may cause bias, particularly if the investigators who update the status of their trials on the registry are also more likely to adhere to the methods described in the registry when writing up their results in the subsequent publications. Third, as we undertook searches in PubMed and three Chinese databases only, we may have missed some studies that have been reported in other databases. Fourth, we only looked at registered trials. A very large number of TCM trials are conducted without being registered, and here we can say nothing about their risks of bias and risks of random errors. In all likelihood, these may be even worse than those we have observed. We conclude that the quality of reporting of TCM trials is not satisfactory. It is inconsistent with the registered trial protocols and may therefore involve publication bias as a consequence of selective outcome reporting. Acknowledgement We thank following people for their contributions in the study identification, data extraction including: Nini Chen, Qianyun Cai, Zhijun Liu, Xun Li, Miao Ding, Qing Li, Pan Deng. We also thank Dr Ke Cheng from University of Maryland School of Medicine, and Ms Xun Li from University of Western Sydney for retrieving full text articles. Jian-Ping Liu had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Contributors: JP Liu: conceived and designed the study, verified data extraction and analyses, drafted the manuscript. M Han: developed search strategies, identified trials, extracted data and analyzed data, revised the manuscript. XX Li, YJ Mu, YY Wang, GY Yang identified trials, extracted data and revised the manuscript. G Lewith: provided methodologic perspectives, made substantial revisions to the manuscript. C Witt: made substantial revisions to the manuscript and provided method perspectives. E Manheimer: made substantial revisions to the manuscript and provided comments. T Snellingen: helped with design of tables and data analyses, made revisions to the manuscript. B Berman: revised the manuscript and provided comments. C Gluud: conceived and designed the study and revised the manuscript and provided comments. Funding: Jian-Ping Liu, Mei Han, Xin-Xue Li, Yu-Jie Mu, Yu-Yi Wang, and Guo-Yan Yang was supported by grant number 0-CXTD-0 from Beijing University of Chinese Medicine. Brian : first published as./bmjopen-0-00 on July 0. Downloaded from on November 0 by guest. Protected by copyright.

14 Page of Berman, and Eric Manheimer were partially funded by grant number R AT00 from the National Center for Complementary and Alternative Medicine (NCCAM) of the US National Institutes of Health ( The contents of this article are solely the responsibility of the authors and do not necessarily represent the official views of the funders. The funders had no role in design and conduct of the study, data collection and analysis, interpretation of the data; and preparation, review, or approval of the manuscript. Data sharing Original data of this study can be accessed through JP Liu via . Competing Interests None REFERENCES. Tang JL, Zhan SY, Ernst E. Review of randomised controlled trials of traditional Chinese medicine. BMJ ;:0-.. Wang G, Mao B, Xiong ZY, Fan T, Chen XD, Wang L, Liu GJ, Liu J, Guo J, Chang J, Wu TX, Li TQ; CONSORT Group for Traditional Chinese Medicine. The quality of reporting of randomised controlled trials of traditional Chinese medicine: a survey of randomly selected journals from mainland China. Clin Ther 00;:-.. He J, Du L, Liu G, Fu J, He X, Yu J, Shang L. Quality assessment of reporting of randomization, allocation concealment, and blinding in traditional Chinese medicine RCTs: a review of RCTs identified from 0 systematic reviews. Trials 0;:.. Wood L, Egger M, Gluud LL, Schulz KF, Jüni P, Altman DG, Gluud C, Martin RM, Wood AJ, Sterne JA. Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: meta-epidemiological study. BMJ 00;:0-.. Savović J, Jones HE, Altman DG, Harris RJ, Jüni P, Pildal J, Als-Nielsen B, Balk EM, Gluud C, Gluud LL, Ioannidis JP, Schulz KF, Beynon R, Welton NJ, Wood L, Moher D, Deeks JJ, Sterne JA. Influence of reported study design characteristics on intervention effect estimates from randomised, controlled trials. Ann Intern Med 0;:-.. Higgins JPT, Altman DG, Sterne JAC (editors). Chapter : Assessing risk of bias in included studies. In: Higgins JPT, Green S (editors). Cochrane Handbook for Systematic Reviews of Interventions Version..0 (updated March 0). The Cochrane Collaboration, 0. Available from Vickers A, Goyal N, Harland R, Rees R. Do certain countries produce only positive results? A systematic review of controlled trials. Control Clin Trials ;:-. : first published as./bmjopen-0-00 on July 0. Downloaded from on November 0 by guest. Protected by copyright.

15 Page of Dickersin K, Rennie D. Registering clinical trials. JAMA 00;0:-.. Abaid LN, Grimes DA, Schulz KF. Reducing publication bias through trial registration. Obstet Gynecol 00;:-.. WHO. International Clinical Trials Registry Platform. Trial registration data set. March, 00: (accessed 0 Oct 0).. Reveiz L, Chan A-W, Krlezˇa-Jeric K, Granados CE, Pinart M, et al. Reporting of methodologic information on trial registries for quality assessment: A study of trial records retrieved from the WHO search portal. PLoS ONE 0; : e. doi:./journal.pone.00. Mathieu S, Boutron I, Moher D, Altman DG, Ravaud P. Comparison of registered and published primary outcomes in randomised controlled trials. JAMA 00;0:-.. Chan AW, Hróbjartsson A, Haahr MT, Gøtzsche PC, Altman DG. Empirical evidence for selective reporting of outcomes in randomised trials: comparison of protocols to published articles. JAMA 00;:-.. Chan AW, Altman DG. Identifying outcome reporting bias in randomised trials on PubMed: review of publications and survey of authors. BMJ 00;0:. Epub 00 Jan.. Liu JP, McIntosh H, Lin H. Chinese medicinal herbs for chronic hepatitis B. Cochrane Database Syst Rev 00;():CD000.. Viergever RF, Ghersi D. The quality of registration of clinical trials. PLoS One 0;:e0. LEGENDS Figure Number of Registered Randomised Trials on Traditional Chinese Medicine by Registry Table Number of Registered Randomised Trials on Traditional Chinese Medicine Table Conditions by ICD- Classification Treated by Traditional Chinese Medicine Table Methodological Information of RCTs on Traditional Chinese Medicine Table Comparison of Methodological Components between Registered Records and Subsequent Publications in RCTs Appendices 0 List of Clinical Trial Registries Appendices 0 Sample Size of Registered Randomised Trials : first published as./bmjopen-0-00 on July 0. Downloaded from on November 0 by guest. Protected by copyright.

16 Page of Figure Number of Registered Randomized Trials on Traditional Chinese Medicine (N=) Notes:. Data in 0 was not complete (Jan to July). There were no TCM trials in the Indian, Cuban, Pan-African, and Sri Lankan registries. USA China UK Australia/ New Zealand : first published as./bmjopen-0-00 on July 0. Downloaded from on November 0 by guest. Protected by copyright.

17 Page of Table Numbers of Registered Randomized Trials on Traditional Chinese Medicine Therapy USA China Australia/ New Zealand UK Iran Japan Korea Germany Brazil EU Netherlands Total Acupuncture 0 Chinese herbal medicine Combination therapy Acupressure Tuina Taichi Qigong Moxibustion Cupping Gua Sha Other therapy Total 0 Chinese herbal medicine included practitioner-prescribed herbal formula, Chinese patent medicine, and herbal extracts. Combination therapy refers to two or more Chinese medicine therapies combined together, or Chinese medicine integrated with conventional therapy; Other therapy included acupoint injection (), acupoint herbal pasters (), acupoint embedding therapy (), and TCM five-element music therapy (). : first published as./bmjopen-0-00 on July 0. Downloaded from on November 0 by guest. Protected by copyright.

18 Page of Table Conditions Based on ICD- Classification Treated by Traditional Chinese Medicine Disease/conditions (ICD- codes) No. RCTs Proportion (%) Pain. M00-M Diseases of the musculoskeletal system and connective tissue. G00-GDiseases of the nervous system. K00-K Diseases of the digestive system. I00-IDiseases of the circulatory system. J00-J Diseases of the respiratory system. F00-FMental and behavioural disorders. C00-D Neoplasms.0 E00-E0Endocrine, nutritional and metabolic diseases. O00-O Pregnancy, childbirth and the puerperium 0. Z00-Z Factors influencing health status and contact with health services. Side effects of chemotherapy for cancer. R00-R Symptoms, signs and abnormal clinical and laboratory findings, not. elsewhere classified N00-N Diseases of the genitourinary system. S00-T Injury, poisoning and certain other consequences of external causes. L00-L Diseases of the skin and subcutaneous tissue. A00-B Certain infectious and parasitic diseases.0 D0-D Diseases of the blood and blood-forming organs and certain disorders involving the immune mechanism. H00-H Diseases of the eye and adnexa. P00-P Certain conditions originating in the perinatal period 0. H0-H Diseases of the ear and mastoid process 0. Q00-Q Congenital malformations, deformations and chromosomal abnormalities 0.0 Total 0 RCTs: randomized controlled trials : first published as./bmjopen-0-00 on July 0. Downloaded from on November 0 by guest. Protected by copyright.

19 Page of Table Methodological Information of RCTs on Traditional Chinese Medicine from International Trial Registries Items Design Parallel Crossover Phase II/ III* CHM N=0 (%) Acupuncture/ Acupressure N=0 (%) Qigong/Taichi N= (%) Tuina N= (%) Other therapies Total N= (%) (.) (.) (.) (.) (.) (.) (.) (.) (.) (.) 0/ (.) / (.) 0/ (.) / (.) 0/ (.) Placebo/sham-controlled (.) (.) 0 (0.00) 0 (0.00) (.) Sample size estimation (.0) (.) (.0) (0.0) (.) Generation of allocation sequence (.) (.) (.) (.) (.) Allocation concealment (.) (.) (.) (.) (.) Blinding (.) (.0) 0 (0.) (.) (.) * Only trials (.%) provided information about phase. RCTs: randomized clinical trials; CHM: Chinese herbal medicine; Blinding included participants, personnel, and/or outcome assessors blinded. : first published as./bmjopen-0-00 on July 0. Downloaded from on November 0 by guest. Protected by copyright.

20 Page of Table Comparison of Methodological Components between Registered Records and Subsequent Publications of RCTs Items Records n= (%) Sample size (.%) Generation of allocation sequence Publication Inconsistency rate (%) n= (%).% (0%) Allocation concealment 0 0 (.%) (.%) (0.0%) Participant blinded (.%) Practitioner blinded (.%) Outcome assessor blinded 0 (.%) Primary outcome (.%) Secondary outcome (.%) Safety (.%) (.%) (.%) (.%) (.%) (.%) (0.%) (.%).% NA.%.%.%.%.%.% RCTs: randomized clinical trials; NA: not available : first published as./bmjopen-0-00 on July 0. Downloaded from on November 0 by guest. Protected by copyright.

21 Page 0 of Appendices 0 List of Trial Registries Name of registries Abbreviations Country Website Date Australian New Zealand Clinical Trials Registry ANZCTR Australia/New Portal Established 00 WHO ICTRP Zealand Brazilian Clinical Trials Registry REBEC Brazil 0 WHO ICTRP Chinese Clinical Trial Registry Chi-CTR China 00 WHO ICTRP Clinical Research Information Service CRIS Korea 00 WHO ICTRP Clinical trials.gov / USA ICMJE Clinical Trials Registry - India CTRI India 00 WHO ICTRP Cuban Public Registry of Clinical Trials RPCEC Cuba 00 WHO ICTRP EU Clinical Trials Register EU-CTR Europe 00 WHO ICTRP German Clinical Trials Register DRKS Germany 00 WHO ICTRP _web/ ICMJE International Standard Randomized Controlled ISRCTN UK 00 WHO ICTRP Trial Number Register ICMJE Iranian Registry of Clinical Trials IRCT Iran 00 WHO ICTRP Japan Primary Registries Network JPRN Japan 00 WHO ICTRP Pan African Clinical Trial Registry PACTR Africa 00 WHO ICTRP The Netherlands National Trial Register NTR Netherlands 00 WHO ICTRP Sri Lanka Clinical Trials Registry SLCTR Sri Lanka 00 WHO ICTRP WHO ICTRP: International Clinical Trials Registry Platform ICMJE: International Committee of Medical Journal Editors : first published as./bmjopen-0-00 on July 0. Downloaded from on November 0 by guest. Protected by copyright.

22 Page of Appendices 0 Estimated Sample Size of Registered Randomized Trials Sample No. of trials Proportion (%) >00. unclear 0. Total 0 : first published as./bmjopen-0-00 on July 0. Downloaded from on November 0 by guest. Protected by copyright.

23 Prospective Registration, Bias Risk and Outcome-Reporting Bias in Randomised Clinical Trials of Traditional Chinese Medicine: An Empirical Methodological Study Journal: Manuscript ID: bmjopen-0-00.r Article Type: Research Date Submitted by the Author: -May-0 Complete List of Authors: Liu, Jian-Ping; Beijing University of Chinese Medicine, Centre for Evidence- Based Chinese Medicine Han, Mei; Beijing University of Chinese Medicine, Centre for Evidence- Based Chinese Medicine Li, Xin-Xue; Beijing University of Chinese Medicine, Centre for Evidence- Based Chinese Medicine Mu, Yu_Jie; Beijing University of Chinese Medicine, Centre for Evidence- Based Chinese Medicine Lewith, George; University of Southampton, Centre for Complementary Medicine Research Wang, Yu-Yi; Beijing University of Chinese Medicine, Centre for Evidence- Based Chinese Medicine Witt, Claudia; Charité Universitätsmedizin Berlin, Institute for Social Medicine, Epidemiology and Health Economics Yang, Guo-Yan; Beijing University of Chinese Medicine, Centre for Evidence-Based Chinese Medicine Manheimer, Eric; University of Maryland School of Medicine, Center for Integrative Medicine Snellingen, Torkel; Beijing University of Chinese Medicine, Centre for Evidence-Based Chinese Medicine Berman, Brian; University of Maryland School of Medicine, Center for Integrative Medicine Gluud, Christian; Copenhagen University, Copenhagen Trial Unit <b>primary Subject Heading</b>: Complementary medicine Secondary Subject Heading: Complementary medicine, Epidemiology, Research methods Keywords: COMPLEMENTARY MEDICINE, EPIDEMIOLOGY, STATISTICS & RESEARCH METHODS, Clinical trials < THERAPEUTICS, Herbal medicine < THERAPEUTICS : first published as./bmjopen-0-00 on July 0. Downloaded from on November 0 by guest. Protected by copyright.

24 Page of Prospective Registration, Bias Risk and Outcome-Reporting Bias in Randomised Clinical Trials of Traditional Chinese Medicine: An Empirical Methodological Study Jian-Ping Liu*, Mei Han*, Xin-Xue Li, Yu-Jie Mu, George Lewith, Yu-Yi Wang, Claudia M. Witt, Guo-Yan Yang, Eric Manheimer, Torkel Snellingen, Brian Berman, Christian Gluud Jian-Ping Liu, MD, PhD, Centre for Evidence-Based Chinese Medicine, Beijing University of Chinese Medicine, Beijing, China ( Liujp@bucm.edu.cn) Mei Han, PhD candidate, Centre for Evidence-Based Chinese Medicine, Beijing University of Chinese Medicine, Beijing, China ( hanmeizoujin@.com) Xin-Xue Li, master student, Centre for Evidence-Based Chinese Medicine, Beijing University of Chinese Medicine, Beijing, China ( lixinxue@.com) Yu-Jie Mu, master student, Centre for Evidence-Based Chinese Medicine, Beijing University of Chinese Medicine, Beijing, China ( muyujie0@.com) George Lewith, MD, Centre for Complementary Medicine Research, University of Southampton, Southampton, UK ( glewith@scmrt.org.uk) Yu-Yi Wang, master student, Centre for Evidence-Based Chinese Medicine, Beijing University of Chinese Medicine, Beijing, China ( wangyuyi@.com) Claudia M. Witt, MD, MBA, Institute for Social Medicine, Epidemiology and Health Economics, Charité Universitätsmedizin Berlin, Germany ( claudia.witt@charite.de) Guo-Yan Yang, master student, Centre for Evidence-Based Chinese Medicine, Beijing University of Chinese Medicine, Beijing, China ( yangguoyanbeijing@.com) Eric Manheimer, MS, Center for Integrative Medicine, University of Maryland School of Medicine, Baltimore, Maryland, 0 USA ( emanheimer@compmed.umm.edu) Torkel Snellingen, MD, Centre for Evidence-Based Chinese Medicine, Beijing University of Chinese Medicine, Beijing, China ( tsnellingen@gmail.com) Brian Berman, MD, Center for Integrative Medicine, University of Maryland School of Medicine, Baltimore, Maryland, 0 USA ( BBerman@compmed.umm.edu) Christian Gluud, MD, Dr Med Sci, Copenhagen Trial Unit, Centre for Clinical Intervention Research, Rigshopitalet, Copenhagen University Hospital, Copenhagen, Denmark ( cgluud@ctu.rh.dk) Contact details for correspondence author: Prof Jian-Ping Liu Centre for Evidence-Based Chinese Medicine, Beijing University of Chinese Medicine, Bei San Huan Dong Lu, Chaoyang District, Beijing 00, China Phone: + 0 Fax: + Liujp@bucm.edu.cn *These two authors are equally contribute to this paper. Word count (excluding title page, abstract, references, figures and tables): 0 : first published as./bmjopen-0-00 on July 0. Downloaded from on November 0 by guest. Protected by copyright.

25 Page of ABSTRACT Background Clinical trials on traditional Chinese medicine (TCM) should be registered in a publicly accessible international trial register and report on all outcomes. We systematically assessed and evaluated TCM trial registration in registries with their subsequent publications. Objective To describe the characteristics of TCM trials, estimate bias risk and outcome reporting bias in clinical trials. Data Sources and Study Selection Fifteen trial registries were searched from their inception to July 0 to identify randomised trials on TCM including Chinese herbs, acupuncture and/or moxibustion, cupping, tuina, qigong, and dietary advice. Data Extraction We extracted data including TCM specialty and treated disease/conditions from the registries and searched for subsequent publications in PubMed and Chinese databases. We compared information in the registries of completed trials with any publications focusing on study design, sample size, randomisation, bias risk including reporting bias and deviations from the register protocol. Results registered randomised trials were identified evaluating TCM, of which 0 were completed studies (.%). The most frequent conditions were pain (.%), musculoskeletal (.%), nervous (.%), digestive (.%), circulatory (.%), respiratory (.%), mental and behavioral disorders (.%), and cancer (.0%). The trial register data identified parallel, phase II/III randomised trials with sample size estimations and blinding, but limited information about randomisation (sequence generation and allocation concealment). Comparing trial registration data of completed trials (.%) with their subsequent publications, inconsistencies were identified with respect to one or more of the following: sample size (.0%), outcome : first published as./bmjopen-0-00 on July 0. Downloaded from on November 0 by guest. Protected by copyright.

26 Page of assessor blinding (.%), primary outcomes (%), and safety (%) reporting. Conclusions Increasing numbers of clinical trials investigating a variety of TCM interventions have been registered in international trial registries. The study design of TCM trials has improved in estimating sample size, use of blinding and placebo/sham controls. However, selective outcome reporting is still widespread and therefore study conclusions should be interpreted with caution. Key words: clinical trials, traditional Chinese medicine, international trial registry, publication bias Word count: : first published as./bmjopen-0-00 on July 0. Downloaded from on November 0 by guest. Protected by copyright.

27 Page of Article focus We wished to evaluate the methodological quality of clinical trials in traditional Chinese medicine; We investigated whether the systematic identification of prior trial registration was associated with an improvement in the methodological quality of the subsequent published studies Key messages: A substantial number of clinical trials in traditional Chinese medicine, covering a broad range of therapies, are now being registered in international trial registries; Registration is associated with more rigorous study methodology and study design (e.g. randomisation protocols, secure blinding and sample size estimates); Outcome reporting bias exists when comparing the registry information and the subsequent publications, and some trials were registered after their publication. Strengths and limitations of this study Systematic searches of all available international trial registries for any clinical trials of traditional Chinese medicine; All interventions involving any traditional Chinese medicine was included as was the diagnosis; The registered information for clinical trials is not uniform across the registries and important methodological information may be missing; Subsequent publications were obtained for those studies recorded as completed in the registry. This may not represent the true situation for trials if the registry data is not updated by the researchers. : first published as./bmjopen-0-00 on July 0. Downloaded from on November 0 by guest. Protected by copyright.

28 Page of Many empirical studies have shown that the methodological quality of randomised clinical trials of traditional Chinese medicine (TCM) is poor with respect to risk of systematic errors (bias) (generation of allocation sequence, allocation concealment, blinding, descriptions of drop-out or losses to follow up, selective outcome reporting) and risk of random errors (play of chance). - Moreover, publications on TCM trials are uniformly positive, raising concerns that trials initiated to investigate TCM are only published if they have positive results. Poor quality trials and risk of publication bias will reduce the strength of evidence when developing clinical practice guidelines or preparing systematic reviews. One of the ways to improve trial quality is to prospectively register clinical trial protocols in international trial registries such as clinicaltrials.gov., The World Health Organization (WHO) established an international clinical trial registry platform (ICTRP) ( in 00, which now links clinical trial registries. Furthermore, several peer reviewed journals such as The Lancet and Trials publish trial protocols to promote transparency and improve trial quality. In order to describe the characteristics of TCM trials, and estimate reporting bias in clinical trials, we systematically searched major international trial registries to identify information about TCM trials, and compared the registered records with subsequent publications regarding outcomes and other data. METHODS Inclusion criteria We included randomised clinical trials for any TCM intervention singly or their combination: Chinese herbs, acupuncture, acupressure, moxibustion, cupping, dietary advice, tuina : first published as./bmjopen-0-00 on July 0. Downloaded from on November 0 by guest. Protected by copyright.

29 Page of (therapeutic massage), taichi, qigong and guasha (scaping massage). We excluded non-randomised studies such as quasi-randomised studies, cohort studies, phase trials, retrospective clinical studies, case series, or case studies. There were no limitations on study type (superiority, noninferiority or equivalence), or study phase. Data source We systematically searched major international trial registries ( linked to WHO ICTRP) from their inception to July 0 (Supplement 0). For trials listed as completed in the registered records, we then searched for published protocols, as well as the full texts of subsequently published articles in PubMed and three Chinese electronic bibliographic databases including Chinese Biomedical Database ( China National Knowledge Infrastructure ( and Chinese VIP Information ( Data extraction Two researchers (from MH, XL, YM, YW or GY) extracted data independently from each trial registry using a standardized, piloted data extraction form. The form was developed by our research group and based on general characteristics of clinical trials, methodology, and the 0 minimum items required for WHO trial registration. The main information collected included the number of trials in each registry, year registered, trial design, methods, sample size, setting, participants and diseases/conditions, differentiation of syndrome (bian zheng lun zhi), interventions, controls, primary and secondary outcomes, inclusions and exclusion criteria, : first published as./bmjopen-0-00 on July 0. Downloaded from on November 0 by guest. Protected by copyright.

30 Page of current status (e.g., completed, ongoing), ethical approval, sponsors, institutions, country of origin, contact details, and funding. The extracted data were cross checked by the authors, and any discrepancy resolved by discussion with a third author (JPL). Conditions were classified according to the WHO international classification of diseases ( (last accessed July 0). We searched for publications in bibliographic databases for those trials listed as completed in the registries, and compared the published trials with the registered records. Data analyses We used The Cochrane Collaboration risk of bias tool to evaluate the registered records. - This tool assesses the following domains: generation of allocation sequence; allocation concealment; blinding; incomplete outcome reporting; and selective outcome reporting (defined as change of primary outcome or new outcome added). We also evaluated the estimated sample size, explicit inclusion and exclusion criteria, and risk of funding bias. Two authors (MH, XL) compared the trial design and methodology from the registered records with the resulting publications to analyze consistency and selective outcome reporting. Selective outcome reporting was defined as when the full paper publications reported different primary outcomes or changed it from the original pre-defined primary outcome in the registered data. Inconsistency was defined as when the items were not the same as described in the registry in the subsequent paper publications. For the sample size estimation a discrepancy of over 0% between the registered and published information was judged as inconsistent. We judged trials as positive results based on () authors conclusion showing the intervention was superior to the : first published as./bmjopen-0-00 on July 0. Downloaded from on November 0 by guest. Protected by copyright.

31 Page of control; or () the comparison of between groups showed statistically significant difference (p <0.0) for primary outcome measures. We compared the dates of registration and full paper publication to assess the proportion of retrospective registrations. RESULTS We identified registered records of randomised clinical trials on TCM. 0 of the (.%) the registered records indicated that the trial had been completed. out of (0.%) were from mainland China (excluding Hong Kong and Macau) (Figure ). The first trial was registered in, in clinicaltrials.gov, and the numbers of registered trials has increased over the last decade (Figure ). Table shows the frequency of the type of TCM intervention included in each registry. The combined therapies studied included trials on integrating Chinese and Western medicine, and trials combining two or more TCM therapies. In addition, our data extraction of TCM syndrome differentiation showed that out of (.%) trials on Chinese herbal medicine utilised syndrome differentiation either in recruitment of participants ( trials) or in the prescription of the herbal formula ( trials). All these trials except one from Australia were registered by institutions in mainland China. Using the international disease classification (ICD-), we identified the most frequent conditions: pain (.%) and musculoskeletal (.%), nervous (.%), digestive (.%), circulatory system (.%), respiratory conditions (.%), mental and behavioral disorders (.%), as well as cancer (.0%), endocrine, nutritional and metabolic diseases (.%), and pregnancy and childbirth (.%) (Table ). : first published as./bmjopen-0-00 on July 0. Downloaded from on November 0 by guest. Protected by copyright.

32 Page of The required reporting items for trial registration varied across registries (Table ). Our analysis of registered records showed that the majority were phase II/III (/, 0.0%) with (.%) using parallel groups design. (.%) included a sample size estimation, (.%) reported that participants, personnel, and/or outcome assessors were blinded, and 0 (0.%) were two-armed (Table ). The reporting of control group varied across interventions. For example, none of the trials of tuina or qigong reported use of a placebo/sham control, while a sham control was used in almost half of the acupuncture trials (.%). Other commonly used controls were western medicine (,.%), no intervention (,.%), non-pharmaceutical interventions (,.%), acupuncture (0,.%), and Chinese herbal medicine (,.%). The information about randomization procedures including generation of allocation sequence or concealment, was under-registered and underreported across all intervention types. The estimated sample size of the trials ranged from less than (mostly pilot/feasibility trials) to over 00 participants, with the majority (.%) of sample sizes between 0 and 00 per trial (Supplement 0). 0 (.%) of the registered trials indicated status as completed. of these 0 trials (.%) had protocol publications. Our searches in PubMed and the Chinese databases identified full paper publications for from the 0 completed trials (.%). The trials produced publications reporting study outcomes, among which publications were in English and in Chinese. Among the trials with publications,.% (/) trials specified primary outcomes (Table ). When comparing the registered trials with their publications, inconsistency was identified in: sample size estimation (.0%), outcome assessor blinding : first published as./bmjopen-0-00 on July 0. Downloaded from on November 0 by guest. Protected by copyright.

33 Page of (.%), secondary outcome (.%), and safety reporting (.%) (Table ). Selective outcome reporting was found in.% (/) of the subsequent publications when comparing the stated primary outcomes in the registries. Incomplete outcome data were addressed in trial publications (%). 0.% (/) of the trial publications reported positive results. Among the trials reporting outcomes, six trials showed other deviations from their original protocols. For example, one study was registered as prospective cohort study but published as randomised trial; one trial was registered as parallel group but published as a cross-over trial; one trial used different intervention from the registered record and three trials showed inconsistent intervention arms from the registered information. To understand adequacy of the trial registration, we compared the date of publications with the date of registration and we found that (.%) trials were registered later than the publication date, suggesting retrospective registration. Furthermore, in trials (.%), the completion date for the trials was earlier than the approval date of the registration, also suggesting inappropriate registration. DISCUSSION The aim of prospective registration of clinical trials is to create transparency about the research methodology, and allow access to the trial protocol before completion of the trial. Researchers and systematic reviewers can then compare the registered information or published protocols and their subsequent publications to assess publication bias or selective outcome reporting., We observed a significant increase in the number of registered TCM trials in international clinical trial registries over the last decade. Our evaluation of trial quality covers a wide variety of TCM : first published as./bmjopen-0-00 on July 0. Downloaded from on November 0 by guest. Protected by copyright.

34 Page of interventions from acupuncture and Chinese herbal medicine to some very ancient interventions such as cupping and gua sha therapy. These studies mainly address chronic and long term conditions such as pain, musculoskeletal and neurological problems. TCM trial design in terms of sample size estimation, use of placebo-control and blinding, and the definition of primary and secondary outcome measures is improving according to the information held in the registers. However, we found that the generation of allocation sequence and concealment were underreported in the registered trials. This might be because these two items were not mandatory for the majority of registries. Interestingly, among trials on Chinese herbal medicine, less than a quarter of the trials (n=) utilised syndrome differentiation, which is considered vital to the TCM diagnosis as the basis for classifying subtypes of participants and for the prescription of herbal formula. Almost all of these trials with syndrome differentiation were registered by institutions in China, which suggest that Chinese researchers pay more attention to the selection of optimal participants and tailored treatment based on different TCM syndromes. We found that % of registered TCM trials showed selective outcome reporting bias (discrepancies between the outcomes registered and the outcomes published). The inconsistency between the trial protocols and subsequent publications also relates to sample size, blinding, primary and secondary outcomes as well as safety. This implies that deviations from protocols might be due to the study findings which then resulted in selective outcome reporting. A previous study demonstrated that % of trials in Western medicine showed similar discrepancies between the registered protocols and the outcomes published in high impact general medical journals suggesting that TCM researchers are not alone in reporting outcomes : first published as./bmjopen-0-00 on July 0. Downloaded from on November 0 by guest. Protected by copyright.

35 Page of selectively. Our data also suggest that there might be inappropriate registration in about half of the trials. Accordingly, there is still ample space to improve the quality of trial registration. Clinicians and policy makers need unbiased results from clinical trials to make informed clinical decisions. Our study has some limitations. First, due to the lack of standardization of the items required for registration in different registries some important information, such as randomization methods, may be underreported. Second, we only searched for publications for those trials indicating a completed status. We do not know how often this information is updated in different registries. Therefore, there may be trials in the registries that are not listed as completed, but that have been completed. This may cause bias, particularly if the investigators who update the status of their trials on the registry are also more likely to adhere to the methods described in the registry when writing up their results in the subsequent publications. Third, as we undertook searches in PubMed and three Chinese databases only, we may have missed some studies that have been reported in other databases. In addition, there is lag time between completing a study and writing for publication, submission and peer review, all of which can be considerable. Fourth, we only looked at registered trials. A very large number of TCM trials are conducted without being registered, and here we can say nothing about their risks of bias and risks of random errors. In all likelihood, these may be even worse than those we have observed. We conclude that the study design and quality of reporting of TCM trials has improved through prospective international trial registration although there are some inconsistencies between the registered trial protocols and subsequent publications. Publication bias as a consequence of : first published as./bmjopen-0-00 on July 0. Downloaded from on November 0 by guest. Protected by copyright.

36 Page of selective outcome reporting is still widespread and therefore study conclusions should be interpreted with caution. Acknowledgement We thank following people for their contributions in the study identification, data extraction including: Nini Chen, Qianyun Cai, Zhijun Liu, Xun Li, Miao Ding, Qing Li, Pan Deng. We also thank Dr Ke Cheng from University of Maryland School of Medicine, and Ms Xun Li from University of Western Sydney for retrieving full text articles. Jian-Ping Liu had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Contributors: JP Liu: conceived and designed the study, verified data extraction and analyses, drafted the manuscript. M Han: developed search strategies, identified trials, extracted data and analyzed data, revised the manuscript. XX Li, YJ Mu, YY Wang, GY Yang identified trials, extracted data and revised the manuscript. G Lewith: provided methodologic perspectives, made substantial revisions to the manuscript. C Witt: made substantial revisions to the manuscript and provided method perspectives. E Manheimer: made substantial revisions to the manuscript and provided comments. T Snellingen: helped with design of tables and data analyses, made revisions to the manuscript. B Berman: revised the manuscript and provided comments. C Gluud: conceived and designed the study and revised the manuscript and provided comments. Funding: Jian-Ping Liu, Mei Han, Xin-Xue Li, Yu-Jie Mu, Yu-Yi Wang, and Guo-Yan Yang was supported by grant number 0-CXTD-0 from Beijing University of Chinese Medicine. Brian Berman, Jian-Ping Liu, and Eric Manheimer were partially funded by grant number R AT00 from the National Center for Complementary and Alternative Medicine (NCCAM) of the US National Institutes of Health ( Jian-Ping Liu and Mei Han were partially funded by the grant number 0ZX () by the Ministry of Science and Technology of China. The contents of this article are solely the responsibility of the authors and do not necessarily represent the official views of the funders. The funders had no role in design and conduct of the study, data collection and analysis, interpretation of the data; and preparation, review, or approval of the manuscript. REFERENCES. Tang JL, Zhan SY, Ernst E. Review of randomised controlled trials of traditional Chinese medicine. BMJ ;:0-.. Wang G, Mao B, Xiong ZY, Fan T, Chen XD, Wang L, Liu GJ, Liu J, Guo J, Chang J, Wu TX, Li TQ; CONSORT Group for Traditional Chinese Medicine. The quality of reporting of randomised controlled trials of traditional Chinese medicine: a survey of randomly selected journals from mainland China. Clin Ther 00;:-. : first published as./bmjopen-0-00 on July 0. Downloaded from on November 0 by guest. Protected by copyright.

37 Page of He J, Du L, Liu G, Fu J, He X, Yu J, Shang L. Quality assessment of reporting of randomization, allocation concealment, and blinding in traditional Chinese medicine RCTs: a review of RCTs identified from 0 systematic reviews. Trials 0;:.. Wood L, Egger M, Gluud LL, Schulz KF, Jüni P, Altman DG, Gluud C, Martin RM, Wood AJ, Sterne JA. Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: meta-epidemiological study. BMJ 00;:0-.. Savović J, Jones HE, Altman DG, Harris RJ, Jüni P, Pildal J, Als-Nielsen B, Balk EM, Gluud C, Gluud LL, Ioannidis JP, Schulz KF, Beynon R, Welton NJ, Wood L, Moher D, Deeks JJ, Sterne JA. Influence of reported study design characteristics on intervention effect estimates from randomised, controlled trials. Ann Intern Med 0;:-.. Higgins JPT, Altman DG, Sterne JAC (editors). Chapter : Assessing risk of bias in included studies. In: Higgins JPT, Green S (editors). Cochrane Handbook for Systematic Reviews of Interventions Version..0 (updated March 0). The Cochrane Collaboration, 0. Available from Vickers A, Goyal N, Harland R, Rees R. Do certain countries produce only positive results? A systematic review of controlled trials. Control Clin Trials ;:-.. Dickersin K, Rennie D. Registering clinical trials. JAMA 00;0:-.. Abaid LN, Grimes DA, Schulz KF. Reducing publication bias through trial registration. Obstet Gynecol 00;:-.. WHO. International Clinical Trials Registry Platform. Trial registration data set. March, 00: (accessed 0 Oct 0).. Reveiz L, Chan A-W, Krlezˇa-Jeric K, Granados CE, Pinart M, et al. Reporting of methodologic information on trial registries for quality assessment: A study of trial records retrieved from the WHO search portal. PLoS ONE 0; : e. doi:./journal.pone.00. Mathieu S, Boutron I, Moher D, Altman DG, Ravaud P. Comparison of registered and published primary outcomes in randomised controlled trials. JAMA 00;0:-.. Chan AW, Hróbjartsson A, Haahr MT, Gøtzsche PC, Altman DG. Empirical evidence for selective reporting of outcomes in randomised trials: comparison of protocols to published articles. JAMA 00;:-.. Chan AW, Altman DG. Identifying outcome reporting bias in randomised trials on PubMed: review of publications and survey of authors. BMJ 00;0:. Epub 00 Jan. : first published as./bmjopen-0-00 on July 0. Downloaded from on November 0 by guest. Protected by copyright.

38 Page of Liu JP, McIntosh H, Lin H. Chinese medicinal herbs for chronic hepatitis B. Cochrane Database Syst Rev 00;():CD000.. Viergever RF, Ghersi D. The quality of registration of clinical trials. PLoS One 0;:e0. LEGENDS Figure Number of Registered Randomised Trials on Traditional Chinese Medicine by Registry Table Number of Registered Randomised Trials on Traditional Chinese Medicine Table Conditions by ICD- Classification Treated by Traditional Chinese Medicine Table Methodological Information of RCTs on Traditional Chinese Medicine Table Comparison of Methodological Components between Registered Records and Subsequent Publications in RCTs Appendices 0 List of Clinical Trial Registries Appendices 0 Sample Size of Registered Randomised Trials : first published as./bmjopen-0-00 on July 0. Downloaded from on November 0 by guest. Protected by copyright.

39 Page of Prospective Registration, Bias Risk and Outcome-Reporting Bias in Randomised Clinical Trials of Traditional Chinese Medicine: An Empirical Methodological Study Jian-Ping Liu*, Mei Han*, Xin-Xue Li, Yu-Jie Mu, George Lewith, Yu-Yi Wang, Claudia M. Witt, Guo-Yan Yang, Eric Manheimer, Torkel Snellingen, Brian Berman, Christian Gluud Jian-Ping Liu, MD, PhD, Centre for Evidence-Based Chinese Medicine, Beijing University of Chinese Medicine, Beijing, China ( Liujp@bucm.edu.cn) Mei Han, PhD candidate, Centre for Evidence-Based Chinese Medicine, Beijing University of Chinese Medicine, Beijing, China ( hanmeizoujin@.com) Xin-Xue Li, master student, Centre for Evidence-Based Chinese Medicine, Beijing University of Chinese Medicine, Beijing, China ( lixinxue@.com) Yu-Jie Mu, master student, Centre for Evidence-Based Chinese Medicine, Beijing University of Chinese Medicine, Beijing, China ( muyujie0@.com) George Lewith, MD, Centre for Complementary Medicine Research, University of Southampton, Southampton, UK ( glewith@scmrt.org.uk) Yu-Yi Wang, master student, Centre for Evidence-Based Chinese Medicine, Beijing University of Chinese Medicine, Beijing, China ( wangyuyi@.com) Claudia M. Witt, MD, MBA, Institute for Social Medicine, Epidemiology and Health Economics, Charité Universitätsmedizin Berlin, Germany ( claudia.witt@charite.de) Guo-Yan Yang, master student, Centre for Evidence-Based Chinese Medicine, Beijing University of Chinese Medicine, Beijing, China ( yangguoyanbeijing@.com) Eric Manheimer, MS, Center for Integrative Medicine, University of Maryland School of Medicine, Baltimore, Maryland, 0 USA ( emanheimer@compmed.umm.edu) Torkel Snellingen, MD, Centre for Evidence-Based Chinese Medicine, Beijing University of Chinese Medicine, Beijing, China ( tsnellingen@gmail.com) Brian Berman, MD, Center for Integrative Medicine, University of Maryland School of Medicine, Baltimore, Maryland, 0 USA ( BBerman@compmed.umm.edu) Christian Gluud, MD, Dr Med Sci, Copenhagen Trial Unit, Centre for Clinical Intervention Research, Rigshopitalet, Copenhagen University Hospital, Copenhagen, Denmark ( cgluud@ctu.rh.dk) Contact details for correspondence author: Prof Jian-Ping Liu Centre for Evidence-Based Chinese Medicine, Beijing University of Chinese Medicine, Bei San Huan Dong Lu, Chaoyang District, Beijing 00, China Phone: + 0 Fax: + Liujp@bucm.edu.cn *These two authors are equally contribute to this paper. Word count (excluding title page, abstract, references, figures and tables): 0 : first published as./bmjopen-0-00 on July 0. Downloaded from on November 0 by guest. Protected by copyright.

40 Page of ABSTRACT Background Clinical trials on traditional Chinese medicine (TCM) should be registered in a publicly accessible international trial register and report on all outcomes. We systematically assessed and evaluated TCM trial registration in registries with their subsequent publications. Objective To describe the characteristics of TCM trials, estimate bias risk and outcome reporting bias in clinical trials. Data Sources and Study Selection Fifteen trial registries were searched from their inception to July 0 to identify randomised trials on TCM including Chinese herbs, acupuncture and/or moxibustion, cupping, tuina, qigong, and dietary advice. Data Extraction We extracted data including TCM specialty and treated disease/conditions from the registries and searched for subsequent publications in PubMed and Chinese databases. We compared information in the registries of completed trials with any publications focusing on study design, sample size, randomisation, bias risk including reporting bias and deviations from the register protocol. Results registered randomised trials were identified evaluating TCM, of which 0 were completed studies (.%). The most frequent conditions were pain (.%), musculoskeletal (.%), nervous (.%), digestive (.%), circulatory (.%), respiratory (.%), mental and behavioral disorders (.%), and cancer (.0%). The trial register data identified parallel, phase II/III randomised trials with sample size estimations and blinding, but limited information about randomiszation (sequence generation of allocation sequence and allocation concealment). Comparing trial registration data of completed trials (.%) with their subsequent publications, inconsistencies were identified with respect to one or more of the following: sample : first published as./bmjopen-0-00 on July 0. Downloaded from on November 0 by guest. Protected by copyright.

41 Page of size (.0%), outcome assessor blinding (.%), primary outcomes (%), and safety (%) reporting. Conclusions Increasing numbers of clinical trials investigating a variety of TCM interventions have been registered in international trial registries. The study design of TCM trials has improved in estimatinged sample size, use of blinding and placebo/sham controls. However, the quality of reporting of these trials is inconsistent with the registered data and involves selective outcome reporting is still widespread and therefore study conclusions biasshould be interpreted with cautionus in future. Key words: clinical trials, traditional Chinese medicine, international trial registry, publication bias Word count: : first published as./bmjopen-0-00 on July 0. Downloaded from on November 0 by guest. Protected by copyright.

42 Page of Article focus We wished to evaluate the methodological quality of clinical trials in traditional Chinese medicine; We investigated whether the systematic identification of prior trial registration was associated with an improvement in the methodological quality of the subsequent published studies Key messages: A substantial number of clinical trials in traditional Chinese medicine, covering a broad range of therapies, are now being registered in international trial registries; Registration is associated with more rigorous study methodology and study design (e.g. randomisation protocols, secure blinding and sample size estimates); Outcome reporting bias exists when comparing the registry information and the subsequent publications, and some trials were registered after their publication. Strengths and limitations of this study Systematic searches of all available international trial registries for any clinical trials of traditional Chinese medicine; All interventions involving any traditional Chinese medicine was included as was the diagnosis; The registered information for clinical trials is not uniform across the registries and important methodological information may be missing; Subsequent publications were obtained for those studies recorded as completed in the registry. This may not represent the true situation for trials if the registry data is not updated by the researchers. : first published as./bmjopen-0-00 on July 0. Downloaded from on November 0 by guest. Protected by copyright.

43 Page 0 of Many empirical studies have shown that the methodological quality of randomised clinical trials of traditional Chinese medicine (TCM) is poor with respect to risk of systematic errors (bias) (generation of allocation sequence, allocation concealment, blinding, descriptions of drop-out or losses to follow up, selective outcome reporting) and risk of random errors (play of chance). - Moreover, publications on TCM trials are uniformly positive, raising concerns that trials initiated to investigate TCM are only published if they have positive results. Poor quality trials and risk of publication bias will reduce the strength of evidence when developing clinical practice guidelines or preparing systematic reviews. One of the ways to improve trial quality is to prospectively register clinical trial protocols in international trial registries such as clinicaltrials.gov., The World Health Organization (WHO) established an international clinical trial registry platform (ICTRP) ( in 00, which now links clinical trial registries. Furthermore, several peer reviewed journals such as The Lancet and Trials publish trial protocols to promote transparency and improve trial quality. In order to describe the characteristics of TCM trials, and estimate reporting bias in clinical trials, we systematically searched major international trial registries to identify information about TCM trials, and compared the registered records with subsequent publications regarding outcomes and other data. METHODS Inclusion criteria We included randomised clinical trials for any TCM intervention singly or their combination: Chinese herbs, acupuncture, acupressure, moxibustion, cupping, dietary advice, tuina : first published as./bmjopen-0-00 on July 0. Downloaded from on November 0 by guest. Protected by copyright.

44 Page of (therapeutic massage), taichi, qigong and guasha (scaping massage). We excluded non-randomised studies such as quasi-randomised studies, cohort studies, phase trials, retrospective clinical studies, case series, or case studies. There were no limitations on study type (superiority, noninferiority or equivalence), or study phase. Data source We systematically searched major international trial registries ( linked to WHO ICTRP) from their inception to July 0 (Supplement 0). For trials listed as completed in the registered records, we then searched for published protocols, as well as the full texts of subsequently published articles in PubMed and three Chinese electronic bibliographic databases including Chinese Biomedical Database ( China National Knowledge Infrastructure ( and Chinese VIP Information ( Data extraction Two researchers (from MH, XL, YM, YW or GY) extracted data independently from each trial registry using a standardized, piloted data extraction form. The form was developed by our research group and based on general characteristics of clinical trials, methodology, and the 0 minimum items required for WHO trial registration. The main information collected included the number of trials in each registry, year registered, trial design, methods, sample size, setting, participants and diseases/conditions, differentiation of syndrome (bian zheng lun zhi), interventions, controls, primary and secondary outcomes, inclusions and exclusion criteria, : first published as./bmjopen-0-00 on July 0. Downloaded from on November 0 by guest. Protected by copyright.

45 Page of current status (e.g., completed, ongoing), ethical approval, sponsors, institutions, country of origin, contact details, and funding. The extracted data were cross checked by the authors, and any discrepancy resolved by discussion with a third author (JPL). Conditions were classified according to the WHO international classification of diseases ( (last accessed July 0). We searched for publications in bibliographic databases for those trials listed as completed in the registries, and compared the published trials with the registered records. Data analyses We used The Cochrane Collaboration risk of bias tool to evaluate the registered records. - This tool assesses the following domains: generation of allocation sequence; allocation concealment; blinding; incomplete outcome reporting; and selective outcome reporting (defined as change of primary outcome or new outcome added). We also evaluated the estimated sample size, explicit inclusion and exclusion criteria, and risk of funding bias. Two authors (MH, XL) compared the trial design and methodology from the registered records with the resulting publications to analyze consistency and selective outcome reporting. Selective outcome reporting was defined as when the full paper publications reported different primary outcomes or changed it from the original pre-defined primary outcome in the registered data. Inconsistency was defined as when the items were not the same as described in the registry in the subsequent paper publications. For the sample size estimation a discrepancy of over 0% between the registered and published information was judged as inconsistent. We judged trials as positive results based on () authors conclusion showing the intervention was superior to the : first published as./bmjopen-0-00 on July 0. Downloaded from on November 0 by guest. Protected by copyright.

46 Page of control; or () the comparison of between groups showed statistically significant difference (p <0.0) for primary outcome measures. We compared the dates of registration and full paper publication to assess the proportion of retrospective registrations. RESULTS We identified registered records of randomised clinical trials on TCM. 0 of the (.%) the registered records indicated that the trial had been completed. out of (0.%) were from mainland China (excluding Hong Kong and Macau) (Figure ). The first trial was registered in, in clinicaltrials.gov, and the numbers of registered trials has increased over the last decade (Figure ). Table shows the frequency of the type of TCM intervention included in each registry. The combined therapies studied included trials on integrating Chinese and Western medicine, and trials combining two or more TCM therapies. In addition, our data extraction of TCM syndrome differentiation showed that out of (.%) trials on Chinese herbal medicine utilised syndrome differentiation either in recruitment of participants ( trials) or in the prescription of the herbal formula ( trials). All these trials except one from Australia were registered by institutions in mainland China. Using the international disease classification (ICD-), we identified the most frequent conditions: pain (.%) and musculoskeletal (.%), nervous (.%), digestive (.%), circulatory system (.%), respiratory conditions (.%), mental and behavioral disorders (.%), as well as cancer (.0%), endocrine, nutritional and metabolic diseases (.%), and pregnancy and childbirth (.%) (Table ). : first published as./bmjopen-0-00 on July 0. Downloaded from on November 0 by guest. Protected by copyright.

47 Page of The required reporting items for trial registration varied across registries (Table Supplement 0). Our analysis of registered records showed that the majority were phase II/III (/, 0.0%) with (.%) using parallel groups design. (.%) included a sample size estimation, (.%) reported that participants, personnel, and/or outcome assessors were blinded, and 0 (0.%) were two-armed (Table ). The reporting of control group varied across interventions. For example, none of the trials of tuina or qigong reported use of a placebo/sham control, while a sham control was used in almost half of the acupuncture trials (.%). Other commonly used controls were western medicine (,.%), no intervention (,.%), non-pharmaceutical interventions (,.%), acupuncture (0,.%), and Chinese herbal medicine (,.%). The information about randomization procedures including generation of allocation sequence or concealment, was under-registered and underreported across all intervention types. The estimated sample size of the trials ranged from less than (mostly pilot/feasibility trials) to over 00 participants, with the majority (.%) of sample sizes between 0 and 00 per trial (Supplement 0). 0 (.%) of the registered trials indicated status as completed. of these 0 trials (.%) had protocol publications. Our searches in PubMed and the Chinese databases identified full paper publications for from the 0 completed trials (.%). The trials produced publications reporting study outcomes, among which publications were in English and in Chinese. Among the trials with publications,.% (/) trials specified primary outcomes (Table ). When comparing the registered trials with their publications, : first published as./bmjopen-0-00 on July 0. Downloaded from on November 0 by guest. Protected by copyright.

48 Page of inconsistency was identified in: sample size estimation (.0%), outcome assessor blinding (.%), secondary outcome (.%), and safety reporting (.%) (Table ). Selective outcome reporting was found in.% (/) of the subsequent publications when comparing the stated primary outcomes in the registries. Incomplete outcome data were addressed in trial publications (%). 0.% (/) of the trial publications reported positive results. Among the trials reporting outcomes, six trials showed other deviations from their original protocols. For example, one study was registered as prospective cohort study but published as randomised trial; one trial was registered as parallel group but published as a cross-over trial; one trial used different intervention from the registered record and three trials showed inconsistent intervention arms from the registered information. To understand adequacy of the trial registration, we compared the date of publications with the date of registration and we found that (.%) trials were registered later than the publication date, suggesting retrospective registration. Furthermore, in trials (.%), the completion date for the trials was earlier than the approval date of the registration, also suggesting inappropriate registration. DISCUSSION The aim of prospective registration of clinical trials is to create transparency about the research methodology, and allow access to the trial protocol before completion of the trial. Researchers and systematic reviewers can then compare the registered information or published protocols and their subsequent publications to assess publication bias or selective outcome reporting., We observed a significant increase in the number of registered TCM trials in international clinical : first published as./bmjopen-0-00 on July 0. Downloaded from on November 0 by guest. Protected by copyright.

49 Page of trial registries over the last decade. Our evaluation of trial quality covers a wide variety of TCM interventions from acupuncture and Chinese herbal medicine to some very ancient interventions such as cupping and gua sha therapy. These studies mainly address chronic and long term conditions such as pain, musculoskeletal and neurological problems. TCM trial design in terms of sample size estimation, use of placebo-control and blinding, and the definition of primary and secondary outcome measures is improving according to the information held in the registers. However, we found that the generation of allocation sequence and concealment were underreported in the registered trials. This might be because these two items were not mandatory for the majority of registries. Interestingly, among trials on Chinese herbal medicine, less than a quarter of the trials (n=) utilised syndrome differentiation, which is considered vital to the TCM diagnosis as the basis for classifying subtypes of participants, and for the prescription of herbal formula. Almost all of these trials with syndrome differentiation were registered by institutions in China, which suggest that Chinese researchers pay more attention to the selection of optimal participants and tailored treatment based on different TCM syndromes. We found that % of registered TCM trials showed selective outcome reporting bias (discrepancies between the outcomes registered and the outcomes published). The inconsistency between the trial protocols and subsequent publications also relates to sample size, blinding, primary and secondary outcomes as well as safety. This implies that deviations from protocols might be due to the study findings which then resulted in selective outcome reporting. A previous study demonstrated that % of trials in Western medicine showed similar discrepancies between the registered protocols and the outcomes published in high impact : first published as./bmjopen-0-00 on July 0. Downloaded from on November 0 by guest. Protected by copyright.

50 Page of general medical journals suggesting that TCM researchers are not alone in reporting outcomes selectively. Our data also suggest that there might be inappropriate registration in about half of the trials. Accordingly, there is still ample space to improve the quality of trial registration. Clinicians and policy makers need unbiased results from clinical trials to make informed clinical decisions. Our study has some limitations. First, due to the lack of standardization of the items required for registration in different registries some important information, such as randomization methods, may be underreported. Second, we only searched for publications for those trials indicating a completed status. We do not know how often this information is updated in different registries. Therefore, there may be trials in the registries that are not listed as completed, but that have been completed. This may cause bias, particularly if the investigators who update the status of their trials on the registry are also more likely to adhere to the methods described in the registry when writing up their results in the subsequent publications. Third, as we undertook searches in PubMed and three Chinese databases only, we may have missed some studies that have been reported in other databases. In addition, there is lag time between completing a study and writing for publication, submission and peer review, all of which can be considerable. Fourth, we only looked at registered trials. A very large number of TCM trials are conducted without being registered, and here we can say nothing about their risks of bias and risks of random errors. In all likelihood, these may be even worse than those we have observed. We conclude that the study design and quality of reporting of TCM trials has improved through prospective international trial registration although there are somehas not satisfactory. It is : first published as./bmjopen-0-00 on July 0. Downloaded from on November 0 by guest. Protected by copyright.

51 Page of inconsistenciest betweenwith the registered trial protocols and subsequent publications. may therefore involve ppublication bias as a consequence of selective outcome reporting selective outcome reporting is still widespread and therefore study conclusions should be interpreted with cautionshould be cautioned in future trials. Acknowledgement We thank following people for their contributions in the study identification, data extraction including: Nini Chen, Qianyun Cai, Zhijun Liu, Xun Li, Miao Ding, Qing Li, Pan Deng. We also thank Dr Ke Cheng from University of Maryland School of Medicine, and Ms Xun Li from University of Western Sydney for retrieving full text articles. Jian-Ping Liu had full access to all of the data in the study and takes responsibility for the integrity of the data and the accuracy of the data analysis. Contributors: JP Liu: conceived and designed the study, verified data extraction and analyses, drafted the manuscript. M Han: developed search strategies, identified trials, extracted data and analyzed data, revised the manuscript. XX Li, YJ Mu, YY Wang, GY Yang identified trials, extracted data and revised the manuscript. G Lewith: provided methodologic perspectives, made substantial revisions to the manuscript. C Witt: made substantial revisions to the manuscript and provided method perspectives. E Manheimer: made substantial revisions to the manuscript and provided comments. T Snellingen: helped with design of tables and data analyses, made revisions to the manuscript. B Berman: revised the manuscript and provided comments. C Gluud: conceived and designed the study and revised the manuscript and provided comments. Funding: Jian-Ping Liu, Mei Han, Xin-Xue Li, Yu-Jie Mu, Yu-Yi Wang, and Guo-Yan Yang was supported by grant number 0-CXTD-0 from Beijing University of Chinese Medicine. Brian Berman, Jian-Ping Liu, and Eric Manheimer were partially funded by grant number R AT00 from the National Center for Complementary and Alternative Medicine (NCCAM) of the US National Institutes of Health ( Jian-Ping Liu and Mei Han were partially funded by the grant number 0ZX () by the Ministry of Science and Technology of China. The contents of this article are solely the responsibility of the authors and do not necessarily represent the official views of the funders. The funders had no role in design and conduct of the study, data collection and analysis, interpretation of the data; and preparation, review, or approval of the manuscript. REFERENCES. Tang JL, Zhan SY, Ernst E. Review of randomised controlled trials of traditional Chinese medicine. BMJ ;:0-.. Wang G, Mao B, Xiong ZY, Fan T, Chen XD, Wang L, Liu GJ, Liu J, Guo J, Chang J, Wu TX, Li TQ; : first published as./bmjopen-0-00 on July 0. Downloaded from on November 0 by guest. Protected by copyright.

52 Page of CONSORT Group for Traditional Chinese Medicine. The quality of reporting of randomised controlled trials of traditional Chinese medicine: a survey of randomly selected journals from mainland China. Clin Ther 00;:-.. He J, Du L, Liu G, Fu J, He X, Yu J, Shang L. Quality assessment of reporting of randomization, allocation concealment, and blinding in traditional Chinese medicine RCTs: a review of RCTs identified from 0 systematic reviews. Trials 0;:.. Wood L, Egger M, Gluud LL, Schulz KF, Jüni P, Altman DG, Gluud C, Martin RM, Wood AJ, Sterne JA. Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: meta-epidemiological study. BMJ 00;:0-.. Savović J, Jones HE, Altman DG, Harris RJ, Jüni P, Pildal J, Als-Nielsen B, Balk EM, Gluud C, Gluud LL, Ioannidis JP, Schulz KF, Beynon R, Welton NJ, Wood L, Moher D, Deeks JJ, Sterne JA. Influence of reported study design characteristics on intervention effect estimates from randomised, controlled trials. Ann Intern Med 0;:-.. Higgins JPT, Altman DG, Sterne JAC (editors). Chapter : Assessing risk of bias in included studies. In: Higgins JPT, Green S (editors). Cochrane Handbook for Systematic Reviews of Interventions Version..0 (updated March 0). The Cochrane Collaboration, 0. Available from Vickers A, Goyal N, Harland R, Rees R. Do certain countries produce only positive results? A systematic review of controlled trials. Control Clin Trials ;:-.. Dickersin K, Rennie D. Registering clinical trials. JAMA 00;0:-.. Abaid LN, Grimes DA, Schulz KF. Reducing publication bias through trial registration. Obstet Gynecol 00;:-.. WHO. International Clinical Trials Registry Platform. Trial registration data set. March, 00: (accessed 0 Oct 0).. Reveiz L, Chan A-W, Krlezˇa-Jeric K, Granados CE, Pinart M, et al. Reporting of methodologic information on trial registries for quality assessment: A study of trial records retrieved from the WHO search portal. PLoS ONE 0; : e. doi:./journal.pone.00. Mathieu S, Boutron I, Moher D, Altman DG, Ravaud P. Comparison of registered and published primary outcomes in randomised controlled trials. JAMA 00;0:-.. Chan AW, Hróbjartsson A, Haahr MT, Gøtzsche PC, Altman DG. Empirical evidence for selective reporting of outcomes in randomised trials: comparison of protocols to published articles. JAMA 00;:-. : first published as./bmjopen-0-00 on July 0. Downloaded from on November 0 by guest. Protected by copyright.

53 Page 0 of Chan AW, Altman DG. Identifying outcome reporting bias in randomised trials on PubMed: review of publications and survey of authors. BMJ 00;0:. Epub 00 Jan.. Liu JP, McIntosh H, Lin H. Chinese medicinal herbs for chronic hepatitis B. Cochrane Database Syst Rev 00;():CD000.. Viergever RF, Ghersi D. The quality of registration of clinical trials. PLoS One 0;:e0. LEGENDS Figure Number of Registered Randomised Trials on Traditional Chinese Medicine by Registry Table Number of Registered Randomised Trials on Traditional Chinese Medicine Table Conditions by ICD- Classification Treated by Traditional Chinese Medicine Table Methodological Information of RCTs on Traditional Chinese Medicine Table Comparison of Methodological Components between Registered Records and Subsequent Publications in RCTs Appendices 0 List of Clinical Trial Registries Appendices 0 Sample Size of Registered Randomised Trials : first published as./bmjopen-0-00 on July 0. Downloaded from on November 0 by guest. Protected by copyright.

54 Page of WHO International Clinical Trials Registry Platform (ICTRP) Clinicaltrials.gov n= Registries n= Registry Protocols n=0 Duplicates, non-tcm, CCT, cohort study, case-control, case series, etc RCT protocols n= n= Data extraction Uncompleted n= Completed n=0 Searching PUBMED, CNKI, VIP, CBM Data analysis Publications n= ( Trials) Data extraction Data analysis Figure Flow diagram of included trials in this study Abbreviation: RCT: Randomised control trial CCT: Controlled clinical trial TCM: Traditional Chinese medicine CNKI: China National Knowledge Infrastructure VIP: VIP Database CBM: Chinese Biomedical Database : first published as./bmjopen-0-00 on July 0. Downloaded from on November 0 by guest. Protected by copyright.

55 Page of Figure Number of Registered Randomized Trials on Traditional Chinese Medicine (N=) Notes:. Data in 0 was not complete (Jan to July). There were no TCM trials in the Indian, Cuban, Pan-African, and Sri Lankan registries. USA China UK Australia/ New Zealand : first published as./bmjopen-0-00 on July 0. Downloaded from on November 0 by guest. Protected by copyright.

56 Page of Table Numbers of Registered Randomized Trials on Traditional Chinese Medicine Therapy USA China Australia/ New Zealand UK Iran Japan Korea Germany Brazil EU Netherlands Total Acupuncture 0 Chinese herbal medicine Combination therapy Acupressure Tuina Taichi Qigong Moxibustion Cupping Gua Sha Other therapy Total 0 Chinese herbal medicine included practitioner-prescribed herbal formula, Chinese patent medicine, and herbal extracts. Combination therapy refers to two or more Chinese medicine therapies combined together, or Chinese medicine integrated with conventional therapy; Other therapy included acupoint injection (), acupoint herbal pasters (), acupoint embedding therapy (), and TCM five-element music therapy (). : first published as./bmjopen-0-00 on July 0. Downloaded from on November 0 by guest. Protected by copyright.

57 Page of Table Conditions Based on ICD- Classification Treated by Traditional Chinese Medicine Disease/conditions (ICD- codes) No. RCTs Proportion (%) Pain. M00-M Diseases of the musculoskeletal system and connective tissue. G00-GDiseases of the nervous system. K00-K Diseases of the digestive system. I00-IDiseases of the circulatory system. J00-J Diseases of the respiratory system. F00-FMental and behavioural disorders. C00-D Neoplasms.0 E00-E0Endocrine, nutritional and metabolic diseases. O00-O Pregnancy, childbirth and the puerperium 0. Z00-Z Factors influencing health status and contact with health services. Side effects of chemotherapy for cancer. R00-R Symptoms, signs and abnormal clinical and laboratory findings, not. elsewhere classified N00-N Diseases of the genitourinary system. S00-T Injury, poisoning and certain other consequences of external causes. L00-L Diseases of the skin and subcutaneous tissue. A00-B Certain infectious and parasitic diseases.0 D0-D Diseases of the blood and blood-forming organs and certain disorders involving the immune mechanism. H00-H Diseases of the eye and adnexa. P00-P Certain conditions originating in the perinatal period 0. H0-H Diseases of the ear and mastoid process 0. Q00-Q Congenital malformations, deformations and chromosomal abnormalities 0.0 Total 0 RCTs: randomized controlled trials : first published as./bmjopen-0-00 on July 0. Downloaded from on November 0 by guest. Protected by copyright.

58 Page of Table Methodological Information of RCTs on Traditional Chinese Medicine from International Trial Registries Items Design Parallel Crossover Phase II/ III* CHM N=0 (%) Acupuncture/ Acupressure N=0 (%) Qigong/Taichi N= (%) Tuina N= (%) Other therapies Total N= (%) (.) (.) (.) (.) (.) (.) (.) (.) (.) (.) 0/ (.) / (.) 0/ (.) / (.) 0/ (.) Placebo/sham-controlled (.) (.) 0 (0.00) 0 (0.00) (.) Sample size estimation (.0) (.) (.0) (0.0) (.) Generation of allocation sequence (.) (.) (.) (.) (.) Allocation concealment (.) (.) (.) (.) (.) Blinding (.) (.0) 0 (0.) (.) (.) * Only trials (.%) provided information about phase. RCTs: randomized clinical trials; CHM: Chinese herbal medicine; Blinding included participants, personnel, and/or outcome assessors blinded. : first published as./bmjopen-0-00 on July 0. Downloaded from on November 0 by guest. Protected by copyright.

59 Page of Table Comparison of Methodological Components between Registered Records and Subsequent Publications of RCTs Items Records n= (%) Sample size (.%) Generation of allocation sequence Publication Inconsistency rate (%) n= (%).% (0%) Allocation concealment 0 0 (.%) (.%) (0.0%) Participant blinded (.%) Practitioner blinded (.%) Outcome assessor blinded 0 (.%) Primary outcome (.%) Secondary outcome (.%) Safety (.%) (.%) (.%) (.%) (.%) (.%) (0.%) (.%).% NA.%.%.%.%.%.% RCTs: randomized clinical trials; NA: not available : first published as./bmjopen-0-00 on July 0. Downloaded from on November 0 by guest. Protected by copyright.

60 Page of Appendices 0 List of Trial Registries Name of registries Abbreviations Country Website Date Australian New Zealand Clinical Trials Registry ANZCTR Australia/New Portal Established 00 WHO ICTRP Zealand Brazilian Clinical Trials Registry REBEC Brazil 0 WHO ICTRP Chinese Clinical Trial Registry Chi-CTR China 00 WHO ICTRP Clinical Research Information Service CRIS Korea 00 WHO ICTRP Clinical trials.gov / USA ICMJE Clinical Trials Registry - India CTRI India 00 WHO ICTRP Cuban Public Registry of Clinical Trials RPCEC Cuba 00 WHO ICTRP EU Clinical Trials Register EU-CTR Europe 00 WHO ICTRP German Clinical Trials Register DRKS Germany 00 WHO ICTRP _web/ ICMJE International Standard Randomized Controlled ISRCTN UK 00 WHO ICTRP Trial Number Register ICMJE Iranian Registry of Clinical Trials IRCT Iran 00 WHO ICTRP Japan Primary Registries Network JPRN Japan 00 WHO ICTRP Pan African Clinical Trial Registry PACTR Africa 00 WHO ICTRP The Netherlands National Trial Register NTR Netherlands 00 WHO ICTRP Sri Lanka Clinical Trials Registry SLCTR Sri Lanka 00 WHO ICTRP WHO ICTRP: International Clinical Trials Registry Platform ICMJE: International Committee of Medical Journal Editors : first published as./bmjopen-0-00 on July 0. Downloaded from on November 0 by guest. Protected by copyright.

61 Page of Appendices 0 Estimated Sample Size of Registered Randomized Trials Sample No. of trials Proportion (%) >00. unclear 0. Total 0 : first published as./bmjopen-0-00 on July 0. Downloaded from on November 0 by guest. Protected by copyright.

62 Page of STROBE Statement Checklist of items Item No Recommendation Title and abstract (a) Indicate the study s design with a commonly used term in the title or the abstract Introduction (b) Provide in the abstract an informative and balanced summary of what was done and what was found Background/rationale Explain the scientific background and rationale for the investigation being reported Objectives State specific objectives, including any prespecified hypotheses Methods Study design Present key elements of study design early in the paper Setting Describe the setting, locations, and relevant dates, including periods of recruitment, exposure, follow-up, and data collection Participants (a) Give the eligibility criteria, and the sources and methods of selection of participants Variables Clearly define all outcomes, exposures, predictors, potential confounders, and effect Data sources/ measurement modifiers. Give diagnostic criteria, if applicable * For each variable of interest, give sources of data and details of methods of assessment (measurement). Describe comparability of assessment methods if there is more than one group Bias Describe any efforts to address potential sources of bias Study size Explain how the study size was arrived at Quantitative variables Explain how quantitative variables were handled in the analyses. If applicable, describe which groupings were chosen and why Statistical methods (a) Describe all statistical methods, including those used to control for confounding Results (b) Describe any methods used to examine subgroups and interactions (c) Explain how missing data were addressed (d) If applicable, describe analytical methods taking account of sampling strategy (e) Describe any sensitivity analyses Participants * (a) Report numbers of individuals at each stage of study eg numbers potentially eligible, examined for eligibility, confirmed eligible, included in the study, completing follow-up, and analysed (b) Give reasons for non-participation at each stage (c) Consider use of a flow diagram Descriptive data * (a) Give characteristics of study participants (eg demographic, clinical, social) and information on exposures and potential confounders (b) Indicate number of participants with missing data for each variable of interest Outcome data * Report numbers of outcome events or summary measures Main results (a) Give unadjusted estimates and, if applicable, confounder-adjusted estimates and their precision (eg, % confidence interval). Make clear which confounders were adjusted for and why they were included (b) Report category boundaries when continuous variables were categorized (c) If relevant, consider translating estimates of relative risk into absolute risk for a meaningful time period Other analyses Report other analyses done eg analyses of subgroups and interactions, and sensitivity analyses : first published as./bmjopen-0-00 on July 0. Downloaded from on November 0 by guest. Protected by copyright.

63 Page 0 of Discussion Key results Summarise key results with reference to study objectives Limitations Discuss limitations of the study, taking into account sources of potential bias or imprecision. Discuss both direction and magnitude of any potential bias Interpretation 0 Give a cautious overall interpretation of results considering objectives, limitations, multiplicity of analyses, results from similar studies, and other relevant evidence Generalisability Discuss the generalisability (external validity) of the study results Other information Funding Give the source of funding and the role of the funders for the present study and, if *Give information separately for exposed and unexposed groups. applicable, for the original study on which the present article is based Note: An Explanation and Elaboration article discusses each checklist item and gives methodological background and published examples of transparent reporting. The STROBE checklist is best used in conjunction with this article (freely available on the Web sites of PLoS Medicine at Annals of Internal Medicine at and Epidemiology at Information on the STROBE Initiative is available at : first published as./bmjopen-0-00 on July 0. Downloaded from on November 0 by guest. Protected by copyright.

64 Prospective Registration, Bias Risk and Outcome-Reporting Bias in Randomised Clinical Trials of Traditional Chinese Medicine: An Empirical Methodological Study Journal: Manuscript ID: bmjopen-0-00.r Article Type: Research Date Submitted by the Author: -May-0 Complete List of Authors: Liu, Jian-Ping; Beijing University of Chinese Medicine, Centre for Evidence- Based Chinese Medicine Han, Mei; Beijing University of Chinese Medicine, Centre for Evidence- Based Chinese Medicine Li, Xin-Xue; Beijing University of Chinese Medicine, Centre for Evidence- Based Chinese Medicine Mu, Yu_Jie; Beijing University of Chinese Medicine, Centre for Evidence- Based Chinese Medicine Lewith, George; University of Southampton, Centre for Complementary Medicine Research Wang, Yu-Yi; Beijing University of Chinese Medicine, Centre for Evidence- Based Chinese Medicine Witt, Claudia; Charité Universitätsmedizin Berlin, Institute for Social Medicine, Epidemiology and Health Economics Yang, Guo-Yan; Beijing University of Chinese Medicine, Centre for Evidence-Based Chinese Medicine Manheimer, Eric; University of Maryland School of Medicine, Center for Integrative Medicine Snellingen, Torkel; Beijing University of Chinese Medicine, Centre for Evidence-Based Chinese Medicine Berman, Brian; University of Maryland School of Medicine, Center for Integrative Medicine Gluud, Christian; Copenhagen University, Copenhagen Trial Unit <b>primary Subject Heading</b>: Complementary medicine Secondary Subject Heading: Complementary medicine, Epidemiology, Research methods Keywords: COMPLEMENTARY MEDICINE, traditional Chinese medicine, clinical trials, registration, bias risk, methodological study : first published as./bmjopen-0-00 on July 0. Downloaded from on November 0 by guest. Protected by copyright.