RARE-Bestpractices Conference 24 November 2016, Istituto Superiore di Sanità, Rome, Italy Avoiding waste in production, design and conduct of studies and reporting evidence on rare diseases Philippe Ravaud 1
Lack of data sharing? Individual participant data meta-analysis? Personalized medicine? Subgroups analysis? Prognosis studies? Research waste 2
Avoiding the waste of research Good Research Question? yes No Relevant Methods? Yes No yes Publication? Adequate Reporting? No 85% of research is wasted Yes No 1. Chalmers et al, 2009, Lancet 3
Waste of research related to bias and/or poor reporting 205 Cochrane reviews included 1286 trials included in the metaanalysis for the main outcome Yordanov, BMJ 2015 207 (16%) trials at low risk of bias 523 (41%) trials at unclear risk of bias Waste of research related to poor reporting 556 (43%) trials at high risk of bias Waste of research related to poor planning Most of trials with high risk of bias can be improved by simple and cheap changes in methods. 4
Waste of Research in clinical trials Waste can occur at all stages of research Planning Conducting Analysis Reporting Archiving and Datasharing 5
Waste of Research Planning Phase Bad research question (a question impossible to answer) Reinventing the wheel or lagging far behind other researchers A question that has already been answered by others (or is going to be soon answered) Trials related to this question are already completed and going to be published 6
Visiting of CT.gov is not optional but mandatory 81 for Retinoblastoma 7
Waste of Research Planning Phase Inadequate sample size Badly selected study population (selection criteria are too restrictive or too broad, ) Inappropriate follow-up time (too short) Imprecise interventions (e.g., that could not be reproduced mainly for non pharmacological treatments) 8
Waste of Research Planning Phase Inadequate methods leading to trials with high risk of bias (blinding...) Inappropriate endpoints (surrogate endpoints, different from those included in the core outcome set relevant for the studied disease (COMET Initiative)) 9
Waste of Research in clinical trials Planning Conducting Analysis Reporting Archiving and Datasharing 10
Waste of Research Conducting Phase Overestimating recruitment capacities, which leads to unfeasible trials Waste of resources (time, money) Too much data collected Unnecessary heavy monitoring 11
Waste of Research in clinical trials Planning Conducting Analysis Reporting and posting Archiving and Datasharing 12
Authors, editors and publishers all have ethical obligations with regard to the publication of the results of research. Authors have a duty to make publicly available the results of their research on human subjects and are accountable for the completeness and accuracy of their reports. 13
Waste of Research at Reporting Stage No publication at all Publication as an abstract only 50 % of abstracts are not published Delayed publication Poor (incomplete) reporting: - methods - results (selective outcome reporting) Unfair reporting (SPIN) 14
FDA Amendment Act US Federal law enacted in 2007 mandates registration and results reporting for clinical trials of drugs, biological products and devices at clinicaltrial.gov Study sponsors or PI are required to report summary results information within 1 year of completing data collection for the prespecified primary outcome Same regulation in EU taking effect in 2017 15
22 22,233 studies with results 16
Availability of Results of Trials on Rare Diseases Registered at Clinicaltrials.gov 1,531 rare diseases with at least 1 study registered at CT.gov completed between January 2009 and December 2013 802 completed phase 3 or 4 trials assessing drugs 69% were RCTs 62% were single country trials 39 % were single center trials For each trial we systematically assessed wether results were published and/or posted on CT.gov Dechartres et al, JAMA internal medicine 2016 17
Availability of Results of Trials on Rare Diseases Registered at Clinicaltrials.gov At 36 months, results were posted at clinicaltrials.gov for 35% of trials (95% CI, 32%-38%), published in journals for 47% (95% CI, 43%-51%), Overall, results were publicly available only for 63% (95% CI, 59%-66%). Among the 325 trials likely to be subject to the FDAAA, 220 (68%) had results posted at clinicaltrials.gov: 4% (95% CI, 2%- 6%) had results posted at 12 months 57% (95% CI, 51%- 63%) at 36 months. 18
Waste of Research in clinical trials Planning Conducting Analysis Reporting Archiving and Datasharing 19
Improving Data-Sharing Clinical data are too precious to be analyzed Only once With potentially inappropriate methods and strategic choices of analysis more or less biased By only one team Without being checked by another team Without being able to pool them with other data An inevitable evolution towards massive, systematic, and imposed data sharing 20
Conclusion Waste of Research is extremely common for any diseases including rare diseases Waste of Research is largely avoidable Without a collective awakening and effort (clinicians, trialists, methodologists, funders, editors, reviewers) this waste will perpetuate 21
http://www.equator-network.org/ http://rewardalliance.net/ 22