The Role and Importance of Clinical Trial Registries and Results Databases

The Role and Importance of Clinical Trial Registries and Results Databases March 2014 Rebecca J. Williams, Pharm.D., MPH Assistant Director, ClinicalTrials.gov National Library of Medicine, National Institutes of Health

Outline Public health view of clinical trial disclosure Key U.S. disclosure policies and laws Issues in reporting Uses of public registries Emerging issues in clinical trial disclosure 2

Why Conduct Clinical Trials? To obtain new and generalizable knowledge To contribute to the evidence base to inform medical decision making 3

How It s Supposed to Work Clinical Trials Health Outcomes Journals FDA Reviews Systematic Reviews Meta-analyses Guidelines Informed Decisions 4

Evidence Based Medicine Clinical and policy decisions should be informed by evidence regarding the benefits, risks, and other burdens associated with all possible alternatives Clinical trials are a key component of the body of scientific evidence that must be used to make decisions Most decision makers depend on summary data from journal publications 5

Publications: Three Key Issues Not all trials are published in the literature Publications do not always include all prespecified outcome measures or complete adverse event information Unacknowledged changes made to the trial protocol that would affect the interpretation of the findings e.g., changes to the pre-specified outcome measures 6

7

Figure 1. Effect of FDA Regulatory Decisions on Publication Turner EH et al. N Engl J Med 2008;358:252-60. 8

9

Summary of Findings Fewer than half of NIH funded trials registered at ClinicalTrials.gov after September 2005 and completed by December 2008 were published in a peer reviewed biomedical journal indexed by MEDLINE within 30 months of trial completion After a median of 51 months after study completion, a third of NIH-funded trials in the sample remained unpublished Ross JS et al. BMJ 2012;344:d7292. 10

Kaplan-Meier estimates for ulcer complications according to traditional definition. Results are truncated after 12 months, no ulcer complications occurred after this period. Adapted from Lu 2001. Jüni P, Rutjes AW, Dieppe PA. BMJ. 2002;324(7349):1287-8. 11

Internal Corporate Email They swallowed our story, hook, line and sinker 12 Thomas K. NY Times. June 24, 2012. 12

Publications: Three Key Issues Not all trials are published in the literature Publications do not always include all prespecified outcome measures or complete adverse event information Unacknowledged changes made to the trial protocol that would affect the interpretation of the findings e.g., changes to the pre-specified outcome measures 13

Impact of Issues Incomplete disclosure of research results impedes scientific process Lack of availability of clinical trial data is particularly problematic: Trials depend on human volunteers Trial results inform our medical decisions 14 14

What Methods are Available to Counter Such Issues?

Clinical Trial Disclosure Zarin DA, Tse T. Science. 2008. 16 16

Definitions Registration: the process for making key summary information about interventional studies using human volunteers accessible to the public via a web-based system, from study initiation to completion Results Reporting: making summary information about study results available in a structured, publicly accessible web-based results database 17

Rationale for Trial Disclosure Simes (1986) - Call for systematic registration to mitigate publication bias Declaration of Helsinki (2008; 2013 revised) Article 35: Registration of all research studies involving human subjects Article 36: Disclosure of negative and inconclusive as well as positive results World Health Organization: The registration of all interventional studies is a scientific, ethical, and moral responsibility. 18

Rationale for Trial Disclosure Category Human subject protections Research integrity Evidence-based medicine Allocation of resources Reasons Allow potential participants to find studies Assist ethical review boards and others to determine appropriateness of studies being reviewed (e.g., harms, benefits, redundancy) Promote fulfillment of ethical responsibility to human volunteers research contributes to medical knowledge Facilitates tracking of protocol changes Enhances transparency of research enterprise Facilitates tracking of studies and outcome measures Allows more complete identification of relevant studies Promotes more efficient allocation of resources (e.g., investigators, institutional review boards [IRBs], volunteers, funders) 19

History of ClinicalTrials.gov 1997 - FDA Modernization Act (FDAMA) IND trials of serious and life-threatening conditions 2000 ClinicalTrials.gov launched Calls for increased transparency of clinical trials State Attorneys General; Maine state law 2005 - International Committee of Medical Journal Editors (ICMJE)/World Health Organization (WHO) 2007 - FDA Amendments Act (FDAAA) Expands registration scope & adds results reporting 2008 Declaration of Helsinki (updated 2013) IND = investigational new drug application http://clinicaltrials.gov/about-site/history 20

Number of Registered Studies Over Time (Data as of February 27, 2014) ~ 400 per week ~25-30 new records per week ~200-250 per week http://clinicaltrials.gov/ct2/resources/trends 21

Two Key Disclosure Policies ICMJE Journal Editors (2005) Prospective registration of all clinical trials as a condition for publication of the study results FDA Amendments Act, Section 801 (2007) Expanded trial registration requirements (FDAMA 113) to include trials of devices and additional trials of drugs Added new results submission requirement Added enforcement provisions Civil monetary penalties (up to $10,000/day) Withholding of NIH grant funds Current status: Rulemaking pending 22

Why? Which Trials? ICMJE Required for journal publication Interventional Studies - All Phases - All Intervention Types FDAAA Required by US federal law Interventional Studies - Not Phase 1/Feasibility - Drugs, Biologics, Devices Who? Author Sponsor or designated PI When to Register? Prior to enrollment of first participant Within 21 days of enrollment of first participant What to Register? WHO Data Items ClinicalTrials.gov and FDAAA Data Elements Where to Register? When to Submit Results? ClinicalTrials.gov or WHO Primary Registry Not Applicable 23 ClinicalTrials.gov Within 12 months of final data collection for the primary outcome

Interventional vs. Observational Interventional Study ( Clinical Trial ) Participants assigned to receive one or more (or no) interventions based on a protocol Observational Study Participants identified as belonging to study groups, not assigned by researcher [Currently excluded from most registration policies, but increasing attention to this topic] Note: Many Diagnostic studies are interventional

Scope of Registry ClinicalTrials.gov permits the registration of any biomedical or health-related research studies in humans that meet the following two requirements: 1.Conformance with any applicable human subject protections or ethics review regulations (or equivalent) (e.g., institutional review board (IRB) approval) AND 2.Conformance with any applicable regulations of the national (or regional) health authority (or equivalent)

ClinicalTrials.gov Home Page http://clinicaltrials.gov 26

ClinicalTrials.gov Statistics (as of 2/27/2014) 98 million pages views monthly 64,000 unique visitors daily Registration Results Total 162,059 11,513 Type of Trial Observational 30,272 (18%) 740 (6%) Interventional* 131,037 (80%) 10,773 (93%) - Drug & Biologic 85,890 9.087 - Behavioral, Other 33,749 1,346 - Surgical Procedure 14,520 520 -Device** 12,204 1,052 * Intervention types not additive; study record may include more than one type of intervention **518 applicable device clinical trials submitted, but qualify for delayed posting under FDAAA

Content of ClinicalTrials.gov Records One record per trial Registration section Submitted at trial initiation Summarizes trial protocol Condition Interventions Design, etc Includes recruitment information (e.g., eligibility, locations) Results section Submitted after trial completion Summarizes trial results Participant Flow Baseline Characteristics Outcome Measures (including statistical analyses) Adverse Events 28

Public Archive for Records Changes can and should be made to records Estimated dates become actual dates Estimated enrollment becomes actual Other protocol changes Overall recruitment status changes Results may be added or changed All changes are publicly tracked 29

Stages in Disclosure Parallel the Research Life Cycle 30

Results Database Objectives Satisfy legal requirements Promote objective, standardized reporting Facilitate good reporting practices, including publishing (e.g., CONSORT) and regulatory guidelines Provide structured data entry to ensure complete reporting, efficient quality review, and consistent display of data elements Support detailed searches with the use of database structure and other NLM functions Adapted from Table 2 in Zarin DA et al. N Engl J Med 2011;852-60. 31

Sample Publication and Results Posted at ClinicalTrials.gov* *Adapted from NCT00145249 32

33 33

34 34

35

CONSORT Flow Diagram 2010 by British Medical Journal Publishing Group Shulz KF, Altman DG, Moher D. BMJ 2010;340:698-702.

CONSORT Flow Diagram ( Figure 1 ) Pappas PG, Chetchotisakd P, Larsen RA et al. Clin Infect Dis 2009;48:1775-83. 37

ClinicalTrials.gov Format Participant Flow: Overall Study AmphoB Standard AmphoB + Fluc400 AmphoB + Fluc800 STARTED 47 48 48 Treated 45 47 [1] 49 [2] Day 14 visit 46 44 42 70 days therapy 35 32 32 COMPLETED 36 [3] 33 [3] 31 [3] NOT COMPLETED 11 15 17 Study Ineligible 0 0 3 Missed Day 112 Follow-up 11 15 14 [1] 2 subjects randomized to AmphoB received AmphoB+Fluc400 [2] 3 subjects randomized to AmphoB+Fluc400 received AmphoB+Fluc800 [3] Completed Day 112 Follow-up Adapted from NCT00145249 38

Baseline Data Journal Article Format ( Table 1 ) Pappas PG, Chetchotisakd P, Larsen RA et al. Clin Infect Dis 2009;48:1775-83. 2009 Infectious Diseases Society of America 39

ClinicalTrials.gov Format NCT00145249 40

Outcome Measures Journal Article Format Pappas PG, Chetchotisakd P, Larsen RA et al. Clin Infect Dis 2009;48:1775-83. 2009 Infectious Diseases Society of America 41

ClinicalTrials.gov Format NCT00145249 42

ClinicalTrials.gov Format (cont.) NCT00145249 43

ClinicalTrials.gov Format NCT00145249 44

ClinicalTrials.gov Format NCT00145249 45

Clarifications about Results Reporting Requirements Does NOT prescribe how study should be conducted Summary results at the end of the trial No interim or real time reporting No participant level reporting Information currently targeted at readers of the medical literature Tables of information; just the facts No conclusions or discussion 46

Registration, Results Submission and Publication Deadline for submitting results to ClinicalTrials.gov is independent of publication status Submitting results to ClinicalTrials.gov will not interfere with publication* Failure to register WILL interfere with publication! ClinicalTrials.gov records are linked, via NCT number, to publications Ensure the registration record is up-to-date ~ 50% of ClinicalTrials.gov results not initially available elsewhere (Zarin et al. N Engl J Med 2011: 852-860.) *Laine C et al. Ann Intern Med. 2007; http://www.icmje.org/faq_clinical.html 47

Linking Publications to NCT Numbers 48

Issues in reporting 49

Outcome Measures, in theory The primary outcome measure is pre-specified as part of the protocol Basis for the power calculation/determining sample size Pre-specification is key to the conduct of statistical analysis The primary outcome measure is registered in a public database at trial initiation 50

Outcome Measures, in practice Studies with results in ClinicalTrials.gov (n=2,178) Primary Outcome Measures Median: 1 Interquartile range: 1 2 Full range: 1 71 Secondary Outcome Measures Median: 3 Interquartile range: 1 7 Full range: 0-122 Zarin DA et al. N Engl J Med 2011;364: 852-60. 51

Outcomes Conceptual Framework N Engl J Med 2011;364:852-60.

SPIRIT 2013 Statement Item 12 Outcomes Primary, Secondary, Other Specific measurement variable (e.g., systolic blood pressure) Analysis metric (e.g., change from baseline, final value, time to event) Method of aggregation (e.g., median, proportion) Time point Chan A et al. Ann Intern Med. 2013;158:200-207. 53

Level of Outcome Measure Specification in Registration Primary Outcome Measures registered in ClinicalTrials.gov Level Primary Outcome Measures (% Total); n=100 1 Domain only 36% 2 Specific Measurement 25% 3 Specific Metric 26% 4 Method of Aggregation 13% Specific Time Frame 72% Zarin DA et al. N Engl J Med 2011;364: 852-60. 54

Experience with Results Database There is no substitute for a well-written protocol (and registration record) It is never too early to plan for results reporting! 55 55

General Review Criteria Protocol and results must be clear and informative Review focuses on: Logic and internal consistency Apparent validity Meaningful entries Formatting, including appropriate use of database structure Not equivalent to peer review; not verified against external sources (e.g., full protocol) Review Criteria: http://clinicaltrials.gov/ct2/manage-recs/resources 56

Problems with Results Entries Time to survival 823 hours of sleep/day Time to response: 12 participants Number of participants changes dramatically between modules 57

Outcome Measure - Example 1. Primary Outcome Measure: Measure Title Measure Description Time Frame Safety Issue? Percent Change in Hemoglobin A1c A1C will be measured at baseline and after 12 and 24 weeks of treatment. 24 weeks Yes Measured Values Remuverol Placebo Number of Participants Analyzed 60 59 Percent Change in Hemoglobin A1c [units: participants %] Least Squares Mean ± Standard Error -0.71 ± 0.12-0.94 ± 0.12

Outcome Measure - Example 1. Primary Outcome Measure: Measure Title Measure Description Time Frame Safety Issue? Percent Reduction in Percentage of Migraine Days per 30 Days Change in the percentage of migraine days per 30 days relative to the OAT run-in (Month 1). Assessed by participant electronic diary. 24 weeks Yes Measured Values Hypertena Placebo Number of Participants Analyzed 53 69 Percent Reduction in Percentage of Migraine Days per 30 Days [units: percent] Mean ± Standard Deviation 15.0 ± 5.2-20.3 ± 6.8

Uses of ClinicalTrials.gov 60

Basic Uses of ClinicalTrials.gov Identify trials of potential interest for an individual Including to specific user communities such as: Breast cancer (e.g., http://www.breastcancertrials.org) Track progress of a specific trial, including availability of summary results Identify all trials that are completed or ongoing for a specific set of conditions/interventions Complement to literature review Useful in planning stages of a new protocol Identify investigators and/or research centers of relevance to a specific set of conditions/ interventions 61

Trials in Acute Kidney Injury (AKI) 126 ongoing trials registered in ClinicalTrials.gov 65% (n=82) prevention trials Appropriate Sample Size with Adequate Power? Accurate estimate of incidence of AKI in group studied Realistic estimate of the effect of the intervention Outcome: Number of Participants with AKI with need for RRT (Renal Replacement Therapy) Estimated that 822 participants per arm needed Only 3 of 28 contrast studies and 3 of 30 cardiac surgery studies had more than 800 participants TOTAL Clin J Am Soc Nephrol. 2012;7:861-873. 62

Other Uses of ClinicalTrials.gov Examination of the Clinical Research Enterprise Clinical Trials Transformation Initiative (CTTI) Creating and maintaining a research data set for aggregate analysis of ClinicalTrials.gov Initial publication in JAMA in May 2012 Clinical specialty analyses also published: oncology, cardiovascular, and peripheral vascular disease Assess quality and completeness of reporting Fidelity of reports to research protocol Publication bias http://www.ctti-clinicaltrials.org 63

Conclusion: Clinical trials registered in ClinicalTrials.gov are dominated by small trials and contain significant heterogeneity in methodological approaches, including reported use of randomization, blinding, and DMCs. JAMA. 2012;307(17):1838-1847 64

Conclusions Clinical trials are critical for advancing knowledge and informing medical decisions Clinical trials require careful planning, use of clinically relevant outcomes, and appropriately trained personnel to ensure good conduct Results disclosure needs to be considered in the planning stages of a protocol What will be collected? How will it be analyzed and summarized? Who is responsible? 65

Thank you! Acknowledgement: Deborah Zarin and Tony Tse for permission to use their slide content 66

Additional Background Califf RM, Zarin DA, Kramer JM, Sherman RE, Aberle LH, Tasneem A. Characteristics of Clinical Trials Registered in ClinicalTrials.gov, 2007-2010. JAMA. 2012 May 2;307(17):1838-1847. Ross JS, Tse T, Zarin DA, Xu H, Zhou L, Krumholz HM. Publication of NIH funded trials registered in ClinicalTrials.gov: cross-sectional analysis. BMJ 2012;344:d7292. Zarin DA, Tse T, Williams RJ, Califf RM, Ide NC. The ClinicalTrials.gov results database--update and key issues. N Engl J Med. 2011 Mar 3;364(9):852-60. Tse T, Williams RJ, Zarin DA. Reporting basic results in ClinicalTrials.gov. Chest 2009;136:295-303. 67

68 Email: register@clinicaltrials.gov

Case Study Outcome Measures Ann Intern Med. 2013;159:65-67. 69

NCT00918255 - Initial and Updated Entries for Primary Outcome Measures Level of Specification Domain (e.g., anxiety ) Specific measurement (e.g., Hamilton Anxiety Rating Scale ) Specific metric (e.g., change from baseline ) Method of aggregation (e.g., proportion of participants with decrease 50% ) Time frame (e.g., 12 weeks ) Initial ClinicalTrials.gov Entry Hidradenitis suppurativa PGA (Physician s Global Assessment) N/A N/A Updated ClinicalTrials.gov Entry Hidradenitis suppurativa PGA Change from baseline Percentage of participants achieving a clinical response, defined as a PGA score of clear, minimal, or mild with at least a 2-grade reduction indicating improvement Week 16 Baseline, week 16 Zarin DA, Tse T. Ann Intern Med. 2013 Jul 2;159(1):65 7. 70

NCT00136318 - Initial and Updated Entries for Primary Outcome Measures Level of Specification Domain (e.g., anxiety ) Specific measurement (e.g., Hamilton Anxiety Rating Scale ) Specific metric (e.g., change from baseline ) Method of aggregation (e.g., proportion of participants with decrease 50% ) Time frame (e.g., 12 weeks ) Initial ClinicalTrials.gov Entry Depression HAM-D (Hamilton Depression Rating Scale) N/A N/A Zarin DA, Tse T. Ann Intern Med. 2013 Jul 2;159(1):65 7. Updated ClinicalTrials.gov Entry Depression MADRS (Montgomery Asberg Depression Rating Scale) MADRS score 13 during time frame Percentage of participants with specific metric 24 weeks 50 weeks after receiving intervention for participants with HCV genotype 1 or 4 OR 26 weeks after receiving intervention for patients with HCV genotype 2 or 3 71