Kasenda B, Schandelmaier S, Sun X,

Transcription

1 Subgroup analyses in randomised controlled trials: a comparison of trial protocols and journal publications Kasenda B, Schandelmaier S, Sun X, von Elm E, Akl EA, Guyatt GH, Briel M on behalf of the DISCO study group

2 Background Randomized controlled trials (RCTs) primarily designed to investigate efficacy of health care interventions Trial populations often heterogeneous regarding patient or disease characteristics Subgroup analysis in RCTs common (40-60%) Subgroup findings often questionable and untrustworthy Criteria to evaluate credibility of reported subgroup analysis exist 1 especially pre specification of subgroups Empirical evidence about reporting of subgroup credibility criteria have yet only been based on published data 2 1 Sun et al BMJ Sun et al BMJ 2012

3 Background Users should look for explicit statements of a priori specification of subgroup hypothesis and subgroup direction in the primary study reports. In view of emerging evidence of differences between protocols and study reports, statements about what was included in registered or publicly available protocols finalized before the study or systematic review are desirable. 1 1 Sun X et al, Is a subgroup effect believable? Updating criteria to evaluate the credibility of subgroup analyses. BMJ 2010

4 Study Aims - Planning of subgroup analyses in protocols of randomized trials - Reporting of subgroup analysese in subsequent full journal publications - Agreement between planning and reporting of subgroup analyses

5 Design Identification of eligible RCT protocols at 6 RECs ( ) Electronically, surveys Identification of corresponding full journal publications Data extraction about planning of subgroup analysis Data extraction about reporting of subgroup analysis Analysis of agreement between planning and reporting of subgroup analyses Definition of subgroup Subset of an overall trial population determined by baseline characteristics at randomisation. Definition of subgroup analysis An analysis that explored whether treatment effects differed according to subgroup status.

6 Results Sample selection 3819 protocols screened 2411 no RCT design 1408 protocols reviewed in detail 192 duplicates 87 no compl protocol 41 no approval 8 no RCT design 1080 approved RCT protocols 53 never started 10 still ongoing 1017 RCT protocols included 123 included healthy volunteers 894 RCTs enrolling patients 515 (58%) full journal publications

7 Results Analysis sets 894 RCTs protocols 515 full journal publications

8 Results Planning of subgroups in protocols Sample size Subgroup analyses* Not planned N=642 (72%) Planned N=252(28%) All N=894 Median (IQR) 200 (80, 460) 521 (230, 1007) 260 (100, 606) Centre status Multi centre 500 (77.9%) 241 (95.6%) 741 (82.9%) Single centre 139 (21.7%) 10 (4%) 149 (16.7%) Unclear 3 (0.5%) 1 (0.4%) 4 (0.4%) Funding Non-industry 286 (44.5%) 57 (22.7%) 356 (39.8%) Industry 356 (55.5%) 195 (77.4%) 538 (60.2%) *In two protocols, extracted information about SG analyses was missing

9 Results Planning of subgroups in protocols Industry N=195 Funding Non-industry N=57 All N=251 Clear SG hypothesis given? 7 (3.6%) 10 (17.5%) 17 (6.8%) Direction of anticipated SG effect given? 3 (1.5%) 7 (12.3%) 10 (4.0%) Interaction term planned? 69 (35.6%) 18 (31.6%) 87 (34.7%) Median number of planned SG analyses* 3 (IQR, 1-6) 3 (IQR, 1-6) 3 (IQR, 1-4) * In 29 cases no information was given about the number of planned subgroup analysis

10 Results Reporting of subgroups in publications Sample size Subgroup analyses Not reported N=269 (52%) Reported N=246 (48%) All N=515 Median (IQR) 176 (68, 452) 572 (245, 1253) 347 (106, 800) Centre status Multi centre 217 (80.7%) 229 (93.1%) 446 (86.6%) Single centre 44 (16.4%) 12 (4.9%) 56 (10.9%) Unclear 8 (3.0%) 5 (2.0%) 13 (2.5%) Funding Non-industry 106 (39.4%) 86 (35%) 192 (37.3%) Industry 163 (60.6%) 160 (65%) 323 (62.7%)

11 Results Reporting of subgroups in publications Industry N=160 (65%) Funding Non-industry N=86 (35%) All N=246 Clear SG hypothesis reported? 11 (6.9%) 10 (11.6%) 21 (8.5%) Direction of anticipated SG effect given? 5 (3.1%) 6 (7.0%) 11 (4.5%) Interaction term used? 60 (37.5%) 36 (41.9%) 96 (39.0%) Median number of reported SG analyses* 4 (IQR, 1-8) 4 (IQR, 2-8) 4 (IQR, 2-8)

12 Results - Agreement between protocols and publications Of those 515 trial publications, 132 (25.6%) reported the conduct of subgroup analyses that were not mentioned in the corresponding protocols 64 (12.4%) did not report subgroup analyses that were planned in the corresponding protocols.

13 Results How often are reported subgroups pre-specified? Given the publication reported at least one subgroup analysis (N=246) At least one subgroup planned in protocol ALL yes 46% 114 no 54% Frequency

14 Results How often are reported subgroups pre specified? Given the publication reported at least one subgroup analysis (N=82) NON-INDUSTRY Given the publication reported at least one subgroup analysis (N=164) INDUSTRY At least one subgroup planned in protocol NON INDUSTRY At least one subgroup planned in protocol INDUSTRY yes 32% 26 no 46% 76 no 68% 56 yes 54% Frequency Frequency

15 Results Can we trust statements about subgroup pre specification? Given the authors stated that at least one reported subgroup was pre specified (N=81) At least one subgroup planned in protocol ALL no 35% 28 yes 65% Frequency

16 Limitations Limited quality of RCT protocols No detailed information about the subgroup defining variables were collected - conservative approximation Protocols were about 10 years old things might have improved No access to separate statistical analysis plans

17 Conclusions Quality of subgroup planning and reporting is low Over 50% of reported subgroup analyses not mentioned in protocol Industry funded trials more often pre-specify subgroup analyses One third of reported subgroup pre specifications in the publication could not be verified

18 DISCO - team Participating RECs Basel, CH Freiburg, GER Hamilton, CAN Lausanne, CH Lucerne, CH Zurich, CH DISCO Core-team Benjamin Kasenda Stefan Schandelmaier Erik von Elm Anette Blümle John You Theresa Bengough Yuki Tomonaga Mihaela Stegert Ramon Saccilotto DISCO investigators Elie A. Akl Alain Amstutz Dirk Bassler Bernard Burnand Jason W. Busse Alonso Carrasco-Labra Shanil Ebrahim Markus Faulhaber Ignacio Ferreira-Gonzalez Viktoria Gloy Lars Hemkens Bradley C. Johnston Francois Lamontagne Joerg J Meerpohl Funding Dominik Mertz Lorenzo Moja Sohail Mulla Ignacio Neumann Alain Nordmann Kelechi Kalu Olu Rachel Rosenthal Matthias Schwenkglenks Xin Sun Kari A.O. Tikkinen Per Olav Vandvik Martin A. Walter Heiner Bucher Gordon Guyatt