CSG-COUSIN The Cochrane Skin Group Core Outcome Set Initiative a critical reflection. Background. History

Transcription

1 Background "Our mission is to develop and implement core outcomes sets in dermatology in order to improve and standardise outcome measurement in clinical trials and to make trial evidence more useful." CSG-COUSIN The Cochrane Skin Group Core Outcome Set Initiative a critical reflection Annual Cochrane Skin Group & CSG-COUSIN Meeting 2017 Berlin, 9th January I Usefulness of clinical trial evidence limited due to non-comparable outcome measures and instruments I Choice of outcome domains often unclear I Measurement instruments heterogeneous within same domain I Performance of measurement instruments unclear or inadequate 2 Core Outcome Set (COS) is minimum set of outcomes to be assessed and reported in clinical trial Outcome Assessment in Clinical Trials I To measure WHAT matters I The relevant OUTCOME DOMAINS I To measure with adequate INSTRUMENTS I Validity, Reliability, Responsiveness, Sensitivity to change I To define a STANDARD for cross-trial comparison I CORE SET to be used in all trials History Cochrane Skin Group Core Outcome Set Initiative CSG-COUSIN I was initiated by Prof. Jochen Schmitt and Prof. Hywel C. Williams in 2014 I The CSG-COUSIN inaugural meeting was in March 2015 at the Annual Cochrane Skin Group Meeting 29 participants with diverse professional backgrounds and perspectives Introduction CSG-COUSIN Exchange of ideas Development of future plans Meeting report published in BJD 3 4 Structure Management Team I Lead Prof. Dr. Jochen Schmitt, MPH I Coordination Center for Evidence-Based Healthcare University of Dresden, Germany I Support Cochrane Skin Group, Nottingham I Teams International Multidisciplinary Open for everyone I Coordination of CSG-COUSIN I First point of contact, organization of regular meetings I Provision of resources, e.g. articles, news I Newsletter, Homepage, Creation of visibility and awareness I Support of Project Teams with Software Tools I Support of publications Management Team 5 6 1

2 1. & 2. CSG-COUSIN Newsletter CSG-COUSIN Homepage e.g. Information Management WebApp for Delphi-studies I Online surveys with feedback of previous rounds Individual feedback Group feedback Meeting Report BJD 2. Meeting CSG-COUSIN I Tool may be used by project groups upon request Poster & Presentations CSG & COMET Newsletter 7 8 Project Groups: the heart of CSG-COUSIN I Development and implementation of Core Outcome Sets I Group composition Lead Patient representative Experts Methodologists Clinicians Others I Liaison person from methods group Project Groups Project Groups CURRENT CSG-COUSIN COS PROJECT GROUPS Acne vulgaris Atopic Eczema Chronic Spontaneous Urticaria Chronic wounds Incontinence-Associated Dermatitis Hidradenitis suppurativa Melanoma Nail psoriasis Pressure Ulcer Trials Rosacea Vascular malformations Vitiligo Appearance of Hand Eczema Facial Aging 9 10 Group Group Meta-Epidemiologic-Study I Provides Methodological Support Guidedance I Internal peer review for CSG-COUSIN project groups I Conducts methodological studies on outcomes research and COS development I Development of methodological standards based on HOME roadmap I Support of implementation of COS 11 Group Peer Review for project groups No standardized structure No communication rules No quality standards 12 2

3 few people with many ideas and many other obligations Project group Lead/Contact Person Team group intern group extern Acne vulgaris (ACORN) Alison Layton Anne Eady Jochen Schmitt Chronic Spontaneous Maulina Sharma, Karsten Weller Urticaria Jane Nixon, Chronic wounds Thomas Wild (für Anke Mayer Stefanie Deckert COUSIN) Hidradenitis John Ingram Jochen Schmitt suppurativa Incontinence- Dimitri Beeckman Associated Dermatitis Group: Challenges for CSG COUSIN mission: to produce and implement high quality COS - Process quality (standard procedures) important - Templates for project group registration, protocols, evidence synthesis, consensus processes, Melanoma Stefanie Deckert, Sanna Prinsen J Schmitt, PhI Spuls, R Stern, CB Terwee Stefanie Deckert, Sanna Prinsen - Standards for stakeholder selection, consensus critiera, Nail psoriasis Phyllis Spuls Phyllis Spuls Susanne Coleman, Pressure Ulcer Trials Katrin Balzer Delia Muir, Jan (OUTPUTs) Kottner, Jane Nixon Rosacea Murad Alam Jochen Schmitt Vascular malformations Chantal van der Horst, Sophie Horbach Phyllis Spuls (OVAMA) Phyllis Spuls Vitiligo Khaled Ezzedine Viktoria Eleftheriadou Viktoria Eleftheriadou Appearance of Facial Murad Alam Jochen Schmitt Aging (IMPROVED) the good ideas need to be realized, continuous effort is required - Internal peer review to be established - Filter to adopt a COS as CSG-COUSIN COS??? - At least 10 experienced and long-term enthusiastic people are needed that commit themselves to spend half a day every month for COUSIN this is also reflected by feedback from COS groups I Conceptual Framework/Approach Decision on methodological framework (HOME roadmap, COMET initiative, OMERACT, COSMIN) Determination of the methodological scope of the literature review and included studies (restrict to RCT s and systematic reviews, or broader) how to create a shortlist I Stakeholder engagement Selection Patient Involvement (children/icu/elderly/acute nature of condition) Approach (all stakeholders, e.g. industry an regulators), ensuring unbiased engagement & retention Structure this is also reflected by feedback from COS groups I Terminology/Definitions/Constructs/different Levels Use and definitions of terminology (outcome, measurement, instruments, domains, sub-domain, items, single construct) Different levels of outcomes/domains/constructs: broad vs. specific (e.g. QoL) Mix of these in COS? Definition of domains: when and by whom? I Funding sources Background I High relevance of standardising outcome measurement in clinical trials I Systematic overview from Stefanie Deckert All 64 CSG reviews (comprising 1566 trials), published until 01/2015 Aim: to systematically compare predefined outcomes in CSG reviews and the reporting of these outcomes in underlying trials in CSG reviews I Results CONCORDANCE BETWEEN PRIMARY AND SECONDARY OUTCOMES IN COCHRANE SYSTEMATIC REVIEWS AND UNDERLYING CLINICAL TRIALS IN DERMATOLOGY A META-EPIDEMIOLOGIC STUDY Annual Cochrane Skin Group & CSG-COUSIN Meeting 2017 Berlin, 9th January Figure: Schmitt J et al. (2016) Report from the kick-off meeting of the Cochrane Skin Group Core Outcome Set Initiative 18 (CSG-COUSIN). British Journal of Dermatology 174(2):

4 Objectives I The objectives of this study are: To systematically assess and critically evaluate the degree of concordance between primary and secondary outcomes in CSGreviews and underlying trials To compare treatment outcomes estimated by meta-analysis of trials with primary outcome concordant vs. discordant to the primary outcome of the systematic review The underlying hypotheses are: I Hypothesis 1: There is a low overlap between the primary and secondary outcomes in reviews and the outcomes in underlying trials. We assume that this discordance in outcomes is bidirectional, i.e. that many review outcomes are not assessed in underlying trials, and that many outcomes used in the trials are not addressed in the corresponding review. We further assume that a relevant proportion of review teams concluded that meta-analysis was not possible due to a high degree of heterogeneity in trial outcomes The underlying hypotheses are: I Hypothesis 2: We expect relevant differences between the pooled results of trials with a concordant vs discordant primary outcome. We assume that pooled results of discordant trials tend to yield stronger or weaker effects than pooled results of concordant trials. I Study design Meta-epidemiologic study I Data source and data collected A random sample of 10 systematic reviews out of all 64 CSG reviews of the Cochrane Skin Group published until 01/2015 The 10 CSG systematic reviews included 242 primary studies (range 7-85) Results exemplary and preliminary I Data extraction from reviews and trials Independently and in duplicate by two reviewers 6 review teams Christian Apfelbacher & Susanne Brandstetter Sanna Prinsen & Phyllis Spuls & Elisabeth Hahnel Tobias Weberschock & Andreas Dreher Jochen Schmitt & Lena Jacobi Matthew Grainge & Esther Burden-Teh All data accessible through Archie were used Via a specific developed software tool (based on Access) Reference Cochrane Review Bath- Hextall, F. J. et al., 2012 I Example Cochrane Review: Bath-Hextall, F. J. et al., 2012 Condition/ Interventio n under study atopic eczema Number of trials included in review Mention primary outcome in abstract 11 only in the Plain language summary Primary Outcome(s) Review Primary efficacy outcome I: (a) Short-term (within six weeks). Changes in participant-rated or parentrated symptoms of atopic eczema, such as pruritus (itching) or sleep loss. Primary efficacy outcome II: (b) Degree of long-term (over six months) control, such as reduction in number of flares or reduced need for other treatments. If meta-analysis was conducted (n of trials included / not included in meta-analysis ) no meta-analysis If no meta-analysis was conducted (Reasons) no data available for the primary outcomes (n=9/11) or only one study (n=2/11) Comment in Discussion concerning outcomes Future studies should use outcome measures that are more readily clinically interpretable and should be informed by the outcome of the Harmonizing Outcomes for Eczema (HOME) project (Schmitt 2010; Schmitt 2011)

5 Secondary review outcomes Secondary Outcomes: 1:Global severity as rated by the participants or their physician. If this outcome was not available then the following was used: (b) Global changes in composite rating scales using a published named scale. Where this was not possible, we used the following: (c) The author s modification of existing scales or new scales. 2:Quality of life 3:Palatability 4:Adverse events including long-term consequences on growth Results exemplary and preliminary I Additional outcomes Types of outcome (Primary author) Not specified: 230 [92.4%] Primary outcome: 10 [4%] Secondary outcome: 9 [3.6%] Number of different Measurement instruments outcomes were assessed in RCTs that were not considered relevant for Cochrane review - These 230 outcomes were assessed by 185 different instruments Critical Reflection Conclusions so far I No funding Few resources Long project length, delayed deadlines Big group I Conditions Technical problems/ high requirements Communication problems Misjudgement of the workload I The choice and definition of outcomes is critical For Cochrane Reviewers For Trial Authors I Cooperation between COUSIN and CSG needs to be strengthened and formalized I Cochrane Reviewers should be connected with COS groups I COUSIN should be engaged in CSG-protocols and reviews Thank you for your attention Address: University hospital Carl Gustav Carus an der TU Dresden Center for Evidence-based Healthcare Fetscherstraße 74, Dresden Contact: Prof. Dr. med. Jochen Schmitt, MPH jochen.schmitt@uniklinikum-dresden.de COUSIN@uniklinikum-dresden.de 5