School for Social Care Research

Transcription

1 School for Social Care Research MRC guidance on developing and evaluating complex interventions: Application to research on palliative and end of life care Catherine J Evans, Katie A Stone, Jill Manthorpe, Irene J Higginson Methods Review 15 Improving the evidence base for adult social care practice

2 The School for Social Care Research The School for Social Care Research is a partnership between the London School of Economics and Political Science, King's College London and the Universities of Kent, Manchester and York, and is part of the National Institute for Health Research (NIHR) The School was set up by the NIHR to develop and improve the evidence base for adult social care practice in England. It conducts and commissions high-quality research. About the authors Dr Catherine J Evans is an NIHR Clinical Lecturer in palliative care. This is a joint clinical academic post between the Cicely Saunders Institute, King s College London and Sussex Community NHS Trust. Catherine s research concerns research methods to develop and evaluate palliative care services and interventions. Professor Irene J Higginson is Head of Department for the Cicely Saunders Institute and Scientific Scientific Director of Cicely Saunders International, a charity seeking to develop research in palliative care to improve care for patients and families. She is an NIHR Senior Investigator. Jill Manthorpe is Professor of Social Work at King s College London and Director of the Social Care Workforce Research Unit. She is also Associate Director of the NIHR School for Social Care Research and an NIHR Senior Investigator. Katie Stone is a Research Assistant at the Cicely Saunders Institute, King s College London. Her background is in anthropology. Her research work has focused on health service research in the areas of palliative care, alcohol and drugs and nursing. NIHR School for Social Care Research London School of Economics and Political Science Houghton Street London WC2A 2AE sscr@lse.ac.uk Tel: +44 (0) Website: School for Social Care Research, 2013 ISBN This report presents an independent review commissioned by the NIHR School for Social Care Research. The views expressed in this publication are those of the authors and not necessarily those of the NIHR School for Social Care Research, the Department of Health, NIHR or NHS.

3 ABSTRACT This methods review examines the Medical Research Council s Guidance on Developing and Evaluating Complex Interventions (MRC GDECI) and its application to research on palliative and end of life care, and potential wider application to research on adult social care practice. The review is a resource for social care researchers, practitioners and managers, educationalists and students involved in developing and evaluating complex interventions, particularly new services or re-configurations of care and support for people at the end of life. The review examines the strengths and limitations of the MRC GDECI and offers recommendations for future use. The review draws on research case examples in particular the Methods of Researching End of Life Care (MORECare) project. This project developed the MORECare statement detailing best practice guidance for research evaluating end of life care to improve study quality and extend the existing MRC Guidance for research on palliative and end of life care. RECOMMENDATIONS FOR EVALUATING END OF LIFE CARE SERVICES IN SOCIAL CARE SETTINGS The MRC GDECI (Campbell et al. 2007, Craig et al. 2008) requires adaptation for research contexts beyond healthcare to encompass social care settings and the research challenges encountered. Social care offers a wide range of care and support. These include integrated interventions (spanning health and social care) with correspondingly diffuse goals of care and intended outcomes. Services are delivered across a multitude of settings to a diverse population group. Study designs developing and evaluating complex interventions in social care ought to consider using the MORECare statement. The statement provides 36 best practice solutions for research evaluating services and interventions in end of life care to improve study quality and set a standard for future research. The statement provides a first step in setting common and much needed standards for evaluative research in end of life care (Higginson et al. 2013), and these will have relevance in other social care settings. To accommodate the complexity of delivering and evaluating services and interventions in end of life care (and other social care contexts) requires a staged approach. This helps to build knowledge moving from development to definitive evaluation, with implementation as a continuous thread. Funders need to invest in researchers to help them develop research expertise in evaluative research methods and to build programmes of research to improve outcomes for users and carers. iii

4 KEYWORDS Complex interventions, evaluation studies, research design, research methods, social care, palliative care, end of life care, terminal care ACKNOWLEDGEMENTS Colleagues in many different parts of the social care sector provided assistance and guidance on this review, in particular suggesting areas to increase the relevance and applicability for social care. The following people contributed papers, references and guidance. We are grateful to them for their support and encouragement: Ann Netten (University of Kent) Margaret Holloway (University of Hull) Tes Smith (Macmillan Cancer Support) Martin Webber (King s College London/University of York) Donald Forrester (University of Bedfordshire) This review drew on the MORECare project funded by the NIHR and managed by the Medical Research Council (MRC) as part of its Methodology Research Programme (MRP) (project number: G /1). MORECare aimed to identify, appraise and synthesise best practice methods to develop and evaluate palliative and end of life care, particularly focusing on complex service-delivery interventions and reconfigurations. The principal investigator was Irene J Higginson and co-principal investigator Chris Todd. The coinvestigators were Peter Fayers, Gunn Grande, Richard Harding, Matthew Hotopf, Penney Lewis, Paul McCrone, Scott Murray and Myfanwy Morgan. The project expert panel comprised Massimo Costantini, Steve Dewar, John Ellershaw, Claire Henry, William Hollingworth, Philip Hurst, Tessa Inge, Karl Lorenz, Jane Maher, Irene McGill, Elizabeth Murray, Ann Netten, Alicia O'Cathain, Sheila Payne, Roland Petchey, Wendy Prentice, Deborah Tanner and Celia A Taylor. The researchers were Hamid Benalia, Catherine J Evans, Marjolein Gysels, Nancy J Preston and Vicky Short. iv

5 CONTENTS Review outline 1 Definition of terms 2 Complex intervention 2 Palliative care 2 End of life care 3 Background 4 What is the MRC Guidance? 7 Study designs to measure the effectiveness of complex interventions 10 Application of the MRC Guidance in end of life care research 14 Step I: Theory, development and modelling 15 Step II: Feasibility of intervention and study design and active ingredients 15 Step III: Evaluation with comparison randomised trial or appropriate alternative 16 Step IV: Rollout and monitor results (implementation considered all steps) 16 How could the MRC guidance inform research on social care? 17 Research challenges and future requirements 17 Conclusions 19 References 20 Resources 29 Appendix I: case studies exampling evaluations of complex interventions 30 v

6 REVIEW OUTLINE This review is a resource for people using or taking part in research on adult social care practice, particularly those wishing to develop, evaluate or implement models of service delivery or re-configurations. This review focuses on social care practice and end of life care to critically consider the potential application of the Medical Research Council Guidance on Developing and Evaluating Complex Interventions (MRC GDECI) for research on social care services and systems. End of life care has been taken as the main focus to illustrate the arguments with regard to evaluating complex interventions. Social care practitioners are major providers of end of life care (Wanless et al. 2006) and work alongside health care colleagues, with increasing calls for (Hughes-Hallet et al. 2011) and policy emphasis on integrated working to deliver care with greatest benefit for people and their families (Department of Health 2008, 2012). The review draws on the research study Methods of Researching End of Life Care (MORECare), which developed the MORECare statement by Higginson et al. (2013) on best practice solutions for research in end of life care to improve study quality, extending the existing MRC and other guidance on undertaking and reporting studies evaluating effectiveness (see Box 1). The arguments made from this context will, it is hoped, have resonance and be illuminating for researching complex interventions in other areas of social care. Box 1: MORECare Methods of Researching End of Life Care MORECare aimed to identify, appraise and synthesise best practice to develop and evaluate palliative and end of life care with a particular focus on complex servicedelivery interventions and reconfigurations. MORECare focused on the prominent challenges in conducting effectiveness research in palliative and end of life care including: participation in research (Gysels et al. 2012a), ethical considerations (Gysels et al. 2013), integration of mixed methods (Farquhar et al. 2013), properties of the best outcome measures (Evans et al. 2013a), managing missing data and attrition (Preston et al. 2013), and economic evaluation (Preston et al. 2012). The project undertook systematic literature reviews (Higginson and Evans 2010, Gysels et al. 2012a, Evans et al. 2013b) and consultations with medical experts, patients, and policymakers to further examine how the MRC GDECI could be incorporated in research on palliative and end of life care, as well as its limitations. The final synthesis of the findings formed the MORECare statement (Higginson et al. 2013). The statement details 36 practical best practice solutions for research in end of life care to improve study quality, extending the existing MRC and other guidance on undertaking and reporting studies evaluating effectiveness. The statement illustrates how the MRC GDECI could be developed to increase specificity to the challenges encountered in different areas of health and social care. 1

7 DEFINITION OF TERMS Complex intervention A complex intervention, by definition, denotes interventions comprising several components, some of which may be used interchangeably. Box 2 presents an example of a complex intervention. Complexity can emerge from the: * number of, and interactions between, components within the experimental and control interventions or groups; * number and difficulty of behaviours required by those delivering or receiving the intervention; * number of groups or organisational levels targeted by the intervention; * number and variability of intended outcomes (Craig et al. 2008), Box 2: Example of a complex intervention at the end of life care: palliative care for people with Multiple Sclerosis (MS) The work of Higginson and colleagues (Higginson et al. 2006, 2011) illustrates an example of a complex intervention. They designed a study to evaluate a new short-term palliative care service for people severely affected by MS. The complex intervention comprised existing MS services, with specialist palliative care and rehabilitation services, and collected data covering both physical and emotional symptoms to assess the intended outcomes for both people affected by MS and their carers. The intervention crossed different service streams with patients seen in various settings, for example their own home or rehabilitation service, while the analysis used both qualitative and quantitative methods. It is one of a limited, but growing, number of studies in palliative care that incorporate the MRC framework to model and evaluate a complex intervention. Palliative care The World Health Organization (WHO) defines palliative care as: an approach that improves the quality of life of patients and their families facing the problems associated with life threatening illness, through the prevention and relief of suffering by means of early identification and impeccable assessment and treatment of pain and other problems, physical, psychosocial and spiritual (WHO 2002 p.xv). 2

8 This definition is widely used in health care settings, but has limitations. Older people are the main users of social care. The WHO term life threatening illness has little resonance with older people living with increasing frailty and uncertainty about when they may near the end of life. Alternative definitions for this age group have drawn on the European Association of Palliative Care (EAPC) taskforce report by Froggatt and Reitinger (2011): End-of-life care for seniors requires an active, compassionate approach that treats, comforts and supports older individuals who are living with, or dying from, progressive or chronic life-threatening conditions. Such care is sensitive to personal, cultural and spiritual values, beliefs and practices and encompasses support for families and friends up to and including the period of bereavement. End of life care The term end of life care is often used interchangeably with palliative care, particularly in England and Wales since the advent of the National End of Life Care Programme and publication of the End of Life Care Strategy in 2008 (Department of Health 2008). There is no single definition of end of life; however, the following components are often suggested: 1. the presence of a chronic disease(s) or symptoms or functional impairments that persist but may also fluctuate; and 2. the symptoms or impairments resulting from the underlying irreversible disease which require formal (paid, professional) or informal (unpaid) care and can lead to death. Older age and frailty may imply life-threatening illness and co-morbidity; however, there is insufficient evidence for understanding these variables as components of end of life (National Institutes of Health 2004). End of life means different things to different people ranging from time of diagnosis, to a period of time after diagnosis encompassing a year or longer, to the last days of life (Shipman et al. 2008). In health and social care policy end of life care is generally seen as the last year of life (Department of Health 2008). In this review we use the term end of life as it is the one most commonly used in social care in England. Further discussion about the ethics and undertaking of research in social care settings on end of life care is contained in the SSCR methods review by Goodman and colleagues (Goodman et al. 2012). 3

9 BACKGROUND People with advanced conditions and increasing disability are major users of social care services, particularly towards the end of life (National Audit Office 2008). Social care support is provided across care settings encompassing people s own homes, community facilities and communal accommodation, for example, care homes. The demand for social care is rising and correspondingly the cost of providing care, both for the State and individuals. Yet, despite the increasing demands and cost, there is little evidence on the extent to which public expenditure achieves policy aims of promoting older people s choice and independence (Wanless et al. 2006) or enhances end of life care (Department of Health 2008). Much of the evidence base to inform social care practice in end of life care is weak, contributing to inequities in service provision, and there is little agreement on how to provide optimal support at the end of life, improve individuals quality of life or pursue cost-effectiveness. The provision of social care for individuals nearing the end of life is a prominent and increasing area of practice (Department of Health 2008, National Audit Office 2008, National End of Life Care Programme 2010). Higher demand is associated with an increasingly older population, an estimated 17% rise in the annual death rate by 2030 (Gomes and Higginson 2008), and preference for care in usual residence both from individuals (Gomes et al. 2011) and policymakers (Department of Health 2008). Usual residence, or home, is an environment that is familiar, where autonomy is preserved and loved ones are nearby (Gott et al. 2004) encompassing a range of settings, for example, at home or in a care home. Over a third (34.7%) of people die in their usual residence (including care homes and at home), but most (58%) die in hospital (National End of Life Care Intelligence Network 2010). Hospital admission is a major cost driver in end of life care services (Teno et al. 2013). Since the implementation of the National End of Life Care Programme in England in 2004, a slow, steady increase is reported in the number of people dying at home, but the greatest increase in numbers has been for people with cancer (Gomes et al. 2012, Gao et al. 2013). The National End of Life Care Programme (Department of Health 2008) and framework for social care (National End of Life Care Programme 2010) advocates promoting high quality care across the country for all adults approaching the end of life. The development of social care services requires identification and dissemination of good practice and evidence (National End of Life Care Programme 2010). This necessitates robust, timely research to develop the evidence base on the effectiveness and cost-effectiveness of service provision and interventions to inform the components of the best care, the intended outcomes, and requirements and cost of delivering this. Bardsley and colleagues (2010) estimated the total expenditure for social care as 50 million for 16,479 individuals who died across three primary care trust (PCT)/local authority areas in England in The study sites showed marked variation in people s use of social care and NHS services. The authors concluded there is some evidence across all age 4

10 groups that higher social care costs at the end of life tend to be associated with lower inpatient costs (p.3). An analysis of the quality of social care services in England echoed the findings of Bardsley and colleagues in detailing wide variations in social care services and called for substantial reform in social care to address and meet increasing demand effectively (MPH Health Mandate 2012). Better provision of social care at the end of life could improve care (Hughes-Hallet et al. 2011) and potentially reduce reliance on acute hospital beds (Bardsley et al. 2010), and redistribution of funding for services would help people to attain their preferences for care (National Audit Office 2008). However, determining how to develop social care services to provide the best care for people at the end of life requires research to fill the significant gaps in the evidence base (Department of Health 2012). The context of social care poses many challenges when undertaking research. Care is provided across diverse care settings (e.g. own home, a care home) by practitioners with a range of expertise and roles (e.g. home care workers, social workers, nurses working in care homes), and is sometimes guided by policies and procedures of numerous employers (e.g. local authorities, private or voluntary sector providers). These complexities are exacerbated by the often fluctuating capacity and vulnerability of people receiving end of life care (Koffman et al. 2009). Social care services provision can be regarded as a complex intervention comprising multiple layers of organisation and interacting components (Craig et al. 2008). Complexity surrounds the provision of end of life care and is embedded in, for example, the multiple component nature encompassing physical, psychosocial and spiritual needs of both the person and family, the multitude of organisations involved (for example, home care agency, primary health care, pharmacies) and practitioners (for example, GP, specialist palliative care nurse, home care worker, social worker, care home assistant) and provision of care across care settings. As such, the Medical Research Council Guidance on Developing and Evaluating Complex Interventions (MRC GDECI) is clearly relevant, and the MORECare Statement (Higginson et al. 2013) complements this by furthering the guidance for end of life care. What is uncertain is the applicability of the MRC guidance in addressing the specific challenges for research on other aspects of social care services. This methods review aims to add to the discussion about the applicability of the MRC GDECI for research on social care by drawing on evaluative research undertaken in palliative and end of life care. The review intends to: n n analyse the strengths and limitations of the MRC GDECI to develop and evaluate complex interventions, and approaches to overcome these limitations in research on palliative and end of life care; examine the contribution of the MRC GDECI to develop and evaluate palliative and end of life care services and interventions and its potential for use in research social care; 5

11 n n n highlight good research practice on how and when to apply the MRC GDECI to develop and evaluate social care, including situations where treating or managing advanced illnesses or conditions is not the focus of the intervention; identify the resource implications of incorporating the MRC GDECI in research that seeks to develop and evaluate complex interventions in social care settings by social care practitioners; and consider the ethical questions raised by using the MRC GDECI to develop and evaluate social care support for adults at the end of life. 6

12 WHAT IS THE MRC GUIDANCE? The MRC GDECI (Campbell et al. 2000, 2007 Medical Research Council 2000, Craig et al. 2008, Medical Research Council 2008) takes researchers and research funders through a staged process of developing, evaluating and implementing a complex intervention or treatment in which multiple structures, people and processes are involved, notably in the delivery of the service. The MRC GDECI (Medical Research Council 2008) addresses the development of a complex intervention and it incorporates detailed understanding on the many influences within the context, the research problem and the linkage with the intervention and the intended outcomes, stating: Best practice is to develop interventions systematically, using the best available evidence and appropriate theory, then to test them using a carefully phased approach, starting with a series of pilot studies targeted at each of the key uncertainties in the design, and moving on to an exploratory and then a definitive evaluation (p.8). The MRC GDECI is widely used by researchers, both nationally and internationally, to inform study designs evaluating the effectiveness of a complex area of practice. The Guidance is mainly and increasingly used in health services research, for example, in evaluative studies of palliative and end of life services (Higginson et al. 2006, 2008, 2009 Hall et al. 2009, 2011b, Higginson 2013) and adapted for research on patient safety (Brown et al. 2008a,b,c,d). However, there is growing use and interest in incorporating the MRC Guidance in research that is developing and evaluating social care services and interventions (Webber 2010). The MRC Guidance was first published in 2000 (Medical Research Council 2000) as a framework to aid researchers and funders in selecting appropriate research methods to develop and evaluate the effectiveness of complex interventions. It was initially informed by the phased approach used in clinical trials of pharmacological interventions, which uses a systematic phased and linear trajectory of increasing evidence (see Table 1). The 2000 MRC Guidance used a similar step-wise approach (see Figure 1), developing the clinical trials approach to include evaluations of complex interventions in health services research. The initial focus was primarily on randomised control trials (RCTs), but with a view that the approach could be adapted for non-rct studies. The 2000 MRC Guidance recognised that often healthcare evaluations comprise a number of complex interacting components, but the Guidance only partially considered the context in which the intervention occurred and the influence of approaches to understand this, notably the incorporation of mixed methods in trial designs (Craig et al. 2008). 7

13 Table 1: Phases of a clinical trial Phase Objective Typical number of patients Phase I Phase II Phase III Phase IV To explore possible toxic effects and determine tolerance of the intervention (and tolerated dose, if a drug study). To determine if treatment has a therapeutic effect or if there is any hope for benefits to outweigh the risks. To compare new treatment to the standard therapy or a control or placebo (if no standard therapy exists). To obtain long-term, large-scale information on morbidity and late effects (postmarketing study). 0 to to to 1000 Hundreds or thousands Source: Stanley Figure 1: Framework for development and evaluation Implementation Preclinical or theorectical Why should this intervention work? Explore relevant theory to identify best intervention, hypothesis, confounders and strategic design issues Modelling How does it work? Provide evidence that you can predict how the intervention components relate and interact with the outcomes Exploratory or pilot trial Optimising trial measures Describe the constant and variable components of the intervention AND a feasible protocol for comparing the intervention to an appropriate alternative Definitive RCT Compare a fullydefined intervention to an appropriate alternative using a protocol that is theoretically defensible, reproducible and adequately controlled, in a study with appropriate statistical power Determine if others can replicate your intervention and results in uncontrolled settings Phase 0 Phase 1 Phase 2 Phase 3 Phase 4 CONTINUUM OF EVIDENCE Reprinted with permission from BMJ Publishing Group Ltd under Scientific, Technical & Medical Publishers Guidelines Sources: Campbell et al. 2000, Medical Research Council

14 As researchers began to apply the 2000 MRC Guidance, particularly in research on primary health care, it became evident that the framework was not wholly comprehensive and required updating in some areas. These included how to approach intervention development, greater emphasis on the context in which the intervention took place, mixed methods approaches as opposed to 'traditional' RCTs, methods of reporting studies, and the implementation of complex interventions (Campbell et al. 2007, Craig et al. 2008). In 2008, the MRC published new Guidance (Medical Research Council 2008) tackling these issues and re-emphasising key points, but in a more flexible way illustrating its use through case studies and making it more digestible, in particular for early level researchers. In the 2008 Guidance, the emphasis shifted to focus more on the development phase to better understand the problem, its ingredients and context, and to define the intervention and linkage between the problem and the intended outcomes. Instead of a linear trajectory, the 2008 Guidance followed a more cyclical fluid approach (see Figure 2) making it more amenable to other areas of research and care by recognising that often complex interventions do not follow a straightforward linear sequence. Figure 2: Key elements of the development and evaluation process Feasibility/piloting 1. Testing procedures 2. Estimating recruitment/retention Development 1. Identifying the evidence base 2. Identifying/ developing theory 3. Modelling process and outcomes 3. Determining sample size Implementation 1. Dissemination 2. Surveillance and monitoring 3. Long term follow-up Evaluation 1. Assessing effectiveness 2. Understanding change process 3. Assessing costeffectiveness Reprinted with permission from BMJ Publishing Group Ltd under Scientific, Technical & Medical Publishers Guidelines Sources: Craig et al. 2008, Medical Research Council

15 STUDY DESIGNS TO MEASURE THE EFFECTIVENESS OF COMPLEX INTERVENTIONS A major challenge to designing and implementing a research study is ensuring the best research methodology and methods are used to answer the research question. The MRC Guidance informs researchers decision-making on designing studies. Box 3 outlines what makes an intervention complex using the example of social care and Figure 3 maps the decision-making process in designing a research study to evaluate a complex intervention. The Guidance asserts that randomised designs are the most robust method to reduce bias, through randomly allocating participants to intervention and control groups. However, the 2008 Guidance acknowledges the requirement to use the best available research methods and fit according to the area of investigation, limitations and ethical considerations. For example, the evaluation of an existing service may preclude an RCT. The Guidance discusses alternative study designs to an RCT, for example, a stepped-wedge design, cluster randomisation or quasi-experimental methods (see Box 3, definitions of methods). Non-RCT designs or observational designs are important when randomisation is impracticable or undesirable (Black 1996), for example when there are ethical issues concerning randomising to the control group (Grande and Todd 2000), and are increasingly used in research on palliative and end of life care (Evans et al. 2013b). However, observational designs in palliative care are often weakened by sample selection bias with patient self-selection and limited rigour in the comparative element (Higginson et al. 2003, Evans et al. 2013b). Box 4 details the types of research methods which may be used to evaluate the effectiveness of a complex intervention. It begins with experimental design, for example RCTs, and moves onto observational and quasi-experimental designs. Box 3: What makes an intervention complex in social care? a wide range of interacting interventions; many care settings, each with interacting layers of organisational complexity; complexity is not simply the number of elements in the intervention itself, but the interplay between the multiple dimensions involved in the delivery and receipt of services; an overriding value base with profound methodological implications; for example, at end of life, the involvement of people approaching the dying phase and their families and the importance placed on their desired choices and outcomes; multiple outcome measures are needed to capture change when delivering the intervention at the individual and/or organisation level and among those receiving the intervention. Source: Craig et al

16 Figure 3: Choosing a study design Research study yes Can you manipulate the variables? no Experimental or analytical study design possible Observational study design possible Is there a comparison group? Do you have a hypothesis to test? no yes no yes Are there multiple measurements? Allocation to groups controlled by researcher? Desciptive study who/what/where Analytical study to test hypothesis no Not an experiment yes Allocation at random? yes Intervention manipulated by researcher? Cross sectional study Cohort study Cross sectional study Cross sectional study Cohort study Case control study yes no yes no Some form of randomised controlled trial Some form of Quasi matching study experiment Natural experiment Source: Todd

17 Box 4a: Research methods to evaluate the effectiveness of a complex intervention Experimental designs Individually randomised trials: Individuals are randomly allocated to receive either an experimental intervention or an alternative such as standard treatment, a placebo or remaining on a waiting list. Such trials are sometimes dismissed as inapplicable to complex interventions, but there are many variants of the basic methods, and often solutions can be found to the technical and ethical problems associated with randomisation. RCTs have been used in palliative care, for example, to evaluate the effectiveness of home palliative care services (Grande et al. 1999) and interventions (Bakitas et al. 2009, Maloney et al. 2013) and dignity therapy for older people in care homes (Hall et al. 2011a). Cluster randomised trials: Contamination of the control group, leading to biased estimates of effect size, is often cited as a drawback of randomised trials of population level interventions (Eccles et al. 2003). Cluster randomisation is a solution and exampled in palliative care (Jordhoy et al. 2000, Costantini et al. 2011, 2013a, 2013b), but requires a more complex study design than an RCT. In cluster trials participants are randomised at a defined level. For example, by community healthcare districts in a single city to evaluate home care palliative intervention (Jordhoy et al. 2000), by regional areas in a country to evaluate the Liverpool Care Pathway (Costantini et al. 2011, 2013a, 2013b), by GP practice to evaluate educational outreach visiting in palliative care (Abernethy et al. 2006), or by nurse specialist to evaluate cognitive behavioural therapy for patients with advanced cancer (Moorey et al. 2009). Stepped wedge designs: The randomised stepped wedge design may be used to overcome practical or ethical objections to experimentally evaluating an intervention for which there is some evidence of effectiveness or which cannot be made available to the whole population at once. It allows a randomised controlled trial to be conducted without delaying roll-out of the intervention. Eventually, the whole population receives the intervention, but with randomisation built into the phasing of implementation (Brown and Lilford 2006). Preference trials and randomised consent designs: Practical or ethical obstacles to randomisation can sometimes be overcome by the use of non-standard designs. Where patients have very strong preferences among treatments, basing treatment allocation on patients preferences, or randomising patients before seeking consent, may be appropriate to facilitate people s participation in the study (Critchley et al. 1999). N-of-I designs: Conventional trials aim to estimate the average effect of an intervention in a population, and provide little information about within or between person variability in response to interventions, or about the mechanisms by which effective interventions achieve change. N-of-I trials, in which individuals undergo interventions with the order or scheduling decided at random, can be used to assess between and within person change, and to investigate theoretically predicted mediators of that change. Delayed intervention, deferred intervention or fast-track trial: Trials in which patients are not denied access to an intervention under study but are either randomised to receive the intervention more quickly than they would normally (the fast-track group) or receive it after a period on a waiting list (control group). Trials such as these are often able to assess the effects of waiting times. This type of design is used in palliative care as it enables all to receive the intervention, but mainly for people with non-malignant conditions with anticipated prognosis enabling intervention delay (Higginson and Booth 2011, Bausewein et al. 2012). Source: Campbell et al

18 Box 4b: Research methods to evaluate the effectiveness of a complex intervention Observational and quasi-experimental designs Observational designs: can be either descriptive or analytic studies. Descriptive studies describe the prevalence of an event and are usually based on case reports or case series describing the area in detail, in which often little is known, revealing something original. Palliative care has used the observational design of ethnography, in which a researcher becomes actively involved in the lived experiences of a particular patient group, recording the details and interpreting the findings, thus providing in-depth understanding of an area (Gysels et al. 2012b). Analytic studies are often used when enough is known about the problem, but an understanding of causation is lacking. Analytic studies therefore examine the association between a dependent variable and one or more independent variables examining possible causative factors. Different methods used in analytic studies are: Case-control study: compares groups of cases against groups of controls, for example, to examine the effectiveness of an existing hospital palliative care service (Jack et al. 2006); Cohort study: compares the experience of one group exposed to a study factor to another group not exposed to a study factor, for example, to compare the performance of a prognostication tool for people with dementia to predict six month survival (Mitchell et al. 2010). Quasi-experimental: studies the effectiveness of an intervention, but without the designation of a randomised group. In a quasi-experimental study individuals are self-selected to receive the intervention, for example, using a service such as a day hospice (Higginson et al. 2010) or palliative home care team (Costantini et al. 2003). Source: Costantini and Higginson Box 4c: Research methods to evaluate the effectiveness of a complex intervention Mixed methods designs Mixed methods research is sometimes referred to as the triangulation method: different sources are used to explore the same phenomenon. The approach uses techniques to combine the results of qualitative and quantitative studies to provide researchers with more knowledge than separate analysis (O Cathain et al. 2010). Mixed method approaches are used both to develop and evaluate complex interventions (Farquhar et al. 2011), particularly within RCTs (Flemming et al. 2008, Lewin et al. 2009, O Cathain et al. 2010). Their wider incorporation in research on end of life care is advocated (Higginson et al. 2013) for example, to evaluate dignity therapy for older people in care homes (Hall et al. 2013). Mixed method evaluations often use case studies to frame the research, most commonly to understand the problem and develop the intervention, combining both quantitative and qualitative data, for example surveys, interviews, observation, and documents (O Cathain et al. 2013). Mixed methods intend to enhance understanding of how an intervention works (or not) and inform the design of subsequent studies. However, they are challenging to undertake, in terms of time, resource and expertise, and the integration and reporting of results. This requires mixed method studies to use agreed protocols, work with multidisciplinary teams and to engage staff with relevant skill sets. Sources: Ingleton and Davies 2007, O Cathain et al. 2010, Farquhar et al

19 APPLICATION OF THE MRC GUIDANCE IN END OF LIFE CARE RESEARCH There are many challenges in undertaking evaluative research on end of life care services and interventions. The MRC Guidance is clearly relevant. The MORECare study supported the use of the MRC Guidance to inform study design, but identified shortcomings and developed the MORECare Statement, proposing ways to address these and accommodate the research challenges common in research on end of life care (Higginson et al. 2013) (see Figure 4). Figure 4: MORECare Statement key steps in developing and evaluating end of life care interventions Rollout and monitor results of wide implementation Feedback and revisions If successful Evaluation with comparison using randomised trial or appropropriate alternative. Asess outcomes and intervention processes If successful Feasibilty of intervention AND study design and mechanism/ active ingredients If successful Theory, development and modelling try to understand mechanism of intervention Consider implementation implications at each step Reprinted with permission from BMC Medicine under Scientific, Technical & Medical Publishers Guidelines Source: Higginson et al The MORECare study identified three shortcomings of the MRC Guidance: 1. moving from feasibility and piloting of a complex intervention to implementation without robust evaluation; 2. failing to develop the feasibility of the evaluation methods alongside the feasibility of intervention; and 3. lack of a theoretical framework underpinning the intervention (Higginson et al. 2013). 14

20 These shortcomings may contribute to a lack of pragmatic trials, or, when attempted, trials that fail. The authors argue for a requirement to build the intervention and research methods simultaneously. This means that researchers should gather and provide detailed understanding on how the intervention might work, and set out the process of delivering and means of measuring the outcomes. Moreover, the authors assert that implementation requires integration in all phases of the evaluation, forming a continuous thread, not a separate stage as proposed in the MRC Guidance (Evans et al. 2013b, Higginson et al. 2013). Implementation as a continuum intends to ensure that at the rollout stage the intervention is feasible in the context of the service delivery, that the process of implementation and intended outcomes are understood, planned for and resourced, and that the means of monitoring change and impact are in place. Although it is possible to begin at any step in the ladder it is important to progress development with successful interventions. Ways to develop and incorporate the MRC Guidance are explored below through research examples of use in end of life care research, from development, to feasibility assessment, through to evaluation with comparison and rollout with implementation as a continuum. This understanding intends to inform ways the MRC Guidance could be applied in research on social care (see Appendix 1 for further discussion of these research studies). Step I: Theory, development and modelling A delayed intervention randomised controlled trial: from qualitative feasibility to RCT Higginson and colleagues (2006, 2006, 2009) drew on the MRC Guidance to develop and evaluate a new palliative care service for people severely affected by multiple sclerosis (MS). The developmental phase of the research comprised theoretical modelling through reviewing the literature and qualitative interviews with people severely affected by MS, and their carers, together with focus groups and interviews with healthcare professionals. Prior to this study relatively little was known about the quality of life of people severely affected by MS. From the systematic review 15 domains of care were found to be important to people with MS and 12 domains were seen as important to carers of people with MS. To validate these domains, parallel exploratory/developmental work was undertaken using semi-structured interviews with people severely affected by MS, together with focus group discussions with relevant healthcare professionals and stakeholders. The combined data from the developmental phase were then used to develop a new palliative care service for those severely affected by MS and design a comparative feasibility trial to ascertain patient and carer benefit and develop the research methods for a full RCT if benefit was shown. (See also Gruenewald et al. 2004, Edmonds et al. 2007a, 2007b.) 15

21 Step II: Feasibility of intervention and study design and active ingredients A randomised phase II trial of dignity therapy in palliative care using mixed methods Hall and colleagues (2009, 2011b, 2013) undertook a feasibility RCT of dignity therapy in patients with advanced cancer, underpinned by developmental work on dignity therapy (Chochinov et al. 2002a, 2002b, 2008). Participants were recruited to either the intervention (dignity therapy) or the control (standard palliative care). The researchers assessed the primary outcome of dignity-related distress between the two groups, and secondary outcomes of hope, anxiety, depression, quality of life, palliative-related outcomes and self-reported benefits. The findings showed no difference between the intervention and control groups in terms of primary outcomes, but significant difference in the secondary outcome of hope. Hall and colleagues further developed their work by evaluating dignity therapy in care homes to explore the generalisability of the dignity model for older residents (Hall et al. 2011). This illustrates the ways in which studies can move back and forth between the developmental and feasibility phase, highlighting the relevance of the MRC 2008 updated Guidance, which is less linear than its predecessor. Step III: Evaluation with comparison randomised trial or appropriate alternative A prospective RCT of an educational and care management palliative care intervention for persons with advanced cancer and caregivers compared to care as usual Bakitas and colleagues (2009) undertook a proactive intervention to improve palliative care for rural patients with advanced cancer. Half of the participants recruited were randomly assigned to a telephone-based, nurse-led, educational, care coordination, palliative care intervention model. Intervention services were provided to half of the participants weekly for the first month and then monthly until death, including a bereavement follow-up call to the caregiver. The other half of the participants were assigned to care as usual. Highlighted through the findings is the feasibility of undertaking a rigorous randomised controlled trial in palliative care. (See also Bakitas et al. 2009; Maloney et al. 2013). Step IV: Rollout and monitor results (implementation considered all steps) The implementation of research into practice is fundamental to optimising practice. Research findings can only change population health outcomes if adopted and embedded by health care systems, organisations and clinicians (Grimshaw et al. 2004). Yet, there is a consistent failure to apply effective innovations in practice with implementation approaches too unstructured, narrowly focused and largely ineffective (Eccles et al. 2009). This is the part of evaluative research on end of life care that receives least attention (Evans et al. 2013b). The National End of Life Care Programme provides some examples of national implementation of tools to support end of life care, including the End of Life Care Pathway and Gold Standards Framework (see Resources). Although the evidence underpinning the effectiveness of these tools is less strong, they demonstrate national implementation of tools to support practice (Badger et al. 2012). 16

22 HOW COULD THE MRC GUIDANCE INFORM RESEARCH ON SOCIAL CARE? Social care practitioners and practitioners working across health and social care services aim to offer support to people who often have complex health and social care needs, to help them achieve the outcomes they value. Many interventions are complex. The MRC framework has been used in only a few research studies in social care (Forrester et al. 2008, Webber et al. 2012). However, there is growing interest in its application in this field to promote evidence and improve practice. In an SSCR-funded study, Webber and colleagues (see Appendix 1, case study 3) used the MRC Guidance to develop the Connecting People Intervention and to undertake a feasibility study to develop the methods for a definitive quasi-experimental large trial if benefit was shown. The team undertook a two-year study with an ethnographic component to understand how social workers help young people with psychosis to generate and mobilise social capital. Through the ethnography an understanding of different components of service effectiveness emerged, enabling the research team to develop the Connecting People Intervention, which supports people to engage with their local communities, enhance their social networks and improve mental wellbeing. Once the feasibility work is complete, and if results are positive, the research team will undertake a definitive evaluation to measure effectiveness. As champions of evidence-based practice in social work, Webber and colleagues intend to use the findings of their work to inform the National Institute for Health and Clinical Excellence (NICE) guidelines translating social (as opposed to medical) research into national guidelines. If successful in its planned trajectory this study would highlight the value of social care research, promote the implementation of research findings into practice through guidelines and re-emphasise the importance of rigorous, replicable research methods. Research challenges and future requirements In end of life care and social care in general there is a lack of effectiveness studies. There is a call to build on the MRC GDECI for research on social care to debate and address the specific challenges encountered, to ensure researchers use the best methods to evaluate complex interventions in a manageable way, and to encourage research funders to appreciate the importance of a staged approach requiring investment in researchers and programmes of research (Evans et al., Higginson et al. 2013). The MORECare Statement provides solutions and a checklist on addressing the prominent challenges in research on end of life care. This sets a much needed standard for evaluative research on palliative and end of life care, and for addressing the specific challenges and complexities in research on social care more generally. These include: the complexity of measurement of effectiveness of interventions for example among people living with an advanced condition who are deteriorating (Grande and Todd 2000); 17

23 challenging ethical dilemmas, such as that of including people with advanced conditions in studies and difficulties in consent procedures for adults lacking capacity (Grande and Todd 2000, Gong et al. 2010, Gysels et al. 2013); small sample sizes because of attrition and missing data, for example associated with increasing fatigue and death (Preston et al. 2013), which may affect the ability to detect significant change from the intervention (Rinck et al. 1997); resistance to taking part in research trials, particularly from practitioners, but often less so from people using services (Bellamy et al. 2011, Shannon-Dorcy and Drevdahl 2011, Goodman et al. 2012, Gysels et al. 2012a); and unfamiliarity or poor experiences with research among people working in social care services (Knapp and Richardson 2012). The MORECare Statement provides guidance on ways to further the MRC guidance and address these challenges, and these points could be helpful across other sectors of social care beyond end of life care: study designs could extend beyond randomised trials to include robust comparative observational designs and wider use of mixed methods; greater consumer or service user collaboration in developing studies, particularly to help resolve ethical dilemmas, such as about involving people with advanced illness in research; consideration of real-world implications or implementation of an intervention at all stages of the project may yield considerable benefits; and greater emphasis on developing the feasibility of the evaluation methods alongside the feasibility of the intervention. 18