Routine outcome measurement completion rates of the CORE-OM in primary care psychological therapies and counselling

Transcription

1 Counselling and Psychotherapy Research, March 2006; 6(1): 33/40 ORIGINAL ARTICLE Routine outcome measurement completion rates of the CORE-OM in primary care psychological therapies and counselling BRIDGETTE M. BEWICK 1, KAREN TRUSLER 1, TRACY MULLIN 1, STEWART GRANT 2,& GEOFF MOTHERSOLE 3 1 Psychological Therapies Research Centre, University of Leeds, 2 Dumfries and Galloway Department of Psychological Services and Research, and 3 West Sussex Health and Social Care NHS Trust Abstract Formally evaluating and reporting on the progress of clients is now seen as a central component of clinical practice. Generally, research suggests that the percentage of practitioners who use outcome measures as part of their clinical practice is relatively low. However, no national benchmarks have yet been presented which allow both practitioners and services to assess their relative performance in terms of data collection. The present study uses the CORE National Research database (n/31882) and reports pre- and post-therapy assessment completion rates, thereby enabling services to track their performance relative to other services. On average, pre-therapy CORE-OMs were reliably completed by 83% of clients while post-therapy forms were completed by 39%. However, there were large variations between services. The practical use and implications of the results presented are discussed by managers of NHS primary care counselling services. Keywords: Outcome measure completion, primary care, counselling, service quality management Introduction Increasing emphasis has been given to the routine measurement of outcomes in psychological therapy and counselling in a range of National Health Service policy and guideline publications over the past ten years. This introduction starts with a brief summary of the main publications along with their key messages. A recommendation for routine outcome measurement is then given, followed by best practice guidelines. In 1996 the National Health Service Executive commissioned the Review of Strategic Policy on NHS Psychotherapy Services in England (Department of Health, 1996) which explored the quality of national service provision. A later reflective chapter summarised the findings of the review suggesting a picture of poorly targeted, inappropriate interventions, ineffective organisation, and poorly coordinated services, leading to wasted resources (Parry, 1998, p. 11). The proposed solution to improve patient care included the use of evidence-based practice in which research, professional consensus, individual clinical practice, service delivery systems, and outcome evaluation would play a collective part. This model directly led to the subsequent publication of Treatment Choice in Psychological Therapies and Counselling (Department of Health, 2001). In this publication specific mental health problems were matched with evidence on the efficacy of a range of therapy interventions. In parallel with these profession specific quality developments, the rolling publication programme introduced standards across key healthcare sectors providing The National Service Framework for Mental Health (Department of Health, 1999). This framework listed a total of seven standards designed to drive up quality and remove wide and unacceptable variations in provision (p. 5). Standard two of the framework specifically set a performance indicator stating that service users with common mental health problems should be offered effective treatments. Subsequent announcements in the framework document proposed to complement professionally derived measures of health and well-being by including the routine use of quality of life measures. It was proposed that these quality of life measures should ultimately be benchmarked both locally and nationally. Two further documents over the following five years endorsed and formalised these requirements for routine evidence of effectiveness. In 2002 the National Institute for Mental Health in England (Eastern) published A Good Practice Framework for Improving Access to Psychological Therapies for People with Correspondence: Bridgette M Bewick, Psychological Therapies Research Centre, University of Leeds, 17 Blenheim Terrace, Leeds, LS2 9JT. b.m.bewick@leeds.ac.uk (print)/ (online) 2006 British Association for Counselling and Psychotherapy DOI: /

2 34 B. M. Bewick Common Mental Health Problems. This document contained a set of 10 principles by which service quality could be assessed and benchmarked. Amongst the mandatory standards, principle 9.5 required routine, systematic attempts to assess the outcomes of psychological interventions using standardised instruments and to examine these against benchmarking criteria to demonstrate acceptable levels of service effectiveness and to target improvements in services. Two years later the Department of Health publication Organising and Delivering Psychological Therapies (2004) recognised that in many mental health services psychological therapy provision is patchy, uncoordinated, idiosyncratic, potentially unsafe, and not fully integrated into management systems (p. 1). It therefore recommended that services incorporate measures of outcome into your psychological therapy services as a matter of routine (p. 31). Finally, Outcomes Measures Implementation Best Practice Guidance (National Institute for Mental Health in England, 2005) reported the results of a two year programme to introduce routine outcome measurement across NHS mental health services. The findings highlight the challenges of the endeavour showing how the success of routine outcome measurement depends critically on having the right incentives. That is, winning the positive and active engagement of all stakeholders, particularly practitioners and service users on whom the burden of outcome measurement falls. Box 1 highlights relevant research findings on the subject of routine outcome measurement. We look at these findings and relate them to our own in greater detail in the discussion. The current paper includes benchmarks showing services ability to capture CORE-OM data pre- and post-therapy. These benchmarks will provide practitioners and service managers with comparative indicators to help them reflect on their local success in rising to the challenge of routine measurement. Box 1. Contextual research findings There is a considerable pool of outcome measures currently available to assess the effectiveness of psychological therapy and counselling. Froyd et al. (1996) published a review of outcome measures and found 1,430 different outcome measures across 334 reviewed papers. Since Froyd s review there has been continued international debate on the issue with contributions from Australia (e.g. Trauer, 2004; Pirkis et al., 2005), the United States (e.g. Hatfield & Ogles, 2004; Lambert, 2005); as well as the United Kingdom (Barkham et al., 2001; Mellor-Clark & Barkham, 2006). While clinicians often assess therapeutic outcomes informally (Ogles, Lambert, & Fields, 2002), international research suggests that the percentage of clinicians who use outcome measures as part of their routine clinical practice can be highly variable and in general is low. In the United States, Phelps et al. (1998) found that only 29% of licensed psychologists surveyed routinely used outcome measures. Six years later Hatfield and Ogles (2004) reported a slightly larger proportion of measurement use at 37%. In Australia, Eagar et al. (2005) found high rates of measurement completion with 95% of adult clients having at least one point of measurement; however, when they investigated the proportion of adults who had pre and post-therapy measures completed, the figure dropped to 58%. Trauer (2004) found there can be substantial variations in the percentage of outcome measures completed between Australian services (i.e. less than 10% to more than 50%) and this may be explained by the extent to which services and practitioners engage in good practice such as inviting clients to self-rate, following up selfrating with discussion, identifying discrepancies between client and clinician ratings, sharing knowledge with clients and integrating results in individual treatment plans. In the UK, Mellor-Clark et al. (1999) found very high rates of measurement use. In a survey of 220 NHS services, all providers used measurement at assessment/first contact sessions, and 59% measured post treatment outcomes. However, there was a vast range of over 57 different measures used. Despite numerous drivers promoting routine measurement as good practice, practitioner perceptions that measures add extra burden to their clients is consistently identified as a barrier to implementing outcome assessment strategies (Hatfield & Ogles, 2004), and non-engagement studies suggest that characteristics associated with the service provider are more likely to impact on completion rates rather than client characteristics (Trauer, 2004). The reluctance of some clinicians to utilise routine outcome measures within their practice has been attributed to their perception that such assessments cannot take into account the idiosyncratic processes of change within the client-therapist relationship (Roth & Fonagy, 1996); are an intrusion into the client-therapist relationship (Phelps et al., 1998); or perhaps will the result in data being used politically to look at the affordability of care (Corrie & Callanan, 2000). It has been suggested that the way in which practitioners are introduced to the process of using outcome measures within their practice has

3 Psychological therapies and counselling 35 an impact on the level of engagement of practitioners with the assessment process; those that understand the role of the measure in informing good practice are more likely to engage with the process than those who perceive the assessment as an activity unrelated to their clinical practice (Corrie & Callanan, 2001). Method Data set The data were drawn from The CORE National Research Database for Primary Care Psychological Therapies and Counselling (Mellor-Clark et al., 2006) and are profiled in Box 2. Data was omitted in some cases due to missing or inconsistent data and the Appendix provides a flow chart of the data cleaning process (Figure A1). Ethics approval for this study was covered by NHS COREC application 05/Q1206/128. Box 2. Subjects and dataset j j j A total of clients and 637 practitioners from 34 services participated in the current study The data was collected between 1999 and Of the final client sample (n /31882): j the majority were female (69%) j 82% were White/Caucasian. j the mean age was 37.5 years (SD 11.88). Measures The CORE Outcome Measure (CORE-OM; Barkham et al., 2001, 2005; Cahill et al., in press; Evans et al., 2002) comprises 34 items tapping the four domains of subjective well-being; symptoms (anxiety, depression, physical problems, trauma); functioning (general functioning, close relationships, social relationships); and risk (risk to self, risk to others). Half the items focus on low intensity problems (e.g., I have felt tense or anxious ) and half focus on high intensity problems (e.g., I have felt panic/terror ). Eight of these items are positively keyed. Items are scored on a 5-point scale, scored from 0 to 4 (anchored Not at all, Only occasionally, Sometimes, Often, and All or most of the time). Fuller detail can be found in Barkham et al. (2006). Data analysis Data analysis was conducted using SPSS software. The mean completion rates for pre- and post-therapy, pre-therapy only, and post-therapy only were calculated for each service. The mean data was then analysed to obtain values for the 25 th,50 th and 75 th percentiles. Results Descriptives: client and service level At the client level pre-therapy assessment forms were reliably completed (i.e., 5/3 missing items) by 83% (n/26379) of clients. Post-therapy assessment forms were reliably completed by 39% (n /12487) of clients. Table AI (see Appendix) provides a summary of the percentage CORE-OMs completed pre- and/or post-therapy by the overall sample and by gender. Analysis suggests that CORE-OM completion rates are not meaningfully affected by age, gender, ethnicity or pre-therapy CORE-OM score 1. At the service level pre-therapy assessment forms were reliably completed by on average 86% of clients; the corresponding post-therapy figure was 39% (Table AII, see Appendix). As shown by Table AII (see Appendix), there were large variations between services with the percentage of pre- and post-therapy CORE-OM completion ranging from 3% to 99%. Service benchmark indicators Benchmarks were derived from the data in order to provide an indication of the relative performance between services. The percentage of pre- and posttherapy CORE-OM completion was calculated for each service and these percentages were then ranked and the minimum, maximum, 25 th percentile, 50 th percentile and 75 th percentile figures obtained (Table AII, see Appendix). These figures were then used to create benchmarks which services can use to compare their rates of outcome measure completion against those of other primary care services (see Figures 1/3). Figure 1 shows the benchmark for the percentage of clients at the service level that completed the CORE-OM at both pre- and post-therapy. Figure 2 shows the benchmark for pre-therapy service completion rates and Figure 3 shows the benchmark Percentage (%) Pre and Post therapy 53% 38% 22% High Above average Below average Low Figure 1. Benchmark for percentage of CORE-OM completed preand post-therapy at the service level. 1 Further details are available from the lead author on request.

4 36 B. M. Bewick Percentage (%) Total pre-therapy 96% 93% 77% High Above average Below average Low clients can either robustly support, or seriously undermine, their claim to offer effective treatments. Given the increasing trend in NHS commissioning towards payment by results, the implications for services which are less successful in data collection are potentially serious. Put bluntly, how services respond to these challenges now may be crucial in determining their future survival. So against this developing policy context how are we performing, and what does this say about our practice? Figure 2. Benchmark for total pre-therapy completion rates at the service level. Percentage (%) Total post-therapy 54% 38% 24% for post-therapy service completion rates. Box 3 gives an example of how the benchmarks can be used. Taking the first benchmark (Figure 1) for pre- and post-therapy CORE-OM completion, it can be seen that the services that fell below the 25th percentile (white) had less than 22% of clients who completed both pre- and post-therapy CORE-OMs. The services that fell above the 75th percentile (black) had more than 53% of clients who completed the CORE-OM on both occasions. Reflective discussion and service managers comments Why is this benchmark important? High Above average Below average Low Figure 3. Benchmark for total post-therapy completion rates at the service level. We now have an explicit requirement for services to routinely measure outcomes. The relative success of services in routinely collecting outcome data from Box 3. How can the benchmarks be used? Example: If a service had a mean pre- and postpercentage of completed CORE-OM s of 26% they would fall between the 25 th percentile (22%) and the 50 th percentile (38%) (See Table AII; Appendix and Figure 1). The percentage of CORE- OMs completed pre- and post- for that service could therefore be classified as below average relative to other services completion rates. Data collection and attrition Across all services, the mean level of pre-therapy CORE-OM completion by clients at the service level is 86%. Post-therapy completion rates drop considerably with only 39% of clients at the service level completing measures both pre- and post-therapy. These figures are consistent with much of the published research in this area (see Box 1). The range of completion rates across the 34 services highlights the different levels of success achieved by those services in implementing routine outcome measurement. At pre-therapy, those services that are most successful in their completion rates appear to capture data for every client assessed while the least successful achieve a rate of just one in five. At posttherapy this variation is even more extreme with a range stretching from almost 100% to just 3%. As a general principle, it would be expected that for any given service the proportion of post-therapy measures will always be lower than at pre-therapy. The main reasons for this are, firstly, that some clients who are assessed will have just a single session. This might be because a single session meets their needs, because they elect not to enter into therapy, or because they are considered unsuitable for further therapy. Secondly, not all clients who enter into therapy will reach a planned ending. Each of these scenarios represents a point at which data loss may occur and reduce the potential pool of post-therapy measures. These factors are explored in separate papers within this special edition of Counselling and Psychotherapy Research (see Cahill et al., 2006; Connell et al., 2006). The massive variation in CORE-OM completion between services does, however, suggest that there may be a great deal that we can do to minimise data loss. This is explored in the next section. Minimising data loss Ensuring that we collect a high percentage of outcome measurement data requires both the active commitment and engagement of key stakeholders as well as practical and robust systems for administering, collecting, and processing completed measures. It appears to be increasingly common that services that have been collecting CORE System data for periods in excess of 12 months are setting targets for completion rates of CORE-OM measures. These

5 Psychological therapies and counselling 37 targets demonstrate their commitment to accountability and ensure a sufficiently large dataset to draw reliable conclusions about the effectiveness of the service offered. To assist individual practitioners in meeting targets, many services have also developed a range of mechanisms for auditing and enhancing completion rates. In addition, there is some anecdotal evidence amongst service managers that those services where the collection of CORE-OM data is written into practitioners contracts are more successful in data collection. This may be because such clear contracts establish a frame in which data collection is seen as an intrinsic part of the task at hand and not an add-on to be adhered to if there is time. Previous research has shown that attrition is common within psychological therapy (Wierzbicki & Pekarik, 1993) and is likely to impact on post-therapy data collection. Even in the absence of a planned ending, the availability of outcome data at a second time point, even if it is after therapy has ceased, can yield valuable information about the psychological state of the client and may indicate that the intervention helped rather than hindered the client. Some services attempt to follow up clients who do not attend for a final session, usually by sending them a post-therapy CORE-OM. However, these attempts usually seem to yield low response rates. Perhaps the most effective way to address data loss through client attrition is to administer measures routinely at each session. This will become increasingly feasible with shorter versions of the CORE-OM (see Barkham et al., 2006) and internet-based systems that allow for direct entry of data onto the database (see Mellor-Clark et al., 2006). The following sections explore key factors involved in the process of outcome measurement implementation which may enhance or undermine success in meeting the challenge of routine measurement. They draw upon the experience of service managers in implementing the CORE System at both service management and individual practice levels. Leadership and culture The implementation of the CORE System and routine use of the CORE-OM is a significant undertaking for services and their practitioners. In its best practice guidance on outcomes measurement, the National Institute for Mental Health in England (2005) identifies the task of gaining the positive engagement of the service users, carers, and practitioners as the main barrier to successful implementation of outcome measurement. Practitioners are more likely to engage with the process of using standardised outcome measures routinely if they understand the benefits of doing so for both themselves and their clients. The degree to which services engage in preimplementation planning and preparation varies very widely but anecdotal evidence suggests that they are the critical factors in successful implementation. Some services have approached implementation as a process of on the job learning while others have used a more active project management approach, identifying the very practical steps involved in introducing the CORE-OM and anticipating those points at which data collection might fail. Getting to this position requires (as is pointed out elsewhere in this issue) leadership to help in the generation of a culture of inquiry within the service. An essential part of generating such a culture is instilling the idea that the CORE-OM is a clinically relevant tool, providing potentially useful information for our assessment, and also as an indicator of progress. Where CORE-OM data is seen as clinically relevant, then we can expect that clinicians will naturally ensure that most clients complete the appropriate forms. Box 4 illustrates this point with a brief statement taken from a focus group of counsellors using the CORE System in a service with consistently high CORE-OM completion rates. Box 4. Example of use of CORE system in a service with consistency high OM completion rates... looking at it as a puzzle you ve got to start getting the container, the edges in place, and the assessment from CORE can be like building that container... can I work with this client in a way that s safe...if they re coming out with a high risk score it s not really a safe area to be for myself or the client, so I kind of use it as an assessment tool. Mothersole (2005, p. 165) Reaching this stage requires careful attention to questions such as How do I introduce the measure?, and How do I use what the client has told me [from the CORE-OM] within or between sessions? In order to provide practical answers to these questions training may be required to equip practitioners with the skills and confidence to effectively use the CORE-OM. Such training will ensure that practitioners are equipped both to use the measure and to understand its potential to inform clinical practice. At a broader level, a key part of the development of a culture of inquiry lies in providing feedback to practitioners. There is little more that is dispiriting for anyone, practitioners included, than to diligently ensure that forms are completed only for the data then to disappear into some metaphorical black hole, never to emerge again. Even more damaging is a sense amongst practitioners that data to which they have no access will in some way be used to make judgments about their effectiveness. It is vital, therefore, to ensure that practitioners receive something useful back from the entire process as quickly as possible. Thus developing habits of discussing data

6 38 B. M. Bewick and allowing practitioners to examine their own data is crucial in setting up a virtuous circle in which the completion of CORE-OMs is driven by curiosity and habit rather than compliance with external demands. Practice challenges As with all the benchmarks, the key question is What does the data tell us about our practice? Broadly speaking, the percentage of CORE-OMs completed can be taken as an indicator of two things. Firstly, it indicates the robustness of our data and therefore the weight that we can give to it. The greater the percentage of CORE-OMs we collect the more robust our data. Secondly, it acts as a useful indicator of the extent to which practitioners are compliant with, or better still committed to, the principle of collecting outcome data. There are, of course, many ways in which the CORE-OM can be used. It can be used purely as an audit tool with the client completing the measure and the practitioner not seeing it at all. Such an approach seems to deny the rich possibilities for using pre-therapy CORE-OM data to inform the process of assessment and the development of a treatment focus, as well as ignoring potentially crucial risk data. Where administered by practitioners, however, the collection of CORE-OM data can impinge directly on clinical work. It follows from this that if we are to develop and maintain high rates of completion, practitioners must see potential benefits of asking their clients to complete the measure. Potentially the CORE-OM can inform face-to-face clinical work, both in terms of assessment and as an indicator of progress in therapy, or lack of it. Perhaps seeing this direct benefit for the clinical work is the main route to getting high levels of completion. After all, if I as a practitioner see that the OM helps me with this client, it makes sense to use it. Incorporating the routine use of outcome measures can represent significant challenges for practitioners. These challenges may be either ethical or philosophical in nature. For example, practitioners may believe that measures will intrude on the therapeutic process or that measurement is reductionist and cannot reflect the finer nuances of the therapy process. Alternatively they may feel that the time required to administer measures represents an unacceptable time demand or not know when and how to introduce the measures with clients. For these and other reasons some practitioners may overtly reject the use of CORE-OM, or indeed any outcome measure, while others may be more passive in their resistance to measurement by being less diligent in their collection of data. A variety of research studies have explored the relative levels of success in engaging practitioners in the process of implementing routine outcome measurement (see Box 1). It is also likely that some practitioners will feel anxious about having their work scrutinised and will vary in their feelings towards facilitating data collection that could be used as a measure of their clinical effectiveness. This might, in part, help to explain the discrepancy between the amount of data collected at the beginning and the end of therapy. By definition, pre-therapy CORE-OM data records only the client s untreated state. The end of therapy CORE-OM, however, when examined in light of the pre-therapy data, is a measure of the effectiveness of therapy. Obviously there are factors other than the therapy itself which are likely to impact on post-therapy scores but, nevertheless, practitioners, managers and others who are scrutinising services will inevitably take changes in CORE-OM scores as an indication of effectiveness. Given this situation, it is conceivable that some practitioners may be less diligent, or be consciously selective, in collecting post-therapy data. The leadership and sensitivity displayed by the service manager in working with practitioners to address these issues will be a critical factor in winning their confidence and ensuring their full commitment. Conclusion Formally evaluating and reporting on the progress of clients by means of routine outcome measures (e.g., CORE-OM) is an important part of modern practice in counselling and the psychological therapies. It is vital that if a service is to make any useful sense of its outcome measurements, it must have reasonable completion rates. However, introducing and maintaining the use of the CORE-OM in a service is often hampered by the practitioner s view that getting clients to complete the CORE-OM impinges on their clinical work. The current benchmark figures show that those services that are most successful (i.e., in the top quartile) at collecting outcome measures both preand post-therapy manage to collect data for between 53% and 99% of their clients. These figures can act as an indicator to other services that it is possible to achieve and maintain high completion rates. Completion rates for the CORE-OM seem to depend primarily on practitioners attitudes towards routine outcome measurement and on service culture. Because completion rates in the lower quartiles may reflect negative experiences of CORE-OM, it is important to assess which practical, cultural, managerial, and philosophical blocks are preventing the gathering of this increasingly important data. Acknowledgements This work was supported by funding from the Artemis Trust and also from the R&D Priorities & Needs Levy via Leeds Mental Health Teaching NHS Trust. We would like to thank Barry McInnes, Richard Evans and Judith Slaa for their input. We greatly appreciate the feedback given at the Psychological Therapies Research Network (North) 2005 meeting. We also thank Alex Curtis-Jenkins at CORE IMS for his technical input into the data collection process.

7 Psychological therapies and counselling 39 References Barkham, M., Mellor-Clark, J., Connell, J., & Cahill, J. (2006). A CORE approach to practice-based evidence: A brief history of the origins and applications of the CORE-OM and CORE system. Counselling and Psychotherapy Research, 6, 3/15. Barkham, M., Gilbert, N., Connell, J., Marshall, C., & Twigg, E. (2005). Suitability and utility of the CORE-OM and CORE-A for assessing severity of presenting problems in psychological therapy services based in primary and secondary care settings. British Journal of Psychiatry, 186, 239/246. Barkham, M., Margison, F., Leach, C., Lucock, M., Mellor-Clark, J., Evans, C., et al. (2001). Service profiling and outcomes benchmarking using the CORE-OM: Towards practice-based evidence in the psychological therapies. Journal of Consulting and Clinical Psychology, 69, 184/196. Cahill, J., Potter, S., & Mullin, T. (2006). First contact session outcomes in primary care psychological therapy and counselling services. Counselling and Psychotherapy Research, 6, 41/49. Cahill, J., Barkham, M., Stiles, W. B., Twigg, E., Hardy, G. E., Rees, A., et al. (in press). Convergent validity of the CORE-OM measures with measures of depression for clients in cognitive therapy for depression. Journal of Counseling Psychology. Connell, J., Grant, S., & Mullin, T. (2006). Client initiated termination of therapy at NHS primary care and counselling services. Counselling and Psychotherapy Research, 6, 60/67. Corrie, S., & Callanan, M. M. (2000). A review of the scientistpractitioner model: Reflections on its potential contributions to counselling psychology within the context of current health care trends. British Journal of Medical Psychology, 73, 413/427. Corrie, S., & Callanan, M. M. (2001). Therapists beliefs about research and the scientist-practitioner model in an evidencebased health care climate: A qualitative study. British Journal of Medical Psychology, 74, 135/149. Department of Health (1996). NHS Psychotherapy services in England: review of strategic policy. London: HMSO. Department of Health (1999). National Service Framework for Mental Health. Modern Standards and Service Models. London: HMSO. Department of Health (2001). Treatment Choice in Psychological Therapies and Counselling: Evidence-based Practice Guidelines. London: HMSO. Department of Health (2004). Organising and Delivering Psychological Therapies. London: HMSO. Eagar, K., Trauer, T., & Mellsop, G. (2005). Performance of routine outcome measures in adult mental health care. Australian and New Zealand Journal of Psychiatry, 39, 713/716. Evans, C., Connell, J., Barkham, M., Margison, F., Mellor-Clark, J., McGrath, G., et al. (2002). Towards a standardised brief outcome measure: Psychometric properties and utility of the CORE-OM. British Journal of Psychiatry, 180, 51/60. Froyd, J. E., Lambert, M. J., & Froyd, J. D. (1996). A review of practices of psychotherapy outcome measurement. Journal of Mental Health, 5, 11/15. Hatfield, D. R., & Ogles, B. M. (2004). The use of outcome measures by psychologists in clinical practice. Professional Psychology: Research and Practice, 35 (5), 485/491. Lambert, M. J. (2005). Emerging methods for providing clinicians with timely feedback on treatment effectiveness: An introduction. Journal of Clinical Psychology, 61, 141/144. Mellor-Clark, J., Barkham, M., Mothersole, G., McInnes, B., & Evans, R. (2006). Reflections on benchmarking NHS primary care psychological therapies and counselling. Counselling & Psychotherapy Research. Mellor-Clark, J., Barkham, M., Connell, J., & Evans, C. (1999). Practice-based evidence and standardized evaluation: Informing the design of the CORE System. European Journal of Psychotherapy. Counselling and Health, 2, 357/374. Mellor-Clark, J., & Barkham, M. (2006). The CORE System: Developing and delivering practice-based evidence through quality evaluation. In C. Feltham & I. Horton (Eds.), Handbook of counselling and psychotherapy, 2nd Edition. (pp. 207/224). London: Sage Publications. Mellor-Clark, J., Curtis Jenkins, A., Evans, R., Mothersole, G., & McInnes, B. (2006). Resourcing a CORE Network to develop a National Research Database to help enhance psychological therapy and counselling service provision. Counselling & Psychotherapy Research, 6, 16/22 Mothersole, G. (2005). Closing the loop: engaging with Clinical Outcomes in Routine Evaluation. Unpublished doctoral study: Middlesex University/Metanoia Institute. National Institute for Mental Health in England (Eastern) (2002). A Good Practice Framework for Improving Access to Psychological Therapies for People with Common Mental Health Problem. Colchester: NIMHE. National Institute for Mental Health in England (2005). Outcomes Measures Implementation-Best Practice Guidance. Colchester: NIMHE. Ogles, B. M., Lambert, M. J., & Fields, S. A. (2002). Essentials of outcome assessment. New York: Wiley. Parry, G. (1998). Psychotherapy services, policy and audit. In R. Davenhill & M. Patrick (Eds.), Rethinking Clinical Audit: The case for psychotherapy services in the NHS. London: Routledge. Phelps, R., Eisman, E. J., & Kohout, J. (1998). Psychological practice and managed care: Results of the CAPP practitioner survey. Professional Psychology Research and Practice, 1, 31/ 36. Pirkis, J., Burgess, P., Coombs, T., Clarke, A., Jones-Ellis, D., & Dickson, R. (2005). Routine measurement of outcomes in Australia s public sector mental health services. Australia and New Zealand Health Policy, 2(8). com/content/2/1/8. Accessed November, Roth, A., & Fonagy, P. (1996). What works for whom? A critical review of psychotherapy research. New York: Guildford Press. Trauer, T. (2004). Consumer and service determinants of completion of a consumer self-rating outcome measure. Australasian Psychiatry, 12, 48/54. Wierzbicki, M., & Pekarik, G. (1993). A meta-analysis of psychotherapy dropout. Professional Psychology: Research and Practice, 24, 190/195.

8 40 B. M. Bewick Appendix: Methodology and data analysis SOURCE The CORE National Research Database for Primary Care Psychological Therapy and Counselling Services Data set originally contained clients, 637 practitioners across 34 services Omissions made where basic demographics (age and gender) were not available n = 107 Remaining cases n = Remaining cases where age was under 16 or over 65 were omitted (n = 980) Remaining cases n = Clients receiving something other than individual therapy were omitted (n = 618) Final total dataset n = Figure A1. Flow diagram of data cleaning process. Table AI. Percentage of CORE-OM completed pre- and/or post- therapy by gender at the client level. Time of CORE-OM completion Male Female Overall sample N (%) N (%) N (%) Pre- and post- therapy 3300 (34) 8627 (39) (38) Pre-therapy only 4659 (48) 9763 (44) (45) Post-therapy only 147 (1) 383 (2) 530 (1) No assessment completed 1624 (17) 3349 (15) 4973 (16) Total 9760 (100) (100) (100) Table AII. Descriptives and benchmark figures for the percentage of CORE-OM completed at the service level. Time of CORE-OM Completion Pre-and post- therapy Total pre- therapy Total post-therapy Mean Standard Deviation Minimum Maximum th Percentile (95% CI) 22 (7.3 to 34.2) 77 (64.6 to 90) 24 (7.6 to 35.0) 50 th Percentile (95% CI) 38 (31.3 to 48.3) 93 (88.3 to 96.4) 38 (32.8 to 50.2) 75 th Percentile (95% CI) 53 (41.7 to 62.5) 97 (94.5 to 97.1) 54 (42.4 to 63.4)

9