EVALUATION OF UNITINGCARE REGEN CATALYST PROGRAM

Transcription

1 EVALUATION OF UNITINGCARE REGEN CATALYST PROGRAM Final Evaluation Report July June 2012 Final Report Submitted 21 st November 2012 Caraniche Research, Development & Projects 1

2 Authors: Melanie Kiehne & Matthew Berry (2012). Research, Development & Projects. Caraniche Pty. Ltd. 2

3 Table of Contents Executive Summary... 5 Introduction... 9 Purpose and scope of the evaluation... 9 Purpose of this report... 9 Evaluation design Evaluation questions Limitations of the evaluation Methodology Data streams Data collection Consent Ethics Structure of the report Part 1: BACKGROUND & DEVELOPMENT Chapter 1. Context for the pilot Policy decisions Steps and timelines Chapter 2. Overview of the Catalyst program Deliverables Program specifications Target population Part 2: FINDINGS: Program outcomes Chapter 3. The Program Model Program content Strengths Appropriateness and capability of the intervention Strengths Challenges and limitations Recommendations Strengths Challenges and limitations Recommendations Challenges and limitations Recommendations Accessibility of the service Strengths Challenges and limitations Recommendations Chapter 4 Continuous and coordinated care Strengths Recommendations Chapter 5 Resources Infrastructure and program resources Workforce Program management and review processes Strengths

4 Challenges and limitations Recommendations Chapter 6 Episode of care targets Strengths Recommendations Chapter 7 Catalyst within the broader AOD sector Stakeholder feedback Program referrals Strengths Challenges and limitations Recommendations Part 3: FINDINGS: Client Outcomes Chapter 8. Client profile Age, gender, and ethnicity Marital status and household Regions where client live Employment status Literacy Substance use Mental health and ABI Strengths Challenges and limitations Recommendations Chapter 9: Client Outcomes Significant treatment goals Were outcomes achieved in an appropriate timeframe? Strengths Recommendations Long term maintenance of treatment goals Strengths Recommendations Chapter 10. Client Experience Strengths Recommendations

5 Executive Summary This is the final summative evaluation report for the Catalyst program at ReGen, a pilot for an alcohol non-residential rehabilitation program, funded by the Department of Health to run for three-years (June 2009 to June 2012). The purpose of the report is to present the findings and recommendations of the evaluation with regard to program and client outcomes according to criteria stipulated by the department. This report presents findings from the inception of the evaluation in May 2010 through to June The Catalyst pilot program was designed to implement Action 1.7 of Restoring the Balance - Victoria s Alcohol Action Plan (VAAP), which proposed the establishment of a medium-intensity community-based rehabilitation model (that) will provide an additional 15 places for people with alcohol problems exiting hospital or withdrawal services. At the time of its implementation the pilot program was the only publicly funded, intensive, non-residential AOD rehabilitation program in Victoria, and was provided as a state-wide service located in the North-West Metropolitan region. The Catalyst model is designed as an alternative treatment option for those Victorians for whom residential services are an unsuitable or inaccessible post withdrawal option, but who require more intensive support than individual AOD counselling. The Catalyst model involves a six-week structured program of compulsory and optional educative and therapeutic units, as well as a voluntary aftercare component, Momentum, which is a weekly semi-structured support group. Development of the model was guided by a review of the literature. Cognitive behavioural therapy and motivational enhancement therapy were selected as the basis for the compulsory components of the program as they were supported by the literature and accepted within the field as effective AOD treatment approaches. Optional units were included on the basis of clinical wisdom, for the purpose of addressing a wide range of client needs and for increasing client satisfaction and retention. The evaluation found that the Catalyst model is appropriate in meeting the needs of suitable clients within the target population group and effectively meets a gap in service provision that previously existed within the Victorian AOD sector. The suitability criteria for the program are relatively narrow, however, and therefore the Catalyst model cannot be considered as an alternative to existing treatment options such as residential treatment or individual AOD counselling. To be suitable for the program clients need to be self-motivated, capable of selfmanagement, have at least a moderate level of intellectual functioning, and have reasonably stable life circumstances. Catalyst is not an appropriate treatment option for clients with poor English language skills, significant cognitive difficulties, unstable accommodation or unmanaged serious mental health issues. 5

6 Over the course of the three-year pilot period 22 program cycles were run and a total of 272 clients were admitted, reflecting 82% of the initial target of 330 set by the department in the request for tender. Of the 272 clients that commenced Catalyst, 190 completed the program (i.e. were present in week six), equating to a completion rate of 70%. This is a particularly high completion rate compared to completion rates described in the research literature for similar programs, and therefore represents a very positive outcome for the program. The data gathered through the evaluation and presented in this report demonstrates that the Catalyst program is capable of helping clients to achieve a range of positive treatment outcomes, as outlined in their individual treatment plans, including abstinence or reduced alcohol use, better management of triggers and a reduction in risk behaviours, and improved health and wellbeing. Around one quarter of Catalyst clients also reported a reduction in drug use following participation in the program, despite the fact that drug use is not specifically targeted by the program. Other outcomes reported by clients included improved family relationships, social and community re-engagement, and greater quality of life through significant lifestyle changes. Follow-up interviews conducted at six and twelve-months post program provided evidence to suggest that for some clients the skills and knowledge gained through Catalyst, as well as ongoing engagement with the Momentum aftercare program, were important factors in maintaining treatment gains long term. For suitable clients, the non-residential model of the Catalyst program has a number of significant benefits that are likely to play a role in improving treatment outcomes and enhancing the treatment experience. These include, the opportunity to learn and practice relapse prevention skills in the client s real world environment, the capacity to retain social connectedness and other key life responsibilities, and the ability to learn from lapses by maintaining engagement with the program when lapses occur. The emphasis on cognitive behavioural theory in teaching relapse prevention was also found to be appropriate in meeting the treatment needs of the client group. Other strengths of the program model include the peer learning that occurs through a group-based, mixed gender model, the opportunity to address an holistic range of treatment needs through the optional sessions available, and the capacity for clients to remain engaged with the service for ongoing treatment and support through the Momentum aftercare program or through repeat episodes of care with Catalyst. The report discusses several key challenges and limitations for the current model that may be important to consider as part of Catalyst s ongoing implementation and any future rollout, and recommendations are made with regard to these. A key limitation for the Catalyst model relates to access. As a state-wide, non-residential service provided from a single location, the program is not accessible to clients that do not live within a reasonable daily commute of the program site in the North-West Metropolitan region of Victoria. Data presented in this report demonstrates that Catalyst clients tended to reside in the inner Northern Metropolitan suburbs surrounding ReGen. It is therefore recommended that while the program model should continue to be funded as a treatment option in the Victorian AOD sector, at least one additional service should be established within the South East region, and efforts should continue to develop referral pathways for clients residing in neighbouring areas. Consideration might also be given to trialling a program in other areas with sufficient population density such as the Barwon South West region. It may be useful to consider a semi-residential model for rural regions of Victoria, with accommodation provided on program days and clients returning home on non-program days. This would, however, reduce some of the benefits of the real world learning approach of the program. Further modification of the program model may be required to meet current policy objectives with regard to family inclusiveness in AOD treatment. While the current model incorporates 6

7 family and relationship based components that have produced positive outcomes, these are primarily provided to the individual client, with less emphasis on actively addressing the support needs of family members and providing intervention to the client s family system, as a whole, in order to support the client s recovery. As a result, positive gains made through individual work with clients may fail to be maintained upon returning to unchanged social systems. Modifications to the program model may include providing family-based counselling and support as part of the core program, and expanding assessment and discharge processes to include family members and family-based needs. Consideration might also be given to the provision of funding for child care to improve program access for clients with fulltime parenting responsibilities, and the inclusion of a parenting skills component within the program. Processes for addressing post program support needs may also need to be reviewed in order to improve the likelihood that clients maintain treatment gains long term. While referrals for post program support are provided through Catalyst, there are no established follow-up processes, either general or assertive, to determine whether clients have engaged with referrals or require further support to maintain their treatment goals. Given the finding that relapse rates are highest in the first two months following program completion, it may be beneficial to consider an approach to post program support that includes assertive follow-up of discharge referrals to increase the likelihood that clients engage with referrals provided, and general follow up of all clients within two months of discharge to check on progress and provide linkage and support if required. Facilitated linkages to various post program support groups and services as part of the program model should also be considered a priority. Finally, the model of clinical supervision adopted for Catalyst during its inception may require review now the program is established, to ensure that the professional support and development needs of program staff are adequately met. The current supervision model includes one hour of structured individual supervision per six-week episode (i.e. seven hours per year), as well as access to ad hoc individual and group supervision to respond to issues that arise within the program on a daily basis. While the model of supervision was appropriate for providing the level and type of support required during the initial development stage of the pilot program, a more formal model of structured, regular, in-depth and reflective practicedriven supervision, including both individual and group supervision is now required. Conclusion The evaluation of ReGen s Catalyst pilot program finds that the program model developed is effective in meeting a service gap that previously existed in the Victorian AOD sector, and is an appropriate treatment option for suitable clients requiring intensive, non-residential post withdrawal support from problematic alcohol use. The program model, in its current form, should therefore continue to be funded as a primary treatment option, with consideration given to the findings and recommendations made in this report. In particular, future rollout of the program should include the establishment of at least one other program site in the South East Metropolitan region, and consideration for trialling a program in other areas with sufficient population density such as the Barwon South West region, for example, in Geelong. The further development of the Catalyst model to other drug types and/or target groups should also be considered. 7

8 8

9 Introduction Purpose and scope of the evaluation The purpose of the evaluation is to assess and report on the effectiveness and suitability of UnitingCare ReGen s (ReGen), formerly UnitingCare Moreland Hall, Catalyst program, a pilot for an alcohol non-residential rehabilitation program, funded by the Department of Health (hereon referred to as the department ) to run for three-years (June 2009 to June 2012). The evaluation, also conducted over three years, determines the extent to which the program specifications have been met (see chapter 2.2), presents findings in response to a set of evaluation questions proposed by the department, establishes the key strengths and challenges of the program model, and provides recommendations with regard to any future implementation of the program. The evaluation was conducted by Caraniche; a Victorianbased psychological services company with expertise in AOD treatment services, program development, and program evaluation. The evaluation was not commissioned to include a cost-benefit analysis of the program, nor an evidence-based comparison of the Catalyst program with other treatment types. Purpose of this report This is the final summative evaluation report for the Catalyst program at ReGen. The purpose of the report is to present the findings and recommendations of the evaluation with regard to program and client outcomes according to criteria stipulated by the department. This report presents findings from the inception of the evaluation in May 2010 through to June 2012, however client related data reflects only those clients who agreed to participate in the evaluation. Over the course of the evaluation, three annual reports have been produced (including the current report). The first report (2009 to 2010) looked at the initial development and implementation of the program and provided recommendations relating to any ongoing establishment issues. The second report (2010 to 2011) was an interim progress report, with recommendations designed to assist the program in achieving the key deliverables. Therefore, while the first two reports had an emphasis on process aspects of the program, the current report focuses on outcomes for the program, including outcomes for the program model, and outcomes for clients. 9

10 Evaluation design The evaluation employed an action research approach, which involves continuous feedback of the evaluation findings to the program developers and stakeholders. The purpose of this approach is to assist ReGen in meeting the program deliverables over the course of the pilot and to achieve the best possible outcomes for the program, clients and stakeholders. The evaluation was conducted in three stages. In the first year the emphasis was on evaluating and providing feedback in the annual report with regard to the development and implementation of Catalyst. The second year of the evaluation provided feedback with regard to the program s progress towards the deliverables and an interim progress report was produced. The final year provides a summative evaluation of the Catalyst program against the key deliverables as well as outlining the key strengths, challenges and recommendations with regard to the program model. Evaluation questions The evaluation is required to inform the Department of Health and the service provider ReGen of the extent to which the Catalyst program performs within the following framework of performance indicators (AIHW: National Health performance Framework, 2001): 1. Effectiveness: intervention achieves desired outcomes. 2. Appropriateness: intervention is relevant to the clients needs and based on established standards. 3. Responsiveness: service provides respect for persons and is client oriented. 4. Accessibility: able to provide a timely service irrespective of age, gender, cultural background and physical or other barriers. 5. Safety: avoidance of harm from service delivery or the environment in which the service is delivered. 6. Continuousness: ability to provide uninterrupted, continuous treatment over time. 7. Capability: capacity to provide a service based on skills and knowledge. 8. Sustainability: capacity to provide infrastructure, such as workforce, facilities and equipment and be innovative and respond to emerging needs. Accordingly the project encompasses an evaluation of the program model, including infrastructure and staffing resources, program outcomes, client outcomes, and client experiences, under eight broad areas. These are listed below, with specific research questions listed under each area. The research questions originated from the original tender for the evaluation of Catalyst, and were further developed following input from the department, ReGen, and Caraniche. Following each question a brief summary of the evaluation findings is provided, as well as a link to the chapter of the report where the question has been answered in more detail. 10

11 The Program Model 1. Capability, accessibility and appropriateness of the service model, with reference to the development of Catalyst over the three year period. Evaluation Question I. Was the intervention based on established standards such as evidence-based guidelines or accepted clinical practice? (Chapter 3.1) A comprehensive review of the AOD treatment literature was conducted by ReGen prior to commencing the pilot program and the development of the Catalyst model was informed by the findings of this review. The compulsory units of the program are based on cognitive behaviour therapy (CBT) and motivational enhancement therapy (MET), and both of these approaches are supported by the literature and accepted within the field as effective AOD treatment approaches. The optional units for the program were included on the basis of clinical wisdom, for the purpose of addressing a range of recognised client needs, as well as for increasing client satisfaction and retention. Evaluation Question II. How appropriate was the intervention in terms of meeting the clients needs? (Chapter 3.2) The Catalyst model, which is a structured, group-based, non-residential alcohol rehabilitation program, is appropriate in meeting the needs of suitable clients within the target population group whose treatment needs were previously not appropriately matched to either individual counselling or residential rehabilitation, the primary options available in the Victorian AOD sector. To be suitable for the program clients need to be self-motivated, capable of selfmanagement, have at least a moderate level of intellectual functioning, and have reasonably stable life circumstances. These criteria are identified at the assessment and screening process. Catalyst is not an appropriate treatment option for clients with poor English language skills, significant cognitive difficulties, unstable accommodation or unmanaged serious mental health issues. Of the 112 clients that responded to the satisfaction survey (40% of admissions to the Catalyst program), around three quarters felt the program met their expectations completely, while the remaining 25% felt the program met most or some of their expectations regarding treatment. Benefits of the model for the target population include: the opportunity to learn and practice relapse prevention skills in the client s real world environment, the capacity to retain social connectedness and other key life responsibilities, share learning experiences through a group format, and the ability to learn from lapses by maintaining engagement with the program when lapses occur. Clients and Catalyst staff felt that these were particular strengths of the community-based model that enhanced treatment outcomes for suitable clients. The emphasis on cognitive behavioural theory in teaching relapse prevention was also found to be appropriate in meeting the treatment needs of this client group, as evidenced by client feedback indicating that the skills and knowledge learned in CBT sessions were important factors in helping some clients to maintain their substance use goals six and twelve months after program completion. 11

12 Evaluation Question III. Was the treatment tailored to the clients specific needs, requests and prognosis? (Chapter 3.2) As a group-based program there are limitations in the degree to which treatment can be individually tailored to client s specific needs. These limitations include the extent to which program content can be customised to the needs of all clients, and the amount of time a group-based model allows for staff to work individually with clients. The inclusion of a range of relevant and diverse optional units in the program allows clients some choice and autonomy in developing a treatment plan that is suitable to their needs, and has the capacity to engage clients with different interests and learning styles. Evaluation Question IV. How appropriate and capable is the service to meet the needs of clients of all ages, genders, cultural backgrounds and with varying degrees of physical and/or other barriers? (Chapter 3.2) The Catalyst model is likely to be appropriate for meeting the needs of clients across the target age range (18 years and over), however some limitations may exist in meeting the particular needs of clients at different developmental stages, within a group-based setting. For instance, client feedback suggests that when clients of a particular age cohort are in the minority within a participant group (which was the case for clients in 18 to 30 year old age group), they may not receive the degree of peer support and identification that is key to the therapeutic process in a group-based model. The program has engaged with a relatively high proportion of females compared with other AOD treatment options, and this is a particular strength of Catalyst, demonstrating that the program model is seen to be appropriate for both males and females. However, retention rates are lower for females than males. As a mixed-gender, group-based program with no gender-specific component, the Catalyst model may not have the capacity to address the certain types of gender-specific needs with which some clients present. Clients of the Catalyst program were predominantly Anglo-Australian (85%). The Catalyst program is not likely to be appropriate for clients whose cultural values do not support participation in group-based or mixed-gender treatment, for instance, traditional Indigenous or Vietnamese cultures. There is no evidence to suggest that the Catalyst model is not appropriate for people with physical disabilities related to mobility, however, if the need exists, the capacity for people with significant sensory impairments to participate in and benefit from the program may need further investigation, given the amount of visual and auditory content and group discussion. Evaluation Question V. How readily are people able to access care without barriers of distance, discrimination, affordability and restriction of service? (Chapter 3.3) The Catalyst program is generally not accessible to clients that do not live within a reasonable daily commute of the program location. There is no cost to attend Catalyst and therefore for most clients there are no financial barriers to accessing the program, as clients are also provided with lunch and a metropolitan travel ticket. Clients from regional and rural areas of Victoria wanting to attend the program may incur travel (car or V-Line Train) and accommodation costs and this may serve as a barrier for some clients. 12

13 The program may not be accessible to clients with fulltime parenting responsibilities (i.e. those whose children are not yet of school age and who reside fulltime with the client) as there is no provision for childcare within the program model. Given that a community-based model may be a more appropriate option than a residential model for fulltime parents, particularly sole parents, the capacity for providing childcare is an important consideration. Evaluation Question VI. Was treatment provided in an uninterrupted and continuous manner across the continuum of care? (Chapter 4) There are processes in place to provide clients with continuous and coordinated care throughout their engagement with the program. For instance, clients are provided with information and telephone support prior to admission, and efforts are made to minimise wait times between completing withdrawal and commencing Catalyst. Prior to admission Catalyst staff ensure that clients are linked in with any additional supports they require, and collaborate with those supports throughout the client s engagement in the program. Referrals for post program support are made in week five of the program and clients are also provided with information about various peer support groups available post Catalyst, including Catalyst s after care program, Momentum. While there is evidence of referrals being provided for post program support, there is no evidence of follow-up, either general or assertive, of clients by Catalyst to determine whether clients have engaged with referrals or require further support to maintain their treatment goals. Resources: infrastructure and workforce Evaluation Question VII. How sustainable is the structure of Catalyst in its capacity to provide infrastructure in terms of workforce, facilities and equipment? Is it innovative and able to respond to emerging needs? (Chapter 5) ReGen have committed significant resources to the Catalyst program, with an emphasis on ensuring sustainability and the capacity to respond to emerging needs. These resources include a permanent and well-equipped program space including indoor and outdoor areas, a team of qualified and experienced full-time staff, supported by casual, and other support staff, and an efficient program management and review structure. A range of support services for the program are also able to be provided through ReGen, and this helps to ensure that clients are well integrated with the range of services they may require. As issues have been identified throughout the pilot, adequate and effective responses have been promptly implemented by Catalyst staff and management. Evaluation Question VIII. Do practitioners have requisite qualifications and registrations to provide adequate service to clients? (Chapter 5.2.3) Qualifications: All clinical staff working within the program, including fulltime and support staff, have the minimum standard requisite qualifications required to provide AOD services to clients in the Victorian sector. This includes the minimum competencies in AOD and Mental Health required to be recognised as dual diagnosis capable. Three out of the four clinical staff also have tertiary qualifications, including a Bachelor Degree in Social Work, a Graduate Diploma in Art Therapy and a Graduate Diploma in Drug and Alcohol, and a Post Graduate Degree in Mental Health Nursing (degree and registration obtained in the United Kingdom and also currently registered in Victoria). 13

14 Registrations: Professional registration is not required for the level of qualification specified for staff working within the program model. Evaluation Question: IX. Is adequate ongoing supervision and professional development provided? (Chapter 5.2.4) Professional development is provided to Catalyst staff in accordance with the policies of ReGen, and has covered the core AOD and mental health competencies, as a dual diagnosis capable organisation as well as a range of topics relevant to the program. All staff report that they are satisfied with the opportunities provided for professional development. At the request of Catalyst staff, the model includes an approach to clinical supervision that includes one hour of structured individual supervision per six-week episode (seven times per year), as well as access to ad hoc individual and group supervision to respond to issues that arise within the program on a daily basis. This model had been adopted because staff felt that during the development of the program, there was a greater need for immediate, reactive supervision complemented by a single, planned, reflective supervision session, than there was for more frequent (e.g. weekly) reflective-practice orientated supervision sessions, including structured group supervision. The model of supervision was appropriate for providing the level and type of support required by the clinical team during the initial development stage of the program model. As the program model has now matured and is more established, there is likely to be a need for a more formal model of structured, in-depth and reflective practice-driven supervision, including both individual and group supervision. Program Outcomes 1. The meeting of contractual episode of care targets. Evaluation Question X. Were the contractual episode of care targets met? (Chapter 6) The episode of care target set for Catalyst by the department in the original submission was 110 per annum, totalling 330 episodes of care over the three-year pilot phase. Between July 2009 and June 2012, 272 clients commenced Catalyst, reflecting 82% of the original target. Of the 272 clients that commenced Catalyst, 190 completed the program (i.e. were present in week six), equating to a completion rate of 70%. This completion rate compares very well with completion rates reported in the literature for other voluntary, community-based outpatient AOD treatment programs (e.g. 23% reported by Wickizer et al, 1994 and 46% reported by SAMHSA, 2009). 2. The usefulness and connectedness of Catalyst as a component of the broader AOD service sector. Evaluation Question XI. Was the treatment provided as a useful component within the broader AOD service sector? (Chapter 7) The Catalyst program was designed to address a long-identified gap in AOD service delivery in Victoria and the model that has been developed meets this gap very well. This has been recognised by the broader AOD sector as evidenced by positive feedback from clients and other stakeholders, and recognition in the form of a national award for Excellence in Treatment and Support (National Drug and Alcohol Awards, 2011), and as a finalist at the

15 Victorian Public Healthcare Awards for Delivering Innovative and Integrated AOD and Mental Healthcare. Referral data suggests that the AOD sector may not be engaging with Catalyst to the extent that might reflect client need for this program. According to the data provided to the evaluation, the majority (62%) of all referrals to Catalyst came from ReGen s other services, while referrals from AOD services other than ReGen accounted for only 16% of all referrals received for the program. Half of Catalyst clients resided in the inner northern metropolitan municipalities of Moreland, Darebin, Banyule, Yarra, however disproportionately fewer clients were referred from equally proximate municipalities in the inner west. Client Outcomes 3. Achievement of significant treatment goals outlined in clients individual treatment plans, including reference to client status on follow-up. Evaluation Question XII. Were the desired outcomes achieved at discharge as outlined in the client s individual treatment plans? (Chapter 9.1) The data demonstrates that the Catalyst program is capable of helping clients to achieve a range of positive treatment outcomes, as outlined in their individual treatment plans, including abstinence or reduced alcohol use, better management of triggers and a reduction in risk behaviours, and improved health and wellbeing. Of the 112 clients who provided responses to the satisfaction survey, the majority (around 97%) felt the program had helped them to make the changes they wanted to make. Of the clients that reported some level of dissatisfaction with the program in terms of meeting their goals, some felt the program was not long enough and others felt that more one on one time was needed. No unmet needs were identified by clients who have attended focus groups. Of the 70% of program admissions who completed the program, the majority reported meaningful improvements across each of the six key areas, with the greatest impacts occurring in the areas of AOD use, mental health and social functioning. At admission to the program poly-substance use was reported by 43% (n = 49) of clients who participated in the evaluation and completed Catalyst, and of those clients two thirds (n = 32) reported that they were abstinent upon completion of the program and 14% (n = 7) reported a reduction in use. This may suggest that participation in Catalyst has benefits for addressing drug use, as well as alcohol use. Evaluation Question XIII. How effective was the intervention in terms of achieving desired client outcomes in an appropriate timeframe? (Chapter 9.2) For suitable clients, the Catalyst program is an effective intervention approach for achieving positive outcomes in a relatively short time frame. This may be attributed to a range of factors including the intensive nature of the program, the range of treatment and support options provided through the program, the mix of theoretical and experiential learning, and perhaps most significantly, the opportunity to integrate this learning through immediate application of newly acquired skills and knowledge in real world settings. 15

16 Evaluation Question: XIV. Are case-level outcomes post-treatment maintained at sixmonth and twelve-month follow-up? (Chapter 9.3) The response rate for follow up interviews was low and therefore outcomes presented in the report cannot be generalised to all Catalyst clients. Of the 96 clients that were eligible to participate in follow up interviews at six months (clients needed to consent to participate in a follow up interview and complete at least two weeks of the program), 45 (47%) completed interviews. Of the 59 clients that were eligible to participate in follow up interviews at twelve months (clients needed to have agreed to participate, have completed at least two weeks of the program and have their follow up interview fall within the timeframe of the evaluation), 17 (29%) completed follow up interviews. For many of the clients that responded to follow-up interviews, positive outcomes that were achieved during Catalyst were maintained six and twelve months post Catalyst. For instance, at six months follow up 35% of respondents had maintained their drinking goals (abstinence or controlled drinking), while a further 44% had lapsed but were abstinent at the time of the interview. At twelve months follow up, 47% had maintained their drinking goals and a further 47% had lapsed but were abstinent at the time of the interview. Many clients felt that, as a result of participating in the program, their relapse management skills were significantly improved. With regard to health and wellbeing goals (i.e. mental and physical health, social functioning), 90% or more of the clients who responded to interviews reported that they had either maintained or further improved on positive changes made during Catalyst. Client Experiences 4. General service quality considerations and compliance with the standards set out in Shaping the Future The Victorian Alcohol and Other Drug Quality Framework (April 2008). Evaluation Question: XV. Is the service consumer focused? Does the service provide a treatment in which the client is respected and confidentiality and autonomy are maintained? (Chapter 10) Clients reported that their participation in the Catalyst program was a positive experience. In particular, clients found the program and its staff to be welcoming, respectful, client-centred and capable of meeting their needs and expectations regarding treatment. Clients also felt that their confidentiality and autonomy were maintained throughout their participation in the program. Observations of systems and processes in place also reflects this. 5. Key strengths of the Catalyst model. 6. Any limitations or challenges of the program model within the required objectives and standards. 7. Recommendations concerning any future implementation of the Catalyst program model. Strengths, challenges and recommendations are discussed throughout the report. 16

17 Limitations of the evaluation Certain limitations with regard to the evaluation process and outcomes are important to note. Evaluation methodologies, in general, are less tightly controlled than research methodologies and therefore findings presented in this report with regard to the effectiveness of the Catalyst model can only be interpreted within the context of the current pilot program. To determine the effectiveness of any future rollout of the Catalyst model, further evaluation would need to be undertaken. The evaluation was not commissioned to conduct a comparative analysis of the Catalyst model with other existing treatment models, and therefore any comparisons of treatment models discussed within the report are anecdotal or inferential only. Client post-program outcome data is reported only for those clients who, i) consented to participate in the evaluation, and ii) completed the program. Client outcome data collected at six-months and twelve-months post Catalyst is only reported for those clients that i) consented to participate in the evaluation, ii) completed at least two weeks of the program, iii) were contactable, and iv) agreed to provide an interview. As a result of these criteria, the data presented in the report is likely to be skewed towards more positive treatment outcomes. Methodology The evaluation will answer the proposed research questions on the basis of data collected on both an episodic and a calendar basis. Data streams Data collected each episode by ReGen includes: Pre and post psychometrics (WHOQoL-BREF; Situational Confidence Questionnaire) Client individual treatment plans SWITCH data ( episode form ) Satisfaction survey Client completion survey Discharge summary Client focus group Client questionnaires at six-month and twelve-month follow-up (collected by Caraniche) Data collected on a three-month or annual basis by Caraniche includes: Clinician s quarterly focus group Clinician s annual interviews Annual interviews with managers Annual interviews with stakeholders (Department of Health and other services) Demographics of the target group 17

18 Demographic data will be collected through the discharge summary and the episode form as recorded in SWITCH. These data comprise: Gender Age Ethnicity Accommodation Referral Source Pharmacotherapy Injecting drug use Concurrent conditions at commencement Marital status Employment status Living status Current legal status Substance related disorder Primary and secondary substances Concurrent conditions at termination Referral status/destination Data collection A pilot of the data collection tools and processes was conducted in December 2009, and submission of the ethics application to the Department of Health Ethics Committee was made in January Modifications and further information were requested and final ethics approval was granted in April Caraniche began program related data collection in January 2010 and collection of client related data began in May 2010, following ethics approval. Consent Catalyst clients were informed about the evaluation and its purpose by the Catalyst clinicians on the first day of the program (as part of the introduction and Group Guidelines session). The Project Coordinator from Caraniche then visited ReGen on the second day of the program to obtain clients written consent to participate in the study. Clients were read the Participant Information and Consent Form and provided with an opportunity to ask questions. Clients were informed that participation in the evaluation was entirely voluntary and were given the contact details of the evaluators for the purpose of asking questions at a later date or withdrawing their consent. Clients were asked to give consent for the following purposes: to allow ReGen to pass on information collected about them during their time in Catalyst to the evaluators to allow the evaluators to contact them six-months and twelve-months after completing the program for the purpose of obtaining follow-up information to allow the evaluators to contact a nominated friend or family member for the purpose of obtaining follow-up information in the event that the client could not be contacted Clients were informed that they could give consent to all, some, or none of these. Original copies of all consent forms are stored securely by Caraniche for seven years and all clients were provided with a copy of the Participant Information Form and signed Consent Form. At the time of this report 168 clients had consented to participating in the evaluation. This equates to 62% of all clients that commenced Catalyst over the three-year period (N = 272). 18

19 The evaluation sample is smaller than the total number of Catalyst clients due to the fact that the evaluation commenced approximately 10 months after the launch of the Catalyst program, as well as a small proportion of clients that declined to participate (actual number not known). Ethics An application for ethics approval was submitted by Caraniche to the Department of Health Human Research Ethics Committee (DH HREC) on 19 th January 2010 and ethics approval was granted on 8 th April. Structure of the report The current evaluation report focuses on outcomes for the pilot program, including outcomes related to the Catalyst model in terms of its effectiveness and suitability for the target population and for the Victorian AOD sector, and outcomes for clients in terms of the efficacy of the program in helping clients to achieve and maintain their treatment goals. Therefore the strengths, challenges and recommendations cited throughout the report relate to program and client outcomes. The previous two reports produced as part of the three-year external evaluation have a greater emphasis on process factors, providing an evaluation of program development and implementation. While the development and implementation of the program are summarised in this report, readers interested in a more indepth discussion of process factors are directed to the 2010 and 2011 interim reports. Part one of the current evaluation report covers the background and development of the Catalyst pilot program. In chapter one the policy context for the development of the pilot is discussed, followed by the steps and timelines for the initial development and implementation of the program and the evaluation. Chapter two presents an overview of the deliverables and objectives for the program, as well a description of the target population. Part two of the report presents the evaluation outcomes with regard to the Catalyst program model. In chapter three the program model is evaluated, including the Catalyst program and the aftercare program, Momentum, which was developed as part of the Catalyst model. A discussion of the eligibility and suitability criteria, and considerations regarding accessibility are also presented. Chapter four looks at how the Catalyst model provides clients with continuity of care and collaborative care. A discussion of the resources required to deliver the Catalyst program is provided in chapter five, and chapter six presents data and findings with regard to the number of episodes of care achieved over the course of the pilot. Chapter seven looks at the extent to which Catalyst is seen as a useful and appropriate treatment option within the Victorian AOD sector. Part three presents evaluation findings for the clients of the Catalyst program. A demographic profile is presented in chapter eight, including a discussion of treatment needs and the degree to which these have been met by the Catalyst program. Client outcomes, including the achievement and long term maintenance of significant treatment goals, as well as additional benefits of participation, are provided in chapter nine. Finally, chapter ten presents data obtained through client satisfaction surveys and other client feedback channels, to provide a description of the client experience of the Catalyst program. Strengths and challenges are discussed throughout the report, and key strengths, challenges and limitations relating the program model, as well as recommendations regarding any future implementation of the program model, are highlighted at the end of each chapter. 19

20 Part 1: BACKGROUND & DEVELOPMENT Part one describes the initial conceptual background, development and implementation of the Catalyst program, a community-based alcohol rehabilitation program piloted over three years (2009 to 2012) in the North-West metropolitan region of Victoria. Chapter 1. Context for the pilot 1.1 Policy decisions ReGen s Catalyst program implements Action 1.7 of Restoring the Balance - Victoria s Alcohol Action Plan (VAAP), which proposed the establishment of a medium-intensity community-based rehabilitation model (that) will provide an additional 15 places for people with alcohol problems exiting hospital or withdrawal services. Action 1.7 of the VAAP sits within the priority area of Restoring the Balancing for Families, through the provision of quality care for alcohol problems. The VAAP identifies a need for extended support post withdrawal, and a seamless transition from withdrawal to rehabilitation. The Catalyst program was developed to address this need. The VAAP is intended to complement the initiatives outlined in the Victorian Government s drug and alcohol policy framework document, A New Blueprint for Alcohol and Other Drug Treatment Services ( ), however this document has been succeeded by the current government s policy framework, as outlined in the document New Directions for Alcohol and Drug Treatment: A Roadmap. As a result, recommendations proposed in this report will reflect current government policy. 1.2 Steps and timelines A review conducted by the Department of Health (formerly Department of Human Services) of existing AOD service models in Victoria identified that there were limited choices for ongoing treatment post withdrawal, with residential rehabilitation or individual counselling the primary publicly funded options. Following this review the department released a request for tender submission in 2008 for a community-based alcohol rehabilitation pilot program, to be funded under the VAAP. The three-year pilot project was awarded to ReGen in January An initial literature review was conducted by ReGen for the purpose of submitting the tender application, and a more extensive review was completed once the tender had been 20

21 announced. The core elements of the program were initially developed by ReGen s Education and Training (E&T) team, utilising the findings of the literature review and the expertise of staff in the areas of AOD educational program development and delivery and clinical work. Three new clinicians were employed to work fulltime within the program, and were trained initially by the E&T staff that had developed the program. During the first half of 2009 ReGen undertook an extensive renovation of their premises at Jessie St, Moreland, in order to accommodate the new program. The program launch was planned for May 2009, however an extension was granted by the department to allow additional time for sourcing appropriate staff and resources, and the program was officially launched on the 29 th June The name Catalyst was chosen in November 2009 following a survey of staff and client suggested names. An external evaluation of the Catalyst program was a requirement in the original request for tender, and Caraniche was awarded the project in July 2009 with contracts signed in August. Caraniche joined the project steering committee in August 2009, which included representatives from ReGen, the Department of Health, and the Association of Participating Service Users (APSU). An evaluation working group was also set up and reported to the Steering Committee. The working group involved Caraniche and representatives from ReGen and the Department of Health, and met primarily in the first 12 months of the evaluation to oversee the development and implementation of the methodology. Chapter 2. Overview of the Catalyst program 2.1 Deliverables The call for submissions put forward by the department states that; the service will provide a non-residential program that will provide 15 places for people with alcohol problems who are exiting hospital or withdrawal services. The program will provide intensive group work and counselling over a four to six week period and also some out of hours support to prevent relapse. The provider will also deliver to the department a comprehensive program manual, including all relevant treatment tools. The following service deliverables were stipulated by the department in the call for submissions; Provide behaviour changing interventions to assist individuals to reduce or cease harmful alcohol use. Assist clients to establish healthier lifestyles free of problematic alcohol use. Provide a program of behaviour changing interventions on a non-residential basis. Assist clients to avoid and better manage relapse. 21

22 2.2 Program specifications The tender for the evaluation outlined the following service objectives for the Catalyst program. A brief summary of the objectives and the evaluation s findings are presented below, as well as links to more indepth discussion later in the report. Assessment and development of a comprehensive individual treatment plan (ITP), including a care coordination plan. Regular care coordination aimed at monitoring progress with AOD issues and treatment, re-establishing a pattern of work/study, reestablishing social connections, addressing health issues, living issues etc. Assessment and development of an ITP is provided for all clients in their first one-onone session with a Catalyst clinician. Clients are also required to have a care coordination plan in place prior to starting the Catalyst. The care coordination plan is to be the responsibility of the professional/agency referring the client to Catalyst, or, if self-referred, a Catalyst clinician will assume responsibility. Throughout the program and at discharge any additional care coordination needs of clients are identified by the Catalyst clinicians and appropriate referrals are made (see chapter 4). A therapeutic and educative group program focused on alcohol treatment and rehabilitation. The Catalyst program has been designed to include therapeutic and educative components aimed at alcohol treatment and rehabilitation (see chapter 3.1). Regular individual counselling and family-based interventions. Four individual counselling sessions are included in the program. These are provided as motivational enhancement sessions, rather than generalist counselling sessions, but can also include CBT, solution focused therapy and crisis intervention as appropriate to clients presenting needs. Clients are encouraged to maintain engagement with preexisting support services, or are linked in with supports, to provide any additional counselling that may be required during their Catalyst episode. Two sessions on Strengthening Relationships are included in the program (though attendance is not compulsory) and family members are invited to attend. The sessions are facilitated by a Catalyst team member and a family counsellor from ReGen. There are no prescribed individual family-based interventions offered within the program, however ReGen has a family services team available, and cross referrals have been made between the family service and the Catalyst program (see chapter 3.2). A program of rehabilitative group work and activities, possibly including job-seeking skills, living skills and parenting skills components. Rehabilitative group work and activities are the basis of the program and job-seeking skills and living skills are included, however, parenting skills have not been included. ReGen runs a Playgroup that includes parenting skills and Catalyst clients are informed about the group at information sessions, or in one on one sessions if staff feel it may be relevant. The Playgroup is limited in the sense that it is only available to parents of children up to five years of age. At the time of this report no Catalyst clients have accessed the group (see chapter 3.2). Client data presented in this report (e.g. around 18% of clients have their children living with them) suggests there may be a need to consider including a parenting skills component to the program. 22

23 Some time during the week will be left free for homework, including rehearsal of AOD related skills, job-seeking, health related appointments and activities, family responsibilities, social life etc. The Catalyst timetable includes program free periods (part and full days) for the above to occur. The program free days are concentrated towards the end of the six-week timetable to allow for gradual community reintegration by clients (see chapter 3.1). A follow-up session with each client at 6 and 12 months to determine progress in the client s rehabilitation and to offer appropriate linkages for further support. Data gathered at follow-up will be used in an evaluation of the program, which will be conducted after three years of operation. This data must be structured in such a way as to assist the evaluation. While 6 and 12-month follow-up interviews have been conducted by the evaluators, these have been conducted as a research activity for the purpose of providing evaluation data with regard to the maintenance of treatment outcomes, rather than for the purpose of determining the progress of individual client s rehabilitation and of offering linkages for further support, as this is deemed to be a responsibility of the treatment service and not an appropriate role for the program evaluators (see chapter 9.3). Clear and structured connections with a range of appropriate continuing care services that will further assist in rehabilitation, including access to alcohol pharmacotherapy. Clients already engaged with supports such as counsellors, psychologists, GPs or case managers, prior to commencing Catalyst, are encouraged to maintain those supports both during and after Catalyst. Connection with a range of additional continuing care services is offered to clients by Catalyst clinicians on a needs basis both during the program and at discharge (see chapter 4). Through Catalyst clients are directly linked with employment agencies (Jobco and CVGT), a financial counsellor, and post program peer support (Catalyst s weekly aftercare program, Momentum) and receive advice on pharmacotherapy as part of the adjunctive units offered in the program (see chapter 3.1). The outcomes of referrals (i.e. whether a client has been linked successfully with services and supports) are followed up by ReGen for some, but not all clients. Referrals made early in the program, or as part of an individual treatment plan, are more likely to be followed up by staff than referrals made at discharge. Assertive referral was not standard practice at discharge of Catalyst. (see chapter 4). Some out-of-hours support to prevent relapse during the treatment program. Out of hours support is not provided through Catalyst or ReGen. Clients are provided with the contact details for Directline and are encouraged to utilise this service for immediate support outside of Catalyst (see chapter 4). A program manual and facilitators guides. A program manual and facilitators guides have been developed for the program and are currently in use (see chapter 5.1.1). 23

24 2.3 Target population Because Catalyst is funded under the VAAP, the target client group are alcohol dependent adults (18 years and above) who have recently undergone a withdrawal program and wish to participate in rehabilitation in order to further consolidate positive changes made 1. The program is provided as a state wide service and therefore all Victorian residents are eligible to participate. As a non-residential program Catalyst is designed for people who prefer to participate in structured alcohol rehabilitation in a community setting. This group includes those for whom family, work or other commitments preclude further residential treatment. To be accepted into Catalyst clients must have recently completed a withdrawal from alcohol and have the preparedness to abstain from alcohol for the duration of the program. The length of time since withdrawal, whether or not clients have remained abstinent in that time, and motivation with regard to treatment, are important considerations in the selection of appropriate clients. The focus upon alcohol as opposed to other drugs was a result of the funding being drawn from the VAAP, and does not exclude poly-substance users, however it would serve to exclude those who did not report a significant alcohol-related issue. 1 Department of Health (formerly Department of Human Services). (2008). Request for tender submissions for an Alcohol Community Rehabilitation Program. State Government of Victoria. 24

25 Part 2: FINDINGS: Program outcomes Part two presents evaluation findings, including: the program model, client eligibility and suitability criteria, accessibility of the program for the target population, the capacity to provide continuous and collaborative care, resources required to deliver the program, episodes of care achieved over the course of the pilot, and the extent to which Catalyst is seen as a useful and appropriate treatment option for the Victorian AOD sector. Chapter 3. The Program Model The Catalyst program is the only publicly funded, intensive, non-residential AOD rehabilitation program in Victoria, and is a state-wide service located in the North-West Metropolitan region. The program is designed as an alternative treatment option for those Victorians who are unable, or unwilling, to attend residential services, but who require a more intensive form of rehabilitation than AOD counselling. Participants therefore attend the program while continuing to reside in the community, and maintain their own social connections, roles and responsibilities. Each program cycle accepts 15 participants and runs over six weeks on a weekday timetable, between the hours of 9 am to 5 pm. In the first few weeks clients attend every day, and in the final few weeks the attendance days are gradually reduced to aid transition back into everyday life. The program format is structured, group-based and combines psycho-educational and therapeutic elements. The Catalyst model also includes an aftercare component. The Momentum aftercare program was developed to provide ongoing support for Catalyst clients following completion of the program and is run weekly at ReGen and facilitated by a rotating roster of Catalyst clinicians. The program continues the use of cognitive behavioural and motivational enhancement strategies to help clients maintain their drinking goals by coping with urges, solving problems related to their substance use and working to establish lifestyle balance. The sessions are therapeutic and more process oriented than support groups, however they are also client directed and variable in terms of structure and what is covered. Participation in Momentum is voluntary however clients are encouraged to attend at least one session the week after finishing in the Catalyst program. The following section presents findings in response to the evaluation questions regarding the Catalyst program model. 25

26 3.1 Evidence-based best practice Evaluation Question I. Was the intervention based on established standards such as evidence-based guidelines or accepted clinical practice? A comprehensive literature review was conducted by ReGen employee, Mr Trevor King, the former Deputy Director of Turning Point, prior to launching the pilot. Each of the compulsory units for the program was selected and developed on the basis of this review. There are three compulsory psycho-educational units; Coping Skills, Mood Management and Anger Management, which are based on the principles of cognitive behavioural therapy (CBT), and an individual counselling component based on motivational enhancement therapy (MET). A description of each of the compulsory units is provided in Table 1. CBT and MET approaches are well regarded, both in the literature and in practice, as effective interventions for working with AOD clients, and therefore meet the criteria set out by the department for high quality, evidence-based interventions in Victorian AOD service settings 2. Table 1. Compulsory units N Sessions Coping Skills 10 Description Based on the principles of cognitive behavioural therapy (CBT) Clients develop an understanding triggers for alcohol use and learn relapse prevention strategies 1:1 Motivational Enhancement 4 Uses motivational interviewing techniques to assist clients in resolving any ambivalence they may have around engaging in treatment and changing their substance use behaviour Mood Management 2 Uses CBT techniques to help clients develop skills in managing negative moods and increasing positive mood states Clients learn to identify signs of depression and anxiety and to understand how negative self-talk and thinking errors contribute to negative mood Anger Management 2 Uses CBT techniques to teach clients how to identify triggers and thinking errors that lead to anger and to help clients develop strategies for avoiding or managing their anger Clients develop a greater awareness of the link between anger, aggression and negative consequences such as feelings of remorse, damaged relationships and criminal charges In addition to the compulsory CBT and MET units, the Catalyst program includes a range of optional units. Each of the optional units was selected based on their capacity to enhance client satisfaction and retention in the program by providing interesting and diverse options, and to address recognised client needs with regard to overall health, wellbeing and lifestyle factors which are often neglected in people with problematic alcohol use. Descriptions of the optional units are provided in Tables 2 to 4. Research evidence for the inclusion of various adjunct therapies and educational components in AOD rehabilitation programs is somewhat 2 Ibid. 26

27 limited. This, however, is due to a lack of studies, rather than evidence contradicting their effectiveness. The evidence derived from client feedback and treatment outcomes as part of this evaluation does provide support for the inclusion of these units of Catalyst. For instance, when asked about the benefits of the optional sessions, comments from clients have included, The nutrition is a really nice component to the course. The way it s presented really makes sense and helps you see that your recovery isn t just about controlling your thoughts and cravings, it s about being healthy, Mindfulness and yoga have helped me to deal with my triggers, and I m a very visual thinker so the Art Therapy really worked for me. I was really surprised just how much it affected me. Table 2. Optional Health and Wellbeing units N sessions Description Understanding Pharmacotherapy and Smoking Cessation 1 The session is facilitated by a nurse from ReGen The purpose of the session is to provide clients with information about pharmacotherapy and to develop an awareness of the options available to them Gym / Yoga 4 Clients are provided the opportunity to either attend a local gym where they have access to the exercise equipment, or to participate in yoga sessions These sessions are seen to have benefits for both physical and mental wellbeing Mindfulness and Relaxation 5 Sessions are facilitated by one of the Catalyst clinicians The purpose of these sessions is to teach clients techniques in relaxation that they can use as alternative coping methods in times of stress, or to use regularly to prevent stress and anxiety Mindfulness and relaxation are also used as a tool for mood and craving management Gardening NA Gardening is not part of the official program timetable, but the opportunity to spend time in the program s kitchen garden is offered as an optional leisure activity for clients who are interested The benefits of gardening include its ability to provide a source of therapy to individuals, to promote socialisation among client groups, and to offer ongoing connection between current and previous clients, who are welcome to visit the program to work in the garden 27

28 Table 3. Optional Life Skills units N sessions Nutrition 4 Description A nutritionist was contracted to design and deliver nutrition workshops specifically for people recovering from alcohol addiction Sessions involve theoretical learning about nutritional needs, including how food can impact on mood and cravings, and hands-on cooking lessons, which provide an opportunity to put what is learned into practice Finance Management 2 Developed and delivered by a financial counsellor employed by UnitingCare Sessions focus on financial recovery and management and cover topics such as dealing with debts and fines and accessing financial support ReGen have incorporated a finance pre-screen to the Catalyst pre planning session and clients who are identified as being likely to benefit from the finance management sessions are encouraged to participate Employment Assistance 2 Jobco and CVGT, are currently contracted to deliver job assistance sessions in week five of the program Table 4. Optional Therapeutic units N sessions Art Therapy 5 Drumming 5 Description Art therapy combines traditional psychotherapeutic theories and techniques with a psychological understanding of the creative process, to help clients to express their thoughts and emotions The aim of the approach is to help clients increase insight and judgement, to cope better with stress, to work through traumatic experiences, to build self-esteem and to reconnect with pleasant feelings Benefits associated with therapeutic drumming include an increase in social confidence and connection, expression of emotion through creativity, experiencing pleasure in-themoment, and a release of stress and anxiety Strengthening Relationships 2 Sessions aim to engage family members and friends in the program and to help clients and their significant others to understand the impacts of problematic alcohol use on their personal relationships and to provide them with skills for improving those relationships The sessions are held after hours (from 6pm to 8pm) and delivered by one of the Catalyst clinicians and a family therapist from ReGen 28

29 Strengths CBT and MET approaches, which form the basis of the compulsory program content, are well supported by evidence, and are accepted as effective intervention approaches in the field of alcohol and drug treatment. 3.2 Appropriateness and capability of the intervention Evaluation Question II. How appropriate and capable was the intervention in terms of meeting the clients needs? The findings of the evaluation demonstrate that the Catalyst model is appropriate in meeting the treatment needs of suitable clients and has a number of significant benefits as an option for post withdrawal support. The appropriateness and capability of the Catalyst model in meeting a range of client treatment needs are discussed in this chapter, including a description of client suitability criteria for the program model. The rationale for developing a non-residential treatment model was to provide an intensive rehabilitation option for people with problems related to their alcohol use, who were unable to attend a residential program, or for whom out of preference, a community based program would be a preferable modality for treatment. This includes, for example, people with family and work responsibilities, or those who would be at risk of losing income benefits or accommodation (e.g. renters, boarders etc.) as a result of attending a residential facility. This target population reflects those clients whose treatment needs were not appropriately met by individual AOD counselling or residential rehabilitation, which were the primary treatment options available prior to Catalyst. The program model, therefore, effectively fills a gap that previously existed in the Victorian AOD treatment sector. Over the course of the evaluation client feedback has identified several other client subgroups for whom the community based model is a more appropriate treatment option than the residential model. For instance, a number of clients have indicated that they would not attend a residential program because they felt the experience would exacerbate their anxieties, and preferred to maintain familiar supports and surroundings while attending treatment. For instance, one client stated, I would have felt uncomfortable in an unfamiliar environment and with so many unfamiliar people around me. I need my own space to manage my mental health. Many clients also commented that they preferred the emphasis on self-responsibility and the opportunity to practice relapse prevention skills throughout the program, suggesting that this approach better prepared them for maintaining their goals in the long term, as one client commented, I ve done it (residential rehab) before and I couldn t stand it, at Catalyst it s like work and the onus is on you. To be suitable for a community based program such as Catalyst, clients do need to be able to manage the need for self-responsibility with regard to their substance use and program attendance. Clients also need to be motivated to commit to the 6-week program, and capable of participating in a group therapeutic format. Clients who may not be suitable for the program include: those with a high degree of instability in their lives, including serious mental illness that is not being managed, unstable accommodation or chaotic personal lives. It is also 29

30 unlikely to be suitable for people with significant cognitive difficulties such as ABI, or for those with low levels of English literacy due to the amount of written content in the program. Cognitive behavioural focus Client feedback indicates that the knowledge and skills learned in the CBT units; Coping Skills, Mood Management and Anger Management, have been significant in helping them to achieve and maintain their treatment goals. For instance, in program completion surveys clients consistently rated CBT or CBT-based units, particularly Mood Management, as the most beneficial aspects of the program, and in six and twelve-month follow-up interviews clients continued to rate CBT-based skills as the most important factors in helping them to maintain their goals. For instance, one client stated that The CBT was the most vital part of the program, and another commented The CBT really changed my thinking, and that has created the biggest change for me. Clients also appreciate the consistent reference to the CBT-based principles across various program units, and have indicated that this repetition is particularly effective in enhancing their learning. For instance, one client commented, The tools and skills we re learning, everything s related. Whatever we re talking about we keep coming back to the core stuff. It s really well integrated Holistic approach The Catalyst program model takes an holistic approach to rehabilitation from alcohol related problems, by incorporating a range of treatment components that are designed to address the various areas of need for clients. As described in Chapter 3.1, the program targets areas of health and wellbeing, life skills, as well as a diverse range of therapeutic interventions. Inherent in the inclusion of these adjunctive units is a recognition that rehabilitation and recovery requires an integrated approach to addressing primary treatment needs as well as health, social, economic and relationship needs. Learning in one s own environment One of the most significant strengths is the value for clients in learning relapse prevention skills while continuing to live in their own environment. The opportunity to put into practice new skills and knowledge at the end of every day, and to bring those experiences to the program the next day, greatly enhances the learning process. This is recognised by clients, as demonstrated in their feedback. For instance, one client stated that, Going home every day gave me the opportunity to practice what I was learning, which really helped me to understand what the program was trying to teach me. According to staff, this learning model increases the capacity for personal insight regarding triggers and coping behaviours, and helps to develop a sense of self efficacy. This is supported by feedback from clients, particularly those who have completed residential treatment programs in the past, who report that they are well prepared for life after treatment (as a result of participating in Catalyst), and have a greater understanding of their substance use issues and how to manage them. For instance, one client commented, Here, it s different. We re given the tools we need and we re able to work it out for ourselves. Maintaining social connectedness Client feedback has also suggested that the ability to participate in treatment while continuing to live in the community has several benefits for social and psychological wellbeing, including the opportunity to enhance social connectedness, as well as to maintain familiar social supports, surroundings and personal space, which can be a source of reassurance during the recovery process. Many people with problems related to their alcohol use experience 30

31 problems with social isolation and some clients have indicated that they felt a residential program would exacerbate this problem. For instance, one client commented, Ordinarily, I tend to shut myself off from people. Coming here helps me get out of the house and come out of my shell a bit. It reminds me that I have to keep getting myself out there and that it can really help me avoid the traps I ve fallen into in the past and another stated, I need contact with the outside world to stay sane. I found it hard to keep it together for a seven-day detox so I know I wouldn t last long at a resi(dential) rehab. Here, I feel calm and I can feel myself growing as a person. The availability of a non-residential, intensive treatment program may therefore be essential to ensuring that all Victorians have access to suitable post withdrawal support options. The extent to which maintaining social connectedness is a strength of the program model, depends to some degree on the level of supportiveness that exists within the client s social circle. For some clients, existing social connections may be unsupportive of their recovery and have a negative impact on the client s treatment. For these clients, maintaining social connectedness may be a challenge, rather than a strength of the community-based model. Lapses during treatment are used as learning experiences The community-based model also allows Catalyst to take a non-punitive approach to lapses and many clients have found this to be a strength of the program. Unlike residential programs, which commonly require clients to exit when lapses occur, Catalyst clients are encouraged to re-engage with the program following a lapse and to include that experience as part of their learning. The difference in approach lies in the fact that, when a client lapses in a residential setting the broader client group is directly impacted, whereas when a client of a nonresidential treatment program lapses the client group is protected from the direct impacts and therefore the therapeutic space is maintained. According to both staff and clients, this approach helps to avoid a sense of shame and stigma that is often attached to lapsing and increased by punitive responses. As one client reported, I had to drop out for a while because I had a lapse, but the beautiful thing about coming back was that I was welcomed back with open arms and positivity. It was seen as an opportunity to help me get back to where I was, not as a reason to punish me. I feel like I m in a much stronger position now. Clients instead learn to understand that problematic substance use is a chronically relapsing condition and that ongoing management strategies are an important part of relapse prevention. It may be argued that this approach to lapses better prepares clients to self-manage after formal treatment 3. In follow up interviews, many clients reported that their management of lapses had improved since participating in Catalyst. For instance, one client commented, I m more conscious of the lapses now, rather than just ignoring it and pretending it s not happening. I m not as afraid to ask for help. Group therapy and peer support Group therapy models have a number of significant benefits for people with substance use problems, and for some clients these benefits can lead to better outcomes than individual treatment models. Many people with problems relating to their alcohol use also have problems with social isolation, and may lack the social skills required to connect with their 3 Department of Health. (2012). New Directions for Alcohol and Drug Treatment: A Roadmap. Victorian State Government. 31

32 communities, which is essential in recovery. The group-based format of the Catalyst program has been identified by many clients as an important factor in helping them to re-establish social connectedness. For instance, one client commented, The socialisation has helped me a lot. I ve re-entered society by hanging around with these guys, and another stated, It gives us the opportunity to re-socialise ourselves with other people rather than just sitting at home with a bottle. Group therapy models can also enhance opportunities for self-examination, through sharing experiences and receiving feedback from others, and the benefits of this were recognised by some clients, for example one client commented, Being challenged in front of a group of people can be hard, but it has been good for me. Clients also valued the opportunity to relate with, and learn from others, which may be a factor in helping clients to deal with feelings of shame and loneliness that are common among people with alcohol related problems. For instance, one client stated, It s such a positive group of people. Even though we re very different people, we have a lot of shared experience and can really learn from each other, and another reported, The other day in the [Skills Practice] session, with the examples that were discussed, you could just see people all the way through saying, Yep, it s just like that for me too...we have very different lives but we have so much in common. Catalyst has fixed entry points and operates in six-week cycles. Admission is not permitted after the first two days, and this closed-group format of the Catalyst program model helps to facilitate the development of supportive relationships among clients. This has enhanced the therapeutic experience of clients, as evidenced by client feedback. For instance, one client commented, We ve started together, we ve got to know each other as we go and it s made it easier to open up to the group. Other processes incorporated into the Catalyst program model that facilitate the development of supportive peer relationship include the establishment of group guidelines (i.e. rules to be followed by the group) by clients on the first day of the program, encouraging clients to prepare and eat lunches together, Check In/Out sessions and Skills Practice sessions where clients share their experiences, and the inclusion of sessions that involve various non-verbal ways of interacting such as Drumming, Nutrition and Gardening. Family-based treatment needs According to the program specifications outlined by the department in the call for submissions (2008), the program model was intended to include regular family-based interventions and possibly a parenting skills component. The importance of addressing the family needs of AOD clients, through family-inclusive practice, is emphasized strongly in the VAAP (2008 to 2013), under which the Catalyst pilot program was funded, as well as the policy framework outlined in the New blueprint for alcohol and other drug treatment services (2009 to 2013). In particular, the Blueprint states that Treatment services need to actively identify parents in their services and recognise and respond to the family context for their clients. Interpersonal and family issues are incorporated through the individual s treatment program, and Catalyst partially meets the specification for family-based intervention, through the inclusion of the Strengthening Relationships sessions. This unit includes two sessions that are co-facilitated by a Catalyst clinician and a family counselor from ReGen s family services team. The sessions are group-based and clients are encouraged to invite family or friends, and this has typically included partners, parents, and adult children. According to the data available, 69 clients (51.5%) attended the unit, and 39 clients (29.1%) attended both sessions. Clients who attended have provided positive feedback indicating that the sessions have been beneficial in improving the level of support and understanding between themselves and their significant 32

33 others. For instance one client commented, sometimes, your partner thinks they know the reason you drink. It s an eye opener for them and they get a better understanding of what s really going on and another stated My husband gained a lot of knowledge about just what is involved in alcohol treatment. He could see that this isn t a waste of time. As with all group-based programs however, there are limitations in the extent to which the specific needs of individual families can be addressed in the sessions. Clients and their families who are identified as requiring further support are offered a referral to ReGen s family services, or other external services such as Relationships Australia. Further development of the program model may be possible to increase the capacity to provide family-inclusive interventions and support, for example by increasing the degree to which family members and supports are integrated into the program, including both assessment and discharge, and through proactive linkages and referrals offered to the families of clients with support needs. This may involve linking clients more assertively into family counseling and actively identifying family members with significant support needs related to their loved one s substance use, with a particular emphasis upon the needs of children. In feedback provided to the evaluation some clients have suggested that they would like more opportunity for their children, particularly teenage children, to be involved in the program. For instance, one client commented, I wanted my daughter to come to the session so that she could see how hard it is and how well I m doing. Seeing where it can all lead. It could have been a wake-up call for her and another stated, I know what my son has seen and I think that he needs to be involved in the healing process. If my Dad had done that with me, I know it would have made a big difference to me. If I had seen him admit that he had a problem, I might not have ended up where I am now. It may be of value, therefore, to consider how the parenting needs of clients, and the needs of their children, could be further addressed through the program. Parenting skills are not addressed within the program. The option for including a parenting skills component was suggested in the original tender specifications for the Catalyst program and considered by ReGen during the literature review stage, however it was decided that parenting skills would be best accommodated through linkages to parenting groups outside of the Catalyst program. Clients with preschool aged children are able to be linked in with the Family Playgroup Program, a parenting skills program provided by ReGen, and staff report that some cross referrals were made between Catalyst and the Family Playgroup Program. However, the addition of a parenting skills component suited to parents of children of all ages, may be an important consideration for the future given that around 17% of Catalyst clients were parents with their children residing in their home. In addition, clients who are sole or primary carers for young children may require assistance with child care arrangements to attend the program. While this need has not been identified by Catalyst clients, ReGen have suggested there may be potential clients who have not accessed the program due to the fact that child care support was not available. Internally provided aftercare support The availability of the Momentum aftercare program is a significant benefit of the program model. In particular, clients who have attended indicated in follow up interviews that they value the consistency of the approach between Catalyst and Momentum, the structure that the weekly group provides, and the opportunity for ongoing support from peers and program staff. For instance, clients have commented, Momentum has helped me to stay on track, I would not be doing so well without it, You feel supported to keep coming back, even after a lapse, and It (Momentum) reminds me where I was and why I don't want to go back there. 33

34 From the second year of the pilot, core members of the Momentum group have also played a role in the Catalyst program, as spokespeople, mentors and peer supports. For instance, Momentum participants have been invited to attend the weekly information sessions for prospective clients and referrers, to give a brief presentation of their experiences in the program. Momentum participants are also invited to visit the Catalyst program at the start and end of each episode to talk to the clients. Staff report that these have been positive initiatives as it can help to ease the anxieties of new clients, and provides a familiar linkage for clients as they transition from Catalyst to Momentum. In addition, by creating the opportunity for past, current and future Catalyst client to meet and share their experiences, there is greater potential for the development of a supportive client community. The Catalyst team leader reports that this sense of community is important because it, fosters ongoing engagement (with the program) and continuity of care for clients, and clients like to give something back and feel a sense of ownership for the program. It also provides new clients with a different perspective because they are hearing the experiences of other clients, their successes and their challenges. According to data provided to the evaluation, 54 Catalyst clients have attended the Momentum program, over 108 episodes of care. This equates to around 20% of all clients that have accessed Catalyst in the three year pilot phase (N = 268). Staff report that attendance at sessions ranges between four and ten clients, with the number of regular attendees fluctuating over the course of the pilot. These figures suggest that the uptake rate for Momentum, relative to the number of clients that have accessed Catalyst over the course of the pilot, is not high. As attendance records have not been consistently collected for Momentum it is not possible to report on the retention rate for the aftercare program. One limitation noted by Catalyst clients was that Momentum is run in the evenings only, and a number of clients interviewed at follow up indicated that they did not attend the program because they were not willing to travel long distances or use public transport at night. Opportunity to repeat Catalyst Clients are permitted to repeat the Catalyst program (i.e. participate in multiple episodes) and are eligible to do so on the basis that they would receive some benefit from participation in a second or third program cycle. ReGen report that clients do not need to have relapsed to be considered eligible for a repeat episode, allowing clients to access treatment at all stages of the recovery cycle. According to Catalyst staff, clients may have struggled in the first episode for a range of reasons (e.g. personal crises, lapses, poor health) and may be in a better position to engage with the program during their repeat episode. Some clients may not have been ready for change the first time around, or found the content or the experience itself overwhelming and failed to take on board the key learning s, as one client commented, I found I had better insight into what the program was doing so it was more helpful and another stated, The second time round I listened with different ears. I was determined I was going to succeed. Other clients may have achieved their treatment goals, but feel the need to refresh their learning and re-engage with treatment in order to maintain those goals long term. According to ReGen, allowing clients to repeat also has benefits for the program. ReGen have found that clients who repeat Catalyst are more likely to become advocates for the program as they develop a greater foundation of experience and knowledge about the program. This has been of benefit to new clients, as repeating clients have acted as peer supports and mentors. 34

35 Strengths The Catalyst model, for a structured, group-based, non-residential alcohol rehabilitation program, meets a gap in service provision that previously existed in the Victorian AOD sector. Participating in treatment while continuing to live in one s own environment provides the opportunity to practice new alcohol management skills at the end of every day and to bring those experiences to the program the next day, which enhances the learning process and better prepares some clients for managing their alcohol use post treatment compared to residential programs. Clients maintain their existing social supports and connection with the community, which can be particularly important for clients with social anxieties, and problems with social isolation. The group-based format helps clients to improve their social skills and social connectedness, enhances self-examination through the sharing of experiences and feedback provided by others, and encourages the development of supportive peer relationships through the shared treatment experience. Lapses can be treated as learning opportunities in a community based model, because clients do not need to be exited from treatment as is often required in residential models. This reduces the shame and stigma associated with lapses and may therefore lead to improved management of lapses in the long term. The inclusion of Momentum, a weekly after-care program for Catalyst clients, provides ongoing post-program support that is compatible with the Catalyst treatment approach, and is an important resource for supporting client s long term recovery. The opportunity to repeat the Catalyst program, regardless of whether relapse has occurred, has benefits for supporting clients in achieving and maintaining their treatment goals. Challenges and limitations The extent to which the family-based needs of clients, including parenting skills, and family therapeutic and supportive intervention, are met within the program model is limited, although some provision is made through referral to external services or services provided by ReGen. There was no evidence that needs of client s children are being proactively assessed and referrals being assertively made. Some clients report that they have not attended the Momentum aftercare program because it is run in the evenings and they are either reluctant to travel at that time or unable to attend due to responsibilities such as caring for children. 35

36 Recommendations The program model, in its current form, should continue to be funded as a treatment option in the Victorian AOD sector. The eligibility and suitability criteria for the program model are quite narrow and therefore the model cannot replace other treatment types. The Momentum aftercare program should continue to be provided as part of the Catalyst program model. Strategies for improving the number of Catalyst clients that engage in the Momentum aftercare program could be explored. Attendance and retention rates for Momentum could also be explored to further understand how clients engage with the program. The option of providing a day time after care support group could be considered to accommodate those clients that feel they would benefit from Momentum but do not attend due to the need to catch public transport or travel long distances in the evening. Further development of the program model may be required to improve the capacity for providing family-based intervention and support, as outlined within the VAAP and the department s policy framework for AOD treatment services, and as articulated in current policy. This may include providing family-based counselling and support as part of the core program, and actively identifying and addressing the specific support needs of clients who are parents, and the children of clients by expanding assessment and discharge processes to include family-based needs. Investigate the further development of the Catalyst model to other drug types and/or target groups. Evaluation Question III. Was the treatment tailored to the clients specific needs, requests and prognosis? As a group based program there are limitations in the degree to which treatment can be individually tailored to clients specific needs. Individual client needs are able to be addressed to some extent through the one-on-one MET sessions, where specific needs can be identified and a plan for responding can be developed, however the time available for staff to work individually with clients is limited and therefore clients are encouraged to maintain or establish additional supports outside of the Catalyst program for the purpose of responding to any needs that are not addressed within the content of the program. Incorporating a variety of optional units does allow clients some choice and autonomy in planning their treatment episode, by selecting which units are relevant to their treatment needs. The fairly wide range of topics covered in the optional units also has the capacity to engage clients with different interests and learning styles in the therapeutic process. For instance, one client commented, There s been a nice balance. A good balance of things each day. They complement each other. They re fun and they encourage teamwork. In follow-up interviews clients also provided anecdotal evidence of positive long term outcomes stemming from their participation in optional Catalyst units, such as improved physical health due to 36

37 better nutrition and exercise, less stress and anxiety as a result of practising relaxation techniques, and better relationships with others due to new skills and awareness around communication (see chapter 9.3). Client retention rates for the program are also high (i.e. 70%) when compared to those reported in the literature for similar program models (see chapter 6), and this may be partly attributable to the inclusion of the optional units, due to their capacity to increase interest and engagement with the program content. Client feedback obtained through the evaluation does suggest, however, that some clients view the non-compulsory nature of the optional units as a limitation of the program, as this can result in some sessions being poorly attended and, it was felt by some, contributes to a lack of unity or cohesion among the group members. Some clients have also commented that they perceived a lack of clarity with regard to how many units or sessions need to be attended in order to graduate from the Catalyst program. There has been an ongoing interest from clients over the course of the program for more units to be made compulsory. In response to this feedback ReGen made the decision in the second year of the program to make the Mood Management and Anger Management units, which were initially optional, compulsory for clients. Strengths The inclusion of the optional units allows the program to be individually tailored and provides a holistic treatment approach by addressing a range of important treatment needs for the target population. The range of units offered may also enhance retention rates by increasing client engagement and satisfaction with the program. Challenges and limitations Some clients have been critical of the impact of non-compulsory units in the program, suggesting that this contributes to poor attendance and a lack of cohesion within the group, as well as some confusion as to how many sessions or units need to be attended in order to successfully complete the program. Recommendations Consideration could be given to allocating the program components as compulsory, elective, and optional (currently units are either compulsory or optional), and setting a requirement that clients attend all compulsory components, a minimum number of elective components, and any number of optional components. 37

38 Evaluation Question IV. How appropriate and capable is the service to meet the needs of clients of all ages, genders, cultural backgrounds and with varying degrees of physical and/or other barriers? The Catalyst program model is both appropriate and capable of meeting the needs of suitable clients within the target population, however, there are some limitations inherent in the capacity for group-based, mixed-gender programs to meet the treatment needs of all clients. In particular, the program model may not be appropriate or capable of meeting specific age, gender, or cultural related needs of some clients. These limitations are discussed. Age needs The majority of clients (52%) have been in the 35 to 50 year age range and client feedback and treatment outcomes indicate that the program is appropriate and capable in meeting the needs of this age cohort. There were, however, few clients in the 18 to 30 age range (6%; see Chapter 8.1.1). This may suggest the program model is not seen as an appropriate or preferable post withdrawal treatment option for younger clients who may have a lower level of dependence on alcohol and if dependent are more likely to be pre-contemplators in the stage of change and so not be engaged with treatment. The smaller number of younger clients accessing the program has also meant that these clients have tended to be in the minority within the group, and this has led to some young people feeling less engaged in the program. For instance, one client in their early 20 s commented, I was a bit sad that I didn t have anyone my age that I could relate to, that was really challenging. I didn t feel as though I fit in with the group and couldn t relate to the stories they were sharing. Clients who do not develop supportive peer relationships in the program are less likely to benefit therapeutically from their participation. Further, if younger clients feedback a negative experience to referrers and peers, this may impact on the likelihood of future referrals of young people to the program. Gender needs There has been a relatively equal number of males and females accessing program and this is a strength of the program, demonstrating the acceptability of the mixed-gender approach and the perceived suitability of the program model for both genders. However, females have tended to drop out of the program at a higher rate than males. Based on data available to the evaluation, female clients had an overall completion rate of around 49% while male clients had a completion rate of around 68%. There are likely to be a range of factors contributing to the lower completion rate for females. For instance staff have suggested there may be some gender specific reasons for women dropping out of the program, such as a tendency for female clients to feel intimidated by more dominant males within the group, a heightened responsiveness to social tensions between participants, as well as differences in the treatment and support needs of women that may be overlooked when they represent the minority within a given group. ReGen report that, in response to feedback regarding the poorer completion rates of women, female only participant groups will be trialed following the conclusion of the program s pilot phase. Cultural needs According to data available to the evaluation, the majority of clients who have participated in the Catalyst program have been of Anglo-Australian (85%), or European background (11%), with only one client of Aboriginal or Torres Strait Islander descent, one client of African descent and two clients of Asian descent (see Chapter 8.1.3). Given that the North and West 38

39 Metropolitan region, where the Catalyst program is located and most clients reside, is one of the most culturally diverse regions in Victoria, (according to data provided by the department, one third of the population were born overseas and 24% of Victoria s Indigenous population live in the region), this reflects an under-representation of clients from non Anglo-ethnic backgrounds. While it is beyond the scope of the current evaluation to determine the reasons for the lack of cultural diversity within the program, such reasons are likely to include the need for English proficiency and the cultural acceptability of group-based and mixed-gender treatment models. For example, clients with traditional Indigenous values are likely to prefer gender specific programs and Vietnamese clients are likely to prefer address these issues privately within the family system. Disability needs There is no evidence to suggest that the Catalyst model would not be appropriate for meeting the needs of clients with physical disabilities related to mobility, however, if the need exists, the capacity for people with significant sensory impairments to participate in and benefit from the program may need further investigation, given the amount of visual content and group discussion. The program model is not appropriate to meet the needs of clients with significant intellectual disability or cognitive impairment. Challenges and limitations The group-based, mixed-gender model may not be appropriate or capable of meeting the specific age, gender and cultural-related needs of some clients. Recommendations While the program model is suitable for both males and females, as well as younger and older adults, there is a need for age and gender heterogeneity to be balanced within the group to ensure that clients have sufficient opportunities to discuss issues specifically relating to their peer group. It would be recommended that the program have the opportunity for single gender sessions. These should occur after the main group has formed to prevent splintering along gender lines. It may also be worthwhile considering optional sessions for other groups, such as young people, or parents. 39

40 3.3 Accessibility of the service Evaluation Question V. How readily are people able to access care without barriers of distance, discrimination, affordability and restriction of service? Travel Distance Although the Catalyst program is provided as a state-wide service, its location at one site in the North-West metropolitan region of Victoria has meant that the majority of clients who have accessed the service have been those residing within this region (76% of clients participating in the evaluation were from the North and West metropolitan region, see chapter 8.3). As can be seen in Figure 1 below, clients tended to reside within a 15 km radius of ReGen. While the service is easily accessible to those that live within a reasonable daily commute distance, either via their own transport or the various public transport options available within a short walking distance, for those who live in other regions of Victoria access is limited. In interviews with the evaluators, staff from AOD agencies in the Southern Metropolitan region, indicated that distance was a barrier for many of their clients who might otherwise benefit from the program. Data available to the evaluation shows that only 7% of clients were from the Southern metropolitan region (see chapter 8.3). Access for clients living in regional and rural areas is limited by distance, higher travel costs and a lack of public transport options, and evaluation data reveals that only 2% of clients were from regional areas of Victoria (see chapter 8.3) km 6-15 km km km km Figure 1. How far did you travel to get to ReGen? A temporary accommodation option was trialed in the first year of the pilot to improve access for clients for whom distance was a barrier, however this was found to be extremely problematic and was therefore discontinued. ReGen found that an increased duty of care was placed on program staff with regard to the clients utilizing the accommodation (e.g. monitoring sobriety and attendance) and that this impacted negatively on the treatment experience for those clients. It was also felt that clients who resided in temporary accommodation while attending Catalyst did not experience the full benefits of a communitybased model - learning and practicing new skills in one s own environment. ReGen have suggested that for temporary accommodation options to be appropriate for a communitybased treatment model, the accommodation arrangements would need to be managed externally to the program, and clients would need to return home on weekends and other consecutive non-program days. 40

41 Financial Considerations For most clients there are no financial barriers to accessing the Catalyst program. There is no cost for clients to participate, and this is a significant strength of the model for many clients. For instance, when asked what the best thing about Catalyst was, one client commented, The fact that it s free. It sounds flippant, but it s a big deal for us. Other rehab programs leave a big hole in your budget and that can make it very hard to manage. Public transport costs are either fully or partially covered by ReGen through the provision of zone one and two transport tickets. Many clients are reliant on public transport (due to loss of licence or financial constraints associated with running a car), and without the provision of the transport tickets may find the costs of public transport for the duration of the program prohibitive. It was also found that the provision of public transport tickets had benefits for clients outside their attendance of the program. For example, clients used the tickets on weekends to go out and engage with family and friends, which for many who had been previously isolated was a significant and positive lifestyle change. Clients are also provided with lunches, as well as snacks and refreshments. Over the course of the evaluation many clients have commented that having their transport and food costs covered has made it easier to attend the program. For instance, one client stated, The travel cards were the difference between me attending and not attending, while another commented, the lunches were great for me because I am broke. Parenting Responsibilities In addition, the management team at ReGen have suggested that there may be some potential clients in primary carer roles that have not accessed the program at all, due to the fact that childcare support options were not available through the program. Although the need for childcare support was not indicated by Catalyst participants, the possibility of providing childcare support was considered by ReGen during the course of the pilot. While ReGen were unable to provide this service during the pilot, as there was no funding allocated to childcare and the resources were not available in-house, it was felt that future consideration of the option for childcare would be warranted. Given the potential benefits of a day program, compared to a residential program, for people with childcare responsibilities, the impacts of providing childcare support options to participants of the program deserves further attention. Disability There are no barriers to program access for people with physical disabilities related to mobility. Strengths Catalyst is easily accessible by public transport and the provision of transport tickets has been an important support for many clients to attend the program. There is no cost to attend the program which makes it an appropriate treatment option for clients with limited finances. 41

42 Challenges and limitations Despite the fact that Catalyst is intended as a state-wide service, the program is generally not accessible to clients who do not live within a reasonable daily commute of the Coburg site The lack of childcare support provided through the program may mean there are clients who have not accessed the program due to child care responsibilities. Recommendations Community-based models require that clients live within a reasonable daily commute of the program and therefore are not appropriate to be considered state-wide services. The Catalyst program draws clients from the North-West metropolitan region, and therefore at least one additional service should be established within the South East region. Consideration might also be given to trialling a program in other areas with sufficient population density such as the Barwon South West region, for example, in Geelong. While travel distance would make it unlikely that the same model of community-based rehabilitation would be appropriate for rural regions of Victoria, it may be useful to consider a semi-residential model with accommodation provided on program days and clients returning home on non-program days. This would, however, reduce some of the benefits of the real world learning approach of the program. To improve program access for parents with childcare responsibilities there is a need to investigate options for providing child care support. Chapter 4 Continuous and coordinated care Evaluation Question VI. Was treatment provided in an uninterrupted and continuous manner across the continuum of care? Clients are encouraged to engage with a range of supports throughout their engagement with Catalyst. This includes maintaining existing supports while participating in the program, as well as direct links with supports offered through the program, or through referrals to other services within ReGen or externally provided. This chapter discusses those supports, as well as other efforts intended to provide continuous and coordinated care to Catalyst clients. Prior to admission Clients are provided with a range of supports to help prevent relapse before commencing Catalyst and to address any other needs that might otherwise impact on their ability to participate in the program. This includes telephone support from Catalyst clinicians, linkages to support services, and liaison between Catalyst clinicians and other professionals involved in the client s care. Clients are also strongly encouraged to attend an information session prior to starting the program, and a pre planning session, which is designed as an opportunity to meet 42

43 % of Respondents Number of clients Caraniche for the Department of Health staff and other participants and to start preparing for the program, for example by thinking about treatment goals. The Catalyst team leader reports that client attendance at these sessions is important because it helps to prepare clients and ease their anxieties about participating in the program, and also enables staff to assess clients motivation to actively engage with the program. As can be seen in Figure 2 below, the majority of clients felt that they support they were provided prior to admission met their needs Definitely Mostly Somewhat Not really Not at all Figure 2 Did the support prior to admission meet your needs? To further support clients during the time between completing withdrawal and commencing Catalyst, wait times are kept to a minimum. Because the Catalyst program does not have a rolling intake, the time clients must wait to commence the program generally depends on the timing between the end of the withdrawal episode and the start of the next program cycle. This means that if a client completes their withdrawal in week two of the current program, for example, they must wait four weeks to begin the program. As can be seen in Figure 3 below, most clients have been able to start Catalyst within four weeks of completing their withdrawal. The data also shows that the wait time for Catalyst clients has tended to decrease over the course of the pilot. For instance, the number of clients who have waited more than four weeks to commence the program has reduced from 32% in the first year, to 28% in the second year and only 8% in the final year. It is possible that this may be attributable to ReGen s refining of the referral and intake process over time, as well as to an increased awareness of the program within the sector leading to more referrers and clients planning ahead to time withdrawal episodes with Catalyst program cycles to 2010 (n = 27) 2010 to 2011 (n = 54) Program Year 2011 to 2012 (n = 31) Figure 3. How long did it take to get into the program following withdrawal? Less than 1 week 1 to 2 weeks 2 to 3 weeks 3 to 4 weeks more than 4 weeks Clients were asked if they were satisfied with the time they had to wait to begin Catalyst and most clients indicated that they were definitely happy with the wait time. As one client commented, I was discharged on Friday and the program commenced on Monday very impressed. As can be seen in Figure 4 below, clients who attended the program in the second and third years, when the wait time tended to be shorter, were more likely to say they were happy with the wait time between withdrawal and Catalyst. 43

44 Number of clients % of Respondents Caraniche for the Department of Health to 2010 (n = 27) 2010 to 2011 (n = 54) Program Year Figure 4. Was this time acceptable to you? 2011 to 2012 (n = 31) Definitely Mostly Somewhat Not really Not at all This is further supported by the data in Figure 5, which shows that clients who commenced the program within a couple of weeks of completing withdrawal tended to report higher levels of satisfaction with the wait time Definitely Mostly Somewhat Not really Not at all 0 One week or less One to two weeks Two to three weeks Three to four weeks More than one month Figure 5. Were you satisfied with the wait time to enter Catalyst? While a short wait time is generally considered preferable, clients have expressed differing views about the appropriate time between completing withdrawal and starting Catalyst. Some clients prefer to begin Catalyst immediately, while others feel they need some time to prepare themselves to engage in the program, for instance to stabilise their mental state or make arrangements around child care and other responsibilities. It is important, therefore, to provide potential clients with information about the Catalyst program as early as possible in their recovery process to allow them to plan the timing of their withdrawal and rehabilitation appropriately. In instances where clients have not yet completed a detox or withdrawal, this is arranged through Catalyst and may include either arranging a bed in ReGen s residential withdrawal unit, Lesley Ann Curran Place (LACP), or providing support through ReGen s home-based withdrawal team. For clients who have completed their withdrawal, Catalyst is also able to arrange support through ReGen s post withdrawal support services. ReGen s capacity to provide these services and, to some degree, to coordinate the timing between withdrawal and Catalyst, has been a particular strength in providing continuity of care to Catalyst clients. ReGen have also found that closing intake at 20 clients is necessary in order to maintain manageable waitlists. Fifteen clients are accepted into the program and five are placed on standby and will be accepted into the following program if a place in the current program does not become available. According to the Catalyst team leader, the number of places offered per 44

45 Number of participants Caraniche for the Department of Health episode (i.e. 15) meets, on average, the number of referrals made, and therefore is an appropriate number. According to data from satisfaction surveys around 50% of clients are admitted to Catalyst within one week of being discharged from withdrawal services and a further 15% within one to two weeks. During the program To ensure continuity of care during the program clients are encouraged to maintain engagement with any professional supports they may have been accessing prior, such as GPs, psychologists, AOD workers or case managers. Where appropriate, Catalyst clinicians liaise with those support persons regarding the client s treatment and progress through the program. As can be seen in Figures 6 and 7 below, almost 70% of Catalyst clients were accessing other support services during their time in the program, and all clients who required collaborative care indicated that they were happy with the level of communication between Catalyst and their other support services. Throughout the program clients are provided with an opportunity, through the individual counselling sessions, to identify and address any additional, unmet support needs they may have. No 32% Yes 68% Figure 6. Were you accessing other services during you time in Catalyst? Yes They didn't need to No Figure 7. Did you feel that Catalyst worked collaboratively with those services? The original program specifications put forward by the department in the call for submissions (2008) required the service to provide some out of hours support to prevent relapse during the treatment program, however this has not been included as part of the program model. Several clients have also indicated that they would like after-hours telephone support to be provided through Catalyst. ReGen management report that the provision of after-hours telephone support was considered, however it was felt that providing 24-hour telephone support would be excessively burdensome on staff and outside the scope of their professional responsibility, and would be counter to the program s treatment approach, which is to help clients learn to self-manage their substance use. Clients are provided with the contact details for Directline, a 24-hour telephone service offering counselling for people with substance use issues, and are encouraged to utilise this service for immediate support outside Catalyst. 45

46 Support offered during Catalyst to help prevent relapse involves assertive, same day follow-up of clients who do not attend. In the first year of the program Catalyst staff report that clients were called if they did not attend for two or three days, however it was found that allowing several days to pass before calling clients meant that often clients had relapsed and were difficult to re-engage in the program. According to staff, the decision to contact clients on the same day they did not attend increased the likelihood that support could be offered before a full relapse occurred, and improved the chances of re-engaging clients in the program. Post program At the completion of the Catalyst program ongoing care is provided for clients through the establishment of post program goals and linkages to treatment and support services, as well as through the feedback provided to the client s existing support services and professionals. Examples of the types of referrals made include general and AOD counselling, employment assistance, health specialists and welfare and housing support. According to interviews with Catalyst clinicians, the outcomes of referrals (i.e. whether a client has been linked successfully with services and supports) are followed up by ReGen for some, but not all clients. Staff report that referrals made early in the program, or as part of an individual treatment plan, are more likely to be followed up than referrals made at discharge. Unless clients engage with the Momentum aftercare program, or re-engage with Catalyst, there is limited communication between Catalyst clinicians and clients following discharge. To improve the likelihood that clients are successfully engaged with the services they require post program, including ongoing treatment and supports, a more assertive follow-up approach to referrals may be required. Given that the evaluation findings presented later in the report (see Chapter 9.3.2) indicate that clients tend to relapse in the first two months following program completion, this may be an appropriate timeframe in which to provide a general clinical follow-up, for the purpose of reconnecting clients with treatment if required. In addition to referrals made at discharge, clients are provided with information about a range of peer support groups available, including Alcoholics Anonymous, SMART Recovery, New Life, and Catalyst s aftercare program, Momentum. Regular attendees of Momentum are invited to speak to Catalyst clients about the aftercare program and to share their experiences, and clients are strongly encouraged to attend a session following their completion of Catalyst. For the other support groups, clients are provided with information, including an outline of the programs, meeting times, and contact details. Catalyst clients who have attended these groups previously are also encouraged to talk to the other clients about their experiences of the support groups. While clients are provided with information and encouragement to engage with peer support groups following completion of Catalyst, some clients may benefit from being more actively linked with these groups prior to discharge, for instance by providing visits to various support groups as part of the Catalyst program. 46

47 Strengths There are processes in place to provide clients with continuous and coordinated care throughout their engagement with the program, including: providing telephone support through Catalyst prior to admission, minimising wait times between completing withdrawal and commencing Catalyst, ensuring clients are linked in with any additional supports they require and collaborating with those supports around the client s treatment, providing information about the supports available post Catalyst, and making referrals, as needed, for post program support. Comprehensive discharge planning is provided for clients. Challenges and Limitations The program model does not provide assertive linkages to post program supports as part of the discharge process and does not include general post program follow up of clients. Recommendations It may be beneficial to consider both general follow up of all clients within two months of discharge to check on progress and provide linkage and support if required. A more assertive follow-up process around discharge referrals could be implemented to increase the likelihood that clients engage with referrals provided. Optional facilitated visits to peer support groups, or inviting members of peer support groups to come and talk to participants, could be included as a part of the program. Chapter 5 Resources Evaluation Question VII. How sustainable is the structure of Catalyst in its capacity to provide infrastructure in terms of workforce, facilities and equipment? Is it innovative and able to respond to emerging needs? 5.1 Infrastructure and program resources ReGen have dedicated significant resources to deliver and support the Catalyst program, including a large program area with the Jessie St Coburg site, which was renovated specifically to accommodate Catalyst. The Catalyst area allows clients separate access from the main entrance of the building, including wheelchair access, and includes an outdoor area for socialising, a kitchen and dining area for preparing and eating lunch and a good sized space for delivering the program. The back courtyard includes a large vegetable garden which has been developed over time by the clients and put to use for the nutrition sessions. Quality furniture and kitchen supplies have been purchased for the program area, and audio visual equipment, drums, and art supplies are also provided to support the various program units. 47

48 The quality of the program space, and the ways in which clients are invited to use the space, have had positive impacts on the treatment experience provided through Catalyst. For instance, staff report that clients often comment on how relatively small things, such as the quality of the furniture and the provision of new cups and plates in the kitchen, have made them feel valued, comparing this to their experience of run down services that make you feel like you re not valued. The space itself has been intentionally designed to have an inviting, homely feel and this contributes to clients feeling welcome and comfortable, which eases anxieties and encourages clients to attend and engage. The provision of a kitchen and dining area within the space, as well as the vegetable garden, not only supports the nutrition sessions, but also facilitates social interaction and group cohesion, which is important to the therapeutic process. Clients also feel a sense of ownership for the program because they are encouraged to contribute to the program space, including client art work and words of encouragement placed around the room, the opportunity to work in the vegetable garden, and the activity of selecting which groceries are purchased for the kitchen. ReGen have found that these small activities have had a significant impact on the degree to which clients feel valued and respected, which in turn increases the likelihood of positive treatment outcomes through program engagement and greater feelings of self-worth. The Catalyst program is also well integrated with a range of additional support services that are available through ReGen. The co-location of the program within an organisation that is able to provide clients with access to various support services, such as home based and residential withdrawal, individual and family counselling, pharmacotherapy and parenting skills programs, is a substantial strength for the program. As a significant AOD organisation within Victoria, ReGen is also able to support the sustainability of the Catalyst program, including the capacity to source referrals through its range of services, and access to a readily available workforce to provide both program delivery and background support Program manuals The Catalyst resource materials were developed by ReGen during the first two years of the three year pilot. They draw on established best-practice in AOD rehabilitation programs, feedback from Catalyst participants, adult learning and clinical experts and the experience of program staff in using the materials over the course of their development. The Catalyst resources are comprised of the Facilitator s Kit (the Evidence Guide; Cognitive Behavioural Coping Skills Facilitator s Guide; Electives Facilitator s Guide; Motivational Enhancement Therapy Practice Guide) and the Participant Workbook. 5.2 Workforce Service delivery A team of four fulltime clinicians are employed to deliver the Catalyst program, including a team leader. Support staff are also available from within the Education and Training (E&T) department at ReGen to cover staff absences, and are fully trained in delivering the program content. While ReGen have found the team of four fulltime staff to be sufficient, the availability of support staff that are able to deliver sessions at short notice is essential, particularly in the event that more than one of the fulltime staff are unavailable. Therefore, any organisation setting out to deliver the same program model would need to be able to provide a fully-trained support staff for the program. Employing casual workers from external services is not a viable option for covering staff absences within the Catalyst program model, due to the specialised nature of the program. For the same reason it is important that core, fulltime staff working within the program are trained to deliver all of the program content. 48

49 5.2.2 Contract staff Additional members of the Catalyst team are the external providers recruited to deliver the specialist adjunctive units. These include a nutritionist, a family therapist, a nurse, a yoga instructor, a financial counsellor and representatives from two employment agencies Qualifications and experience Evaluation Question VIII. Do practitioners have requisite qualifications and registrations to provide adequate service to clients? All staff employed within the Catalyst program have the appropriate expertise for their role. Three of the four fulltime Catalyst clinicians have formal tertiary qualifications including degrees in mental health nursing, social work, drug and alcohol treatment, and art therapy, and the fourth fulltime clinician has a diploma in counselling. All four clinicians have obtained the minimum required competencies in drug and alcohol and mental health to be deemed dual diagnosis capable. The number of years the clinicians have worked in relevant areas of the health sector ranges from six to 29 years. From their previous experience the clinicians bring to Catalyst a range of skills and knowledge that are utilised in delivering the program. In particular, group facilitation skills and experience working within a CBT framework have been important qualifications for staff working within Catalyst. The E&T staff who also work within Catalyst have clinical qualifications and experience in the drug and alcohol sector, as well as formal qualifications in workplace training and assessment and extensive experience in developing and delivering education and training programs. The external providers all have relevant qualifications in their areas of expertise. ReGen were committed to employing staff within the Catalyst roles that possessed particular personal attributes including a belief in the effectiveness of treatment, warmth, approachability, and a non-judgemental attitude. According to ReGen s Director of Clinical Services, this commitment was driven by the research literature which demonstrated that the personal attributes of clinical staff, with regard to their professional approach, are important determinants of treatment effectiveness. This may be supported by the positive results of the Catalyst program and feedback from clients with regard to program staff, such as The staff were tremendous, they were always there to support you and to help you understand things, and The program is superbly run, the people running it are highly competent Professional development and supervision Evaluation Question IV. Is adequate ongoing supervision and professional development provided? In line with the policies of the organisation, ReGen has provided Catalyst staff with a range of relevant training and professional development opportunities. As a priority, all staff were required to complete the AOD and dual diagnosis competencies. Over the course of the pilot staff have also completed training in such topics as alcohol and violence, first aid, occupational health and safety, and working with different client groups including clients with neurocognitive difficulties, bipolar disorder, and trauma affected clients. ReGen have also supported staff in pursuing relevant tertiary qualifications by providing study leave. All staff report that they are happy with the amount of professional development they have received. Staff also comment that they receive ongoing professional development through the diverse nature of their roles in the program. The Catalyst program provides an excellent model for AOD workers to develop a diverse range of skills, as staff are required to develop 49

50 competencies across a range of subject areas and treatment approaches, to provide both individual and group-based treatment, and also have the opportunity to work along-side other service professionals. The supervision policy of the Catalyst program differs from ReGen s standard policy of clinical supervision. Over the first year of the program s operation Catalyst staff felt that there was a need for ongoing, reactive supervision, due to the fact that the program was in a development stage and the nature of the work was quite intensive. ReGen responded to this need by ensuring that time and staffing resources were available to provide the clinicians with clinical supervision as soon as possible when the need arose. In addition to the provision of reactive supervision the clinicians also have one planned supervision session per 6-week Catalyst cycle, (7 per annum) in which staff are encouraged to reflect on practice-related issues in a more structured way than the reactive supervision. There is no provision for structured group or peer supervision for Catalyst staff. In response to a recommendation made in the preliminary evaluation report (2009 to 2010), Catalyst management report that a peer supervision model was trialed but later discontinued as staff felt that it did not contribute any additional benefit to the supervision already occurring. The model of supervision was appropriate for providing the level and type of support required by the clinical team during the initial development stage of the program model. As the program model has now matured and is more established, there is likely to be a need for a more formal model of structured, in-depth and reflective practice-driven supervision, including both individual and group supervision Training new staff The qualifications and expertise of the Catalyst clinicians have developed over the course of the pilot through the experience of developing and delivering the program. According to ReGen, given that an established program and service delivery team now exists, training a new staff member to work proficiently within the program (i.e. to achieve competency in delivering all program units) would require approximately six months. New staff members would also need to meet the same criteria as existing staff, in terms of qualification level, experience, and personal attributes, (see chapter 7.2.3). With regard to training an entirely new team of staff to deliver a Catalyst-type program it is likely that qualified staff could be adequately trained to commence program delivery in less than one month, with a plan to develop competencies across the program over a six-month timeframe. Provided that staff have appropriate AOD qualifications and experience in CBT and group facilitation, ReGen report that the Catalyst program manual and facilitators guides are likely to serve as excellent resources for training new staff to deliver the program. 5.3 Program management and review processes The Catalyst program sits within the Clinical Services division of ReGen and under the direct line management of the Director of Clinical Services. The program is also co-managed by the Director of Education and Training. The co-management approach has worked well for the program, for instance by increasing the management resources available, adding diversity in terms of management input and allowing greater opportunities for management to promote the program through networking activities. There are several processes in place to facilitate the ongoing management and review of the program. Steering Committee meetings are held every three months to oversee the program 50

51 and are attended by key representatives from the department, ReGen, Caraniche and the Association of Participating Service Users (APSU). The Catalyst managers also attend program planning meetings with the Catalyst team every six weeks, between program episodes. Program planning meetings have a strategic focus and are used to discuss and review the program s development, including its content, delivery, management and promotion. Continual program review and development is a key feature of ReGen s management of Catalyst. This includes ongoing collection and evaluation of both quantitative and qualitative program data, including client demographics, attendance rates, treatment progress and outcome data and written and verbal feedback from clients. ReGen has demonstrated an ongoing commitment to reviewing the program s performance and addressing any key issues that are identified and this has been a significant strength in the program s development and the quality of its service delivery. Strengths ReGen have committed significant resources to the Catalyst program, with an emphasis on ensuring sustainability and the capacity to respond to emerging needs. These resources include a permanent and well-equipped program space including indoor and outdoor areas, a team of qualified and experienced full-time, casual, and support staff covering program development and delivery, and an efficient program management and review structure. ReGen is able to provide additional in-house support services for the program, ensuring that clients are well integrated with the range of services they may require. The cross-collaboration approach between the Education and Training department and the Catalyst clinicians has been a strength in the development of the program content, as a broad range of relevant skills and expertise were able to be utilised. Challenges and limitations The structured, psycho-educational nature of the program means there is a need for support staff who are trained to deliver the program content to be readily available to cover staff absences (in the event that several core staff are absent at one time), as employing casual workers from external services is not a viable option. Planned, structured clinical supervision for Catalyst staff is limited to seven hours per annum and this may be insufficient now that the program is well-established. The focus upon having ad hoc supervision available on demand has meant that there has not been the opportunity for structured formal group supervision. 51

52 Recommendations Any organisations selected to deliver a similar program model in the future will need to have the resources required to support the program, including a permanent, well equipped and welcoming program space, access to support services such as withdrawal and counselling, and the capacity to employ and support appropriately qualified fulltime, casual and support staff for program delivery and ongoing review and development. Research and development resources should continue to be made available to the program in order to enable ongoing internal evaluation. Any future rollout of the Catalyst model is encouraged to adopt ad hoc supervision during the development phase. Once the program has become established, more formal structured individual supervision should be implemented (a minimum of one to two hours individually per program and a minimum of one 90-minute group session per program). Group supervision is an essential component of any group-based program, and in particular it should not focus upon case review (this should occur at other times), rather it should emphasis reflective practice, including awareness of group processes and dynamics, and emphasis peer learning. Chapter 6 Episode of care targets Evaluation Question X. Were the contractual episode of care targets met? The department set episode of care targets for Catalyst as 110 episodes of care per annum. The program was to run with at least seven program cycles per year, each of four to six weeks duration, and with 15 clients admitted in to each program. Catalyst runs as a six-week program and accepts 15 clients per episode. With the potential for eight program cycles per annum this would result in 120 episodes of care if all program cycles have 15 clients completing. In the first year of the program (June 2009 to June 2010) seven program cycles were run and 83 clients were admitted to Catalyst. There were some difficulties in meeting targets initially and this was due to the program being new and the lack of time and resources available for promotion. In the second year of the program (July 2010 to June 2011) there were eight program cycles and 108 clients were admitted to the program, and in the third year (July 2011 to May 2012) there were seven program cycles, including a four-week refresher program, and 81 clients were admitted, resulting in a total of 272 clients admitted to Catalyst over the three year pilot phase. If episode of care targets are deemed to be 330 clients over three years, the actual number of clients that commenced the program reflects 82% of the original target. Of the 272 clients that commenced Catalyst, 190 completed the program (i.e. were present in week six), equating to a completion rate of 70%. This is a particularly high completion rate compared to completion rates described in the research literature for similar programs, and therefore represents a positive outcome for the program. For example, an often cited study investigating adult completion rates for drug and alcohol treatment programs in the US found an average completion rate of 23% for intensive outpatient alcohol treatment programs (n = 52

53 832; Wickizer et al, ). More recently, the Substance Abuse and Mental Health Services Administration (SAMHSA, ) published data demonstrating a completion rate of 46% for outpatient alcohol treatment programs. Strengths The Catalyst program has demonstrated good attendance and completion rates, particularly compared to similar programs reported in the literature. Recommendations It may be useful to consider conducting follow up interviews with clients that do not complete the program in order to investigate reasons for non-completion and subsequent outcomes. These interviews would need to be conducted within one to two weeks of discharging from the program. Chapter 7 Catalyst within the broader AOD sector Evaluation Question XI. Was the treatment provided as a useful component within the broader AOD service sector? 7.1 Stakeholder feedback The Catalyst program has been recognised as a useful treatment option within the broader AOD service sector. This is evidenced by positive feedback from clients and other stakeholders, and recognition in the form of a national award for Excellence in Treatment and Support (National Drug and Alcohol Awards, 2011), and as a finalist at the 2011 Victorian Public Healthcare Awards. ReGen report that feedback from other services engaging with Catalyst has been positive, with most viewing the program as a useful treatment alternative for clients. According to management and clinical team for Catalyst, some particular strengths of the program that have been pointed out by other professionals have included the range of skills that clients learn (i.e. CBT) which can be utilised in counselling and treatment programs other than Catalyst, and the ability for Catalyst to provide case management and therefore to address presenting crises that may otherwise inhibit the treatment progress of the client. 4 Wickizer, T. et al. (1994). Completion rates of clients discharged from drug and alcohol treatment programs in Washington State. American Journal of Public Health, 84, Substance Abuse and Mental Health Services Administration, Office of Applied Studies (April 23, 2009). The TEDS Report: Treatment Outcomes among Clients Discharged from Outpatient Substance Abuse Treatment. Rockville, MD. 53

54 Over the course of the evaluation Caraniche contacted several of the AOD service providers that have referred clients to Catalyst to discuss their views on the program. All of the services contacted felt there was a need for a non-residential program option and that Catalyst was effective in meeting this need. In particular, other service providers commented that the Catalyst program provided structure and meaningful activity for clients post withdrawal, which some felt was lacking within the sector. The opportunity to participate in structured treatment while continuing to live at home was also pointed out as particularly important for clients in terms of re-establishing their social connectedness. The only negative feedback received from service providers was that the program was not available in more locations, and that the wait time to begin the program, which could be up to six weeks, could be too long for some clients. 7.2 Program referrals According to data available for the evaluation, the majority of referrals, almost two thirds, have been made by services provided through ReGen, while referrals from other services account for around one-fifth of all referrals. This may suggest that although the Catalyst program has received positive feedback from the broader AOD sector, the sector is not yet engaging with the program to the extent that might be expected for a state-wide service that is intended to be a primary treatment option, and is the only one of its kind. In response to this finding, Catalyst staff and management have indicated that a number of clients wanting to access the Catalyst program are first referred to other ReGen services (e.g. ReGen s withdrawal unit, Lesley Anne Curran Place, or ReGen s home-based withdrawal services) in order to be eligible for the program, and this may skew the data for internal referrals. The evaluation is unable to provide data regarding the initial referral sources for these clients. An analysis of demographic data representing the geographic spread of Catalyst clients may provide some indication of the extent to which AOD services across the state are engaging with Catalyst, as it is likely that client s first service contact would be with treatment agencies providing service to the region in which they reside. An analysis of client s postcodes shows that approximately 50% of clients reside in the municipalities of Darebin, Moreland, Banyule and Yarra, which are the primary catchment areas for ReGen, and a further 11% reside in the municipalities of Melbourne and Moonee Valley, which share a border with Moreland (see Chapter 8.3). Self-referrals account for around one-sixth of all referrals made to the program, however it is not clear whether this reflects referrals made specifically to Catalyst, or to ReGen s triage service, as a number of referrals were listed (in the client documentation provided to the evaluation) as self / triage. Other Services, 10, 6% Other AOD Services, 24, 16% Self, 24, 16% ReGen Services, 94, 62% Figure 8. Referral sources 2010 to

55 Number of referrals Caraniche for the Department of Health A number of referral trends can be seen in the data presented in Figure 9 below. A comparison of the 2010 to 2011 (year two) and the 2011 to 2012 (year three) data (the 2009 to 2010 data is underrepresented due to the evaluation beginning after the program had commenced, and therefore is presented but not discussed) shows that, among the referrals made through ReGen services, there was a decrease in the number of referrals made through LACP (21% of all referrals in year two and 12% of all referrals in year three) and home-based withdrawal services (10% of all referrals in year two, to 3% of all referrals in year three), and an increase in referrals made through ReGen s counselling services (four C s) (7% of all referrals in year two, to 17% in year three) and to a smaller extent, the triage/assessment service (19% of all referrals in year two, and 25% in year three). There was no real change in the number of self-referrals made between year two (17% or all referrals) and year three (15% of all referrals). Similarly, there was no notable change in the number of referrals made by other services between year two and year three. Over both years, referrals from AOD services in the North and West metropolitan region accounted for around 9% of all referrals, and around 38% of all referrals made by services other than those provided by ReGen. Referrals from AOD services in the Southern and Eastern metropolitan regions each accounted for 4% of all referrals and 16% of referrals made by services other than ReGen. Overall, referrals from community health services, GPs and other support workers accounted for only 8% of all referrals made between years two and three. In interviews with the evaluators Catalyst staff reported that some efforts were made to promote the program among community health services and GPs within the North-West metropolitan region, but this was largely ineffective in increasing the number of referrals received from these sources to 2010 (n = 20) 2010 to 2011 (n = 72) 2011 to 2012 (n = 60) Figure 9. Referral sources by program year Referral source 55

56 Strengths Catalyst has received positive feedback from within the broader AOD sector, and the non-residential, intensive rehabilitation model is generally seen as an important treatment option for AOD clients in Victoria. Challenges and limitations Some referrers felt that the wait time to enter the program could be too long for some clients. Only around one-fifth of referrals to Catalyst have come from services other than those provided by ReGen, suggesting the broader AOD sector has not engaged with the program to the extent that might be expected for a pilot program intended as a new primary treatment option for Victoria. Recommendations In the event that multiple programs are implemented within a particular region in the future, it would be useful to stagger program start dates as this may help to reduce wait times for clients coming out of withdrawal services. Future evaluation should be conducted to investigate reasons for the relatively small number of referrals received from services other than ReGen, in an effort to improve sector-wide engagement with the program. Further efforts to promote the program, and to establish and maintain referral relationships with the broader AOD sector should be undertaken to broaden the referral base. 56

57 Part 3: FINDINGS: Client Outcomes Part three provides the findings of the evaluation with regard to clients of the Catalyst program. This includes a description of the client profile, including their treatment needs and the extent to which these have been met, the achievement and long term maintenance of significant treatment goals as well as other benefits of participation reported by clients, and information regarding the program experience of clients, obtained through satisfaction surveys and other client feedback channels. Chapter 8. Client profile The following section describes the client demographic data from those clients who have given their consent to participate in the evaluation. In total there were 168 clients that consented to participate in the evaluation, representing nearly two thirds of all clients that participated in Catalyst during the pilot phase (June 2009 to June 2012; see Figure 10 below). Due to missing data, however, there are variations in the sample size for each of the questions presented below and therefore the sample N is provided for each set of data. Consented to participate in the evaluation, 168, 62% Did not participate in the evaluation, 104, 38% Figure 10. Clients that participated in the evaluation 8.1 Age, gender, and ethnicity Age According to evaluation data, the average age of Catalyst clients is years (SD = 9.5). The youngest client was 20 years of age and the oldest client was 64 years of age. Males, with an average age of years (SD = 10.02), tended to be older than females, who had an average age of years (SD = 8.43). As can be seen in Figure 11 below there were fewer clients at the younger end of the age spectrum (i.e. 18 to 35 years). It is possible that younger clients are more likely to have 57

58 Number of participants Caraniche for the Department of Health characteristics that make them unsuitable for the Catalyst model, such as more chaotic and unstable lifestyles, poor self-management, lower treatment motivation, or higher degrees of poly-substance use where treatment is required to target both alcohol and drug use. The smaller number of younger clients accessing the program has also meant that these clients have tended to be in the minority within the group, and this has led to some young people feeling less engaged in the program, and, through feedback to referral services and peers, this may impact on the likelihood of future referrals of young people to the program Age in years Figure 11. Age of participants (N = 168) Gender According to data collected by ReGen there have been 155 males and 117 females enrolled in the Catalyst program between June 2009 and June Females therefore represented 43% of all clients who commenced Catalyst over the three-year pilot phase (see Figure 12 below). As discussed earlier in this report, however, the program non-completion rate is higher for females than for males (51% of females do not complete the program, while 32% of males do not complete) and this issue warrants further investigation. While the potential for female-only groups to improve the completion rate of women could be investigated, overall ReGen have found that incorporating a balanced number of males and females into each group has a positive impact on group dynamics and the group therapeutic process. While mixed gender groups may impact on the types or depth of conversations that are had within the group therapy setting, there are significant benefits for healing and recovery in mixed gender therapeutic models. According to Catalyst staff, men and women tend to contribute to the group discussion in different ways and each gender benefits from sharing in the experiences of the other. Clients, particularly males, have also responded positively to the gender mix, with several clients suggesting that it is a particular strength of the program. For instance, one client commented, The last group I was in was only men and, you know, men are funny, they don t really like to open up. Having women in the group changes that. They re more generous with their experiences and they make it easier for others to open up. Furthermore, one of the key strengths of the Catalyst program model is the emphasis on real-world recovery and mixed gender therapies fit well with this approach. Female 43% Male 57% Figure 12. Gender (N = 168) 58

59 8.2.3 Ethnicity The majority of Catalyst clients have been of Australian, non-aboriginal or Torres Strait Islander descent. Reasons for the low number of clients from culturally and linguistically diverse backgrounds may include the need for English proficiency, and cultural acceptability of group based and mixed gender therapy models. Investigation of these factors is beyond the scope of this evaluation. Table 5. Ethnicity / cultural background of participants Ethnicity Total (N = 141) Males (n = 87) Females (n = 54) Australian (non-atsi) 120 (85.0%) 70 (80.5%) 47 (87.1%) Aboriginal/Torres Strait Islander (ATSI) 1 (0.7%) 1 (1.1%) 0 (0.0%) European 15 (11.0%) 11 (12.6%) 4 (7.4%) Asian 2 (1.4%) 2 (2.3%) 0 (0.0%) African 1 (0.7%) 1 (1.1%) 0 (0.0%) Other 3 (2.0%) 2 (2.3%) 3 (5.6%) 8.2 Marital status and household Marital status Most clients indicated that they were not currently married, with nearly 45% reporting that they had never been married. Given that the average age of clients ranged between 40 and 45 years, the number of clients who have never been married is notably high. More than one third of clients were either separated or divorced from their partners. It is likely that for a number of clients there is an association between their problematic alcohol use and problems in their intimate relationships, demonstrating the importance of offering relationship counselling or advice to clients and their significant others. Catalyst provides this to some degree through their Strengthening Relationships sessions, as well as through referrals to ReGen s family counselling service or external services such as Relationships Australia. Table 6. Marital status of participants Marital status Total (N = 157) Males (n = 97) Females (n = 60) Registered marriage 15 (9.5%) 12 (12.4%) 4 (6.7%) De Facto marriage 12 (7.6%) 5 (5.2%) 8 (13.3%) Widowed 4 (2.5%) 4 (4.1%) 0 (0.0%) Never Married 70 (44.6%) 42 (43.3%) 28 (46.7%) Separated / Divorced 56 (35.7%) 34 (35.1%) 20 (33.3%) 59

60 8.2.2 Household Around one third of all clients reported that they lived alone, with males more likely than females to report living alone. Clients who live alone may have fewer social supports than those who reside with family or friends, and may be further isolated as a consequence of their alcohol use and other factors such as unemployment. Given the importance of social supports in promoting general wellbeing and recovery, as well as the tendency for loneliness and relationship crises to lead to relapse, these results demonstrate the need to provide clients in AOD recovery with skills to improve their relationships and social connectedness. Around 18% of clients indicated that they had their children living with them, and 10.7% lived alone with their children. Females were more likely to have their children living with them and were much more likely than males to be living alone with children. Clients with parenting responsibilities may be underrepresented in the Catalyst program due to the fact that childcare support is not provided, however further evaluation would need to be conducted to determine the level of need for childcare support among the target population for the program. Table 7. Household and accommodation status of participants Household Accommodation status Total (N = 149) Males (n = 89) Females (n = 60) Alone 51 (34.2%) 36 (40.4%) 15 (25.0%) Alone with children 16 (10.7%) 6 (6.7%) 10 (16.7%) Partner 17 (11.4%) 7 (7.9%) 10 (16.7%) Partner and children 10 (6.7%) 5 (5.6%) 5 (8.3%) Parents and other relatives 35 (23.5%) 21 (23.6%) 14 (23.3%) Friends 9 (6.0%) 6 (6.7%) 3 (5.0%) Housemates 11 (7.4%) 8 (9.0%) 3 (5.0%) Total (N = 142) Males (n = 91) Females (n = 51) Renting 91 (65.0%) 55 (60.4%) 36 (70.6%) Own house 41 (28.0%) 27 (29.7%) 14 (27.5%) Rooming / boarding 6 (4.0%) 6 (6.6%) 0 (0.0%) Hostel / supported accommodation 4 (3.0%) 3 (3.3%) 1 (2.0%) Accommodation status The majority of clients reported living in rental accommodation while a little less than 30% were living in their own home with a mortgage. For some clients, living in rental accommodation can be a barrier to accessing a residential rehabilitation service, because some part of the client s income must be used to cover the cost of the residential treatment and therefore the client may not be able to afford to maintain their rental accommodation. This makes the Catalyst program, for which there is no cost to attend, a particularly suitable option for these clients. 60

61 Ten percent of clients reported living in unstable accommodation (e.g. boarding houses, supported accommodation, or no fixed abode ), which may be a risk factor for relapse and non-completion of the program. However, according to the data available to the evaluators, the completion rate for clients living in unstable, or temporary accommodation, was comparable to that for clients with more stable housing arrangements, with 65% (i.e. 15/23) completing the program. 8.3 Regions where client live According to data provided for the evaluation, 85% of the clients that accessed the Catalyst program resided in the North and West Metropolitan region of Victoria (see Figure 13). Although Catalyst is available to clients across the state, barriers associated with daily travel distances have meant that few clients from outside the North and West Metropolitan region have accessed the program. According to staff from ReGen, as well as staff interviewed from other referral services, long travel distances can be prohibitive for clients, due to cost, time or lack of transport options, or can impact negatively on motivation in clients who do attend, resulting in higher dropout rates. While higher dropout rates were not observed among those who travelled longer distances to attend the Catalyst program, it has been suggested that referral services would not refer clients for whom distance might be a barrier, unless the referrer felt the client s motivation was exceptionally high. Eastern Metro, 9, 6% Southern Metro, Barwon South 12, 8% West, 1, 1% North and West Metro, 133, 85% Figure 13. Regions where client live (N = 155) Southern Metro, 12, 8% Eastern Metro, 9, 6% West Metro (outer), 1, 0.5% North Metro (outer), 33, 21% Barwon South West, 1, 0.5% North & West Metro (inner), 99, 64% Figure 14. Regions where clients live (N = 155) Of the clients residing in the North and West Metropolitan region (n = 133), around three quarters (n = 99) were from the inner North and West suburbs, and around one quarter (n = 33) were from the outer Northern Metropolitan suburbs. There was one client from the outer Western Metropolitan suburb of Melton. 61

62 A breakdown of where clients from the inner North and West resided (see Figure 14) shows that, nearly 80% (n = 78) were from the inner Northern municipalities of Moreland, Darebin, Banyule, and Yarra. Clients from the inner Western municipalities of Melbourne, Moonee Valley, (which both border Moreland City where Catalyst is based) Maribyrnong, Brimbank and Hobsons Bay (n = 21) accounted for 20% of the clients from the inner North and West region, suggesting a greater degree of take-up by clients within ReGen s primary catchment area. Brimbank, 1, 1% Hobsons Bay, 2, 2% Maribyrnong, 1, 1% Moonee Valley, 9, 9% Melbourne, 8, 8% Yarra, 10, 10% Banyule, 13, 13% Darebin, 23, 23% Moreland, 32, 33% Figure 15. Regions where clients live (inner North and West Metro; N = 99) Clients from the outer Northern Metropolitan regions, who accounted for around one fifth of all clients for whom data was available, were fairly evenly distributed across the Hume, Nillumbik and Whittlesea council areas. Hume, 13, 40% Nillumbik, 10, 30% Whittlesea, 10, 30% Figure 16. Regions where clients live (outer North Metro; N = 33) 8.4 Employment status According to the data available to the evaluators, more than half of the clients who have participated in Catalyst are unemployed. This matches expectations for the program type as alcohol dependent clients who are still maintaining their employment, particularly those who are employed fulltime, are less likely to be able to commit to a six-week intensive program. These clients are more likely to engage in individual AOD counselling, or shorter term private rehabilitation. The data demonstrates the importance of helping clients of the program to develop job seeking skills, as well as providing assistance in finding employment and training opportunities. The Catalyst program incorporates two sessions that are run by representatives from employment agencies and clients seeking employment are encouraged to utilise these services. Feedback from clients regarding the usefulness of the employment agencies has been mixed, with some clients suggesting they would prefer the opportunity to develop job seeking skills, such as resume writing or interview techniques. As one client commented, I was hoping to get some advice, it s been so long since I ve applied for a job that s where I m at, I need to know how to get an interview, and how to get through one. Some clients have been provided 62

63 Number of clients Caraniche for the Department of Health with assistance, through Catalyst, to apply for training opportunities and reported in follow up interviews that this had, had a positive impact on their recovery. For instance, one client reported, I m back at TAFE and I m studying AOD and Mental Health, and I feel really good about that. It s given me a focus. Table 8. Employment status of participants Employment status Total (N = 140) Males (n = 88) Females (n = 52) Unemployed 78 (55.7%) 54 (56.3%) 24 (40.0%) Sickness benefits 20 (14.3%) 11 (11.5%) 9 (15.0%) Pensioner 16 (11.4%) 9 (9.4%) 7 (11.7%) Fulltime 9 (6.4%) 5 (5.2%) 4 (6.7%) Part time 8 (5.7%) 5 (5.2%) 3 (5.0%) Self employed 6 (4.3%) 3 (3.1%) 3 (5.0%) Home duties 3 (2.1%) 1 (1.0%) 2 (3.3.%) 8.5 Literacy The group-based and psycho-educational nature of the program requires clients to be proficient in verbal and written English language. Individuals with poor literacy or spoken language skills are considered not suitable for the program as their capacity to benefit is limited due to the amount of written learning material and group discussions No difficulty Some difficulty Significant difficulty Figure 17. English language proficiency (N = 154) Verbal Written 8.6 Substance use All Catalyst clients have an alcohol use disorder as their primary substance disorder. Clients may present with alcohol dependence, abuse, or both. The data available to the evaluators indicates that 40% of Catalyst clients use other substances in addition to alcohol. No 60% Yes 40% Figure 18. Does the client use drugs other than alcohol? (N = 168) 63

64 Number of clients Caraniche for the Department of Health Clients that indicated the use of other substances are asked to indicate all other substances used. Cannabis was the most commonly reported substance, followed by tobacco (although anecdotal reports suggest that tobacco use may be underreported here). Males were more likely than females to report using cannabis (35.6% compared to 30.9%) and heroin (9.5% compared to 1.9%) and females were more likely than males to report using tobacco (25.9% compared to 20.5%), amphetamines (11.1% compared to 6.1%) and benzodiazepines (13.0% compared to 3.7%). It is unclear, however, whether this data refers to current or lifetime substance use, as the form used to record the data (Episode Form) does not specify Males (n = 97) Females (n = 61) Figure 19. Substance use (other than alcohol) 8.7 Mental health and ABI According to the data available to the evaluators, 75% of the clients who have participated in the Catalyst program have had an identified mental health issue (but not necessary a clinically diagnosed disorder). Depression, anxiety, or a combination of the two, are common among people with a substance use disorder, and are the most often reported mental health concerns of Catalyst clients. Other mental health issues reported by clients include post-traumatic stress disorder, bipolar disorder, schizophrenia, and eating disorders. Females were more likely than males to have a co-occurring mental health issue (89% compared to 67%). The presence of mental health issues is established during the intake process. For a client to be admitted to the program their mental state must be stable, and appropriate supports must be in place. Staff report that a client s mental state will generally stabilise following detox and provided the client is receiving support and/or treatment for their mental health. Where there are no supports in place Catalyst will assist the client to access the necessary supports and monitor their progress to determine when they are ready to be accepted into the program. No 25% Yes 75% Figure 20. Mental health issues present (N = 146) 64

65 Strengths The Catalyst program has engaged the target population group, including clients across the target age range and a fairly equal number of males and females. Challenges and limitations The client profile suggests that the Catalyst program may not be engaging with a representative sample (compared to the broader population of clients accessing treatment for alcohol use) of young people (i.e. 18 to 30 years), CALD clients, employed clients and clients in married or de facto relationships. Recommendations It may be of interest to investigate possible reasons for the underrepresentation of certain subgroups within the target population, as well as whether any measures should be taken to address the lower rates of engagement of these subgroups. The subgroups include younger clients (i.e. 18 to 30 years), CALD clients, married / de facto clients, and employed clients. The prevalence of Cannabis use among Catalyst clients suggests that linking clients into additional programs, such as Caution with Cannabis, run by ReGen, should be a priority. Chapter 9: Client Outcomes This section presents the findings with regard to treatment outcomes for clients. This includes an evaluation of the extent to which significant treatment goals were achieved within the sixweek program, and maintained at six-months and twelve-months post program. The particular treatment goal areas for which data is presented include alcohol and drug use, psychological and physical health, and social functioning. Findings are also presented with regard to the particular skills learned in the program and the broader impact that participation in Catalyst has had on the families of clients. Evaluation Question XII. Were the desired outcomes achieved at discharge as outlined in the client s individual treatment plans? The following sections discuss significant treatment outcomes for Catalyst clients who consented to participate in the evaluation and completed the program (N = 114; see Figure 21 below). Outcome data is not available for clients that did not complete the program. The completion rate for clients that participated in the evaluation (i.e. 68%) is comparable to the overall completion rate for the program of 70%. 65

66 Did not complete the program, 54, 32% Completed the program, 114, 68% Figure 21. Program completion for clients who participated in the evaluation 9.1 Significant treatment goals The Catalyst program assesses client treatment goals in five areas; substance use, high risk behaviours, physical health, social functioning, and mental health/emotional wellbeing. Within those goal areas, clients are encouraged to select specific and measurable individual goals. At the end of the program (in the final one-on-one session) those goals are reviewed for each client by their Catalyst clinician and these clinician-rated outcomes are presented below Alcohol use To be eligible for admission into Catalyst clients must have a desire to abstain from alcohol during the course of the program and therefore all client goals with regard to alcohol use reflect a desire to maintain abstinence during the program. As can be seen in Figure 22 below, around two fifths of the clients who participated in the evaluation and completed the program, maintained abstinence from alcohol during the sixweek program. Of the three fifths of clients who lapsed during the program, most were either abstaining or consuming less than pre-treatment levels by the final week of the program. Four clients had returned to their pre-treatment level of alcohol consumption. Reduced consumption 34% No change / increase 3% Maintained abstinence 42% Figure 22. Alcohol use at discharge (N = 114) Achieved abstinence 21% Confidence in Maintaining Drinking Goals Clients were asked to complete the Brief Situational Confidence Questionnaire (BSCQ; n = 114) on commencement and completion of the program. The BSCQ is a measure that assesses selfconfidence in resisting the urge to consume alcohol in a variety of situations 6. Clients are asked to imagine yourself as you are right now in each of the situations described in the questionnaire and to indicate how confident they are that they could resist the urge to drink 6 Brief Situational Confidence Questionnaire (BSCQ; 2009) 66

67 Confidence level % Caraniche for the Department of Health heavily in that situation. Examples of situations described in the questionnaire include if I felt that I had let myself down, if I were at a party and other people were drinking and if I started to think that just one drink could cause no harm. Responses are scored as percentages with 0% representing not at all confident and 100% representing very confident. There was a significant increase in scores across all domains of the BSCQ over the course of the six week program, demonstrating that participation in the Catalyst program is capable of improving clients confidence to resist alcohol in a range of situations that are known to trigger the urge to drink. Clients tended to be least confident in resisting the urge to drink in situations where they were testing their self-control, where urges and temptations were present, where there was conflict with others, or when experiencing unpleasant emotions. The Catalyst program aims to address these areas of concern through increasing clients understanding of the various triggers for problematic alcohol use and teaching a range of management and coping strategies designed to help clients both avoid and resist the temptation to drink Pre program Post program Figure 23. Brief Situational Confidence Questionnaire (N = 114) Drug use According to data presented earlier (see chapter 8.5) 43% of clients were using substances other than alcohol prior to commencing treatment. Of those clients who were using drugs and completed the program (n = 49), two thirds either maintained or achieved abstinence from their drug use during the program and 14% reduced their consumption. No change / increase 20% Reduced consumption 14% Figure 24. Drug use at discharge (n = 49) Maintained abstinence 35% Achieved abstinence 31% There was no data available to provide an explanation as to why drug use decreased for some clients but not others. Varying levels of drug dependence may be a factor, or for some clients, changing their drug use may not have been a treatment goal. Another explanation may be the relationship between alcohol and other drug use, i.e. client s who typically use alcohol and other drugs in conjunction with one another may be more likely to reduce their drug use as a 67

68 result of reducing their alcohol use, than clients who did not typically combine their alcohol and drug use. While it is not possible to determine from this data whether the Catalyst program model would be effective in addressing drug use as the primary treatment goal (as opposed to alcohol), there is also nothing in the program model itself to suggest that it would not be an appropriate and effective intervention for drug use. This issue does, however, require further investigation in order to determine, for example, whether any modifications would need to be made to the program model to ensure it was suitable for clients with problems related primarily to their drug use High risk behaviours High risk behaviours include behaviours that could lead to health risks, physical harm or legal consequences, such as driving while intoxicated, unsafe sex practices, and violent or aggressive behaviour, or trigger behaviours such as going to the pub, spending time with certain friends, and boredom. As can be seen in Figure 25, 82% of clients reported a decrease in high risk behaviours associated with their alcohol use following completion of the Catalyst program. Throughout the Catalyst program clients are provided with information to help them identify trigger and high risk situations, and to develop strategies either for avoiding or managing these situations. For instance, clients learn about the health and physical harm risks associated with problematic alcohol use, the relationship of reciprocal harm between alcohol use and mental health, and the negative financial impacts of problematic use, while also being provided with information about available supports and strategies for reducing harm. Feedback from clients demonstrates that participation in the program is capable of increasing clients awareness of the risks and triggers that are relevant to them, and of improving their capacity to manage such situations. For instance, one client commented, I can see the warning signs and potential risks to avoid and I do something about it, and another stated, The planning strategies have helped me to control my drinking in high risk situations. The community-based model of Catalyst provides ongoing opportunities to practice these strategies as clients must manage the risks and triggers present in their everyday lives while attending the program. Clients and staff suggest that the ability to address personal risk and trigger factors during treatment enhances learning and ultimately improves treatment outcomes. As one client commented, We re learning tools and using them as soon as we leave here each day. I ve been to plenty of places before but nothing really worked for me. Here, it s different. I can notice the difference already. Moderate decrease 25% No change 14% Increase 4% Significant decrease 57% Figure 25. High risk behaviour at discharge (N = 114) 68

69 9.1.4 Physical health Problematic alcohol use had various negative impacts on physical health, including the direct effects of alcohol on the body, as well as the effects of unhealthy behaviours such as poor nutrition, sleep disturbance and lack of exercise. The Catalyst program provides clients with information about the physical effects of alcohol, as well as the impacts of an unhealthy lifestyle. Clients are also engaged in health promoting activities such as physical exercise and meditation, and are taught skills for maintaining good heath, such as nutritious meal planning and cooking. As can be seen in Figure 26, 86% of clients reported an improvement in their physical health following completion of the program. The types of improvements reported by clients include better nutrition, regular health checks and medication compliance, improvements in liver function and other health concerns such as cholesterol and hepatitis C, better sleep patterns, regular exercise and a cessation or reduction of tobacco smoking and other substance use. In follow up interviews clients have linked some of these improvements directly to aspects of the Catalyst program. For example, many clients have commented that they have continued to engage in regular exercise following their participation in weekly gym sessions with Catalyst, and have made improvements to their diet using recipes from the cookbook provided in nutrition sessions. Moderate improvement 39% No change 12% Decline 2% Significant improvement 47% Figure 26. Physical health at discharge (N = 114) Social functioning For people who have been using alcohol problematically, poor social functioning may have been a contributing factor in their substance use issues, for example by using alcohol as a way of coping; or may be a consequence, with problematic use leading to relationship breakdowns and social isolation. The Catalyst program helps clients to improve their social functioning in several ways. The group format requires regular and meaningful social interaction and therefore clients must confront social anxieties and develop effective communication skills in order to participate. Program content builds on this experience with a session on communication skills, as well as the Strengthening Relationships unit designed to help clients improve their personal relationships. Clients are also encouraged to re-establish meaningful links with their community through activities such as returning to work or study, engaging in volunteer work or reconnecting with friends and family. Following participation in the Catalyst program, more than 90% of clients reported an improvement in their social functioning. According to staff and client feedback, improvements include reconnecting with the community, meeting personal responsibilities, establishing routine and engaging in meaningful activities. Examples of comments made by clients with regard to their improved social functioning include, My quality of life has improved, not just to do with my substance use, but my understanding of the community and the things that are accessible to me, like libraries, tutors, information and people to support me when I need it 69

70 and before, I didn t have any routine. I eventually want to get back into work and I know that I need to get more organised. This is helping me get used to having a structure in my day again and join the real world. No change Moderate 8% improvement 46% Decline 2% Significant improvement 44% Figure 27. Social functioning at discharge (N = 114) Mental health and emotional wellbeing As discussed in chapter 8.6, around 75% of Catalyst clients have a mental health concern, with many clients presenting with symptoms of depression or anxiety, or both. While clients with significant mental health concerns are encouraged to maintain their existing supports during Catalyst, the program also seeks to help clients improve their mental and emotional wellbeing through various activities. The units in Mood Management, Anger Management and Relaxation help clients to better understand negative emotional states and teach strategies for managing negative emotions and increasing feelings of wellbeing and positivity. Clients also learn about the connections between physical and psychological wellbeing in Nutrition, Gym and Yoga sessions. At completion of the Catalyst program, more than 90% of clients reported an improvement in their mental health and emotional wellbeing. Improvements reported by clients include engaging with appropriate supports (e.g. a psychiatrist, psychologist or counsellor), compliance with medications, alleviation of symptoms, increased positive mood, greater feelings of selfworth, and better management of negative moods. For instance, one client reported, I feel more motivated and in control, and the meditation we learned has helped me with my anxiety. No change 4% Moderate improvement 38% Decline 4% Significant improvement 54% Figure 28. Psychological health at discharge (N = 114) Quality of Life Client were asked to complete the World Health Organisation Quality of Life BREF questionnaire 7 on commencement and completion of the program, to assess improvements in 7 World Health Organisation Quality of Life (WHOQOL) BREF (World Health Organisation, 2004) 70

71 Wellbeing score (raw) Wellbeing score % Caraniche for the Department of Health general wellbeing as a result of participating in Catalyst. This measure has been designed for use with Australian populations and has sound psychometric properties. There are two standalone items measuring self-perceived quality of life and quality of health, and 24 additional items assessing self-perceived quality of life in four domains; physical health, psychological health, social functioning and environment (i.e. how happy and safe one feels in their physical environment). Clients are asked to assess their feelings over the previous two weeks with regard to each of the questionnaire items, keeping in mind their own standards, hopes, pleasures and concerns, and to circle the most appropriate response. Responses are rated on a scale of one to five with higher scores indicating greater satisfaction or wellbeing. As can be seen in Figure 29, Catalyst clients improved in all four quality of life areas following completion of the program. The results of a paired samples t-test show that the increase in all scores post program was statistically significant at the p <.001 level. Australian population norm scores are provided for comparison and demonstrate that the wellbeing of Catalyst clients is poorer than that of the general population, however, participation in the six week program is capable of lessening that gap Physical health Psychological health Figure 29. WHO-QoL BREF (N = 114) All results significant at p<.001 Social relationships Environment Pre program Post program Australian norms The results for the stand alone items are presented in Figure 30. Paired samples t-tests demonstrate a significant improvement (p <.001) in clients perceptions of their quality of life and their health over the course of the six week program. Post program scores for both items are much closer to the Australian population norm scores Pre Program Post Program Australian Norms 0 Quality of life Health Figure 30. WHO-QoL BREF Stand Alone Items (N = 114) All results significant at p<

72 Feedback from clients further supports the positive changes represented in the data. For instance, in follow up interviews clients described a range of improvements in their physical health and general wellbeing following completion of the program. As one client commented, There has been a big change in me. I look different, I look healthier, my whole outlook is so different. When I look back I was a mess, now I m fine, I m healthy and nothing can stop me. It been a 100%, all over change. Clients also commented on the impact participation in Catalyst had on their quality of life, for instance one client commented I can't describe it, it's turned my life around. I was really heading to a bad place. It's amazing the difference and another stated, Catalyst saved my life, my marriage and my family. 9.2 Were outcomes achieved in an appropriate timeframe? Evaluation Question XIII. How effective was the intervention in terms of achieving desired client outcomes in an appropriate timeframe? The Catalyst program produces positive outcomes for its target client group in a relatively short time frame. As one client stated, I can see the change in myself. I ve noticed how quickly I ve begun to feel more at peace. I can t wait to see what happens next! If I feel this good after two weeks, I can only imagine how I will be after six. Other clients have suggested that the transition from treatment into everyday life may be faster for a community-based model than a residential model, and that this made Catalyst a more attractive treatment option. For instance, one client commented, I ve heard people say that it takes up to a year to deinstitutionalise yourself after spending time in resi rehab. I can t wait that long. I need something that allows me to get on with my life and, so far, this program seems to be working. The fact that the Catalyst program is capable of producing change in clients behaviour, attitudes and wellbeing in a relatively short period of time may be attributed to a range of factors including the intensive nature of the program, the range of treatment and support options provided through the program, the mix of theoretical and experiential learning, and perhaps most significantly, the opportunity to integrate this learning through immediate in vivo application of newly acquired skills and knowledge. Strengths The majority of clients who completed the program achieved their treatment goals in the program, including abstinence from alcohol use, better management of triggers and high risk behaviours, and improved mental and physical health and social functioning. Following completion of the program clients reported significant improvements in their confidence to resist alcohol and in their general wellbeing. Client feedback supports the finding that the Catalyst program has been a significant factor in helping clients to achieve their substance use and other treatment goals. Recommendations Client outcome data should continue to be collected and reviewed to enable ongoing evaluation of the program 72

73 9.3 Long term maintenance of treatment goals Evaluation Question XIV. Are case-level outcomes post-treatment maintained at sixmonth and twelve-month follow-up? Investigating the long term impact of treatment is important, given the chronic and recurring nature of substance use disorders. As part of the evaluation clients are contacted six months and twelve months after completing the program and asked to provide feedback about their recovery in the time since Catalyst, with specific questions around maintenance of their individual treatment goals Response rates for follow-up interviews For the six-month follow-up interviews, there were 124 clients that initially consented to participate. Around one quarter of these clients were discharged from the program early (in the first two weeks) and were therefore deemed unsuitable for follow-up (see Figure 31 below). Of the 96 clients that remained eligible for follow-up at six-months, nearly half completed interviews, a small number declined to participate and a large proportion could not be contacted (see Figure 32 below). Not eligible (discharged early from the program), 28, 23% Eligible for follow-up, 96, 77% Figure 31. Eligibility for 6-month follow-up interview (total N = 124) Declined interview, 11, 11% Completed follow-up interview, 45, 47% Could not be contacted, 40, 42% Figure 32. Response outcomes for 6-month follow-up interviews (total N = 96) At the twelve-month follow-up interview, 59 clients were eligible to be contacted (see Figure 33 below). Of these clients, less than one third completed interviews. Two thirds of eligible clients could not be contacted and a small number declined to participate (see Figure 34 below). 73

74 Follow-up not within the timeframe of the evaluation, 37, 30% Not eligible (discharged early or declined 6-month interview), 28, 23% Eligible for 12-month follow-up, 59, 47% Figure 33. Eligibility for 12-month follow-up interview (total N = 124) Could not be contacted, 39, 66% Declined interview, 3, 5% Completed follow-up interview, 17, 29% Figure 34. Response outcomes for 12-month follow-up interviews (total N = 59) It is generally recognised that there are challenges in obtaining longitudinal data for clients of drug and alcohol treatment services. As can be seen in the data presented above, the response rates for follow-up interviews were fairly low, and declined substantially between the sixmonth and twelve-month time points. Among those clients that could not be contacted for an interview, the most common reasons were: there was no response to the phone calls made (i.e. the client did not answer the phone or respond to messages left), the client avoided the interview (i.e. the client agreed to an interview at a later date but then did not respond to the phone call at the agreed time), the client was no longer able to be contacted at the phone number provided and an alternate number was not available, or the phone number provided was incorrect or disconnected. While it is likely that some of the clients who could not be contacted or declined to participate had relapsed, it is not possible to provide any data with regard to the treatment outcomes of these clients. It is important to note, however, that the following data reflects only those clients that were contactable and agreed to participate in an interview six and twelve months after completing the program, and therefore the data is likely to be skewed towards the more positive outcomes for clients of the Catalyst program. Given that the following outcomes are not likely to be reflective of all clients that have participated in the Catalyst program, the findings presented in this chapter must not be quoted out of context due to the risk that misleading conclusions may be drawn. 74

75 Abstinent Controlled drinking Minor lapse/s Major lapse/s Relapsed Abstinent Controlled drinking Minor lapse/s Major lapse/s Relapsed Number of clients Caraniche for the Department of Health Alcohol use Figure 32 below shows client outcomes with regard to their alcohol use at six-months and twelve-months post Catalyst. Outcomes included: maintaining abstinence or controlled drinking goals for the previous six months, minor lapses, which reflected the unintended consumption of alcohol on one or more occasions, major lapses, which reflected a longer, continual period of heavy drinking, and relapse, which indicates that the client was currently consuming alcohol at problematic levels. At six-months, 36% of clients interviewed had maintained their goal (either abstinence or controlled drinking), 44% had lapsed at some time in the previous six-months but were abstaining at the time of the interview, and 20% had relapsed. At twelve-months post Catalyst 47% of clients interviewed had maintained their drinking goals in the previous six months, another 47% had experienced a lapse but were abstinent at the time of the interview, and 6% had relapsed. All clients who had maintained their drinking goals felt that their participation in Catalyst had been a significant factor in achieving these outcomes. For instance, clients commented that the program has, challenged my reasons for drinking or changed my experience of drinking and how I see myself. One client also commented, Thanks to Catalyst I can actually see sobriety as an achievable goal now Figure 35. Alcohol use at follow up 6 months post program (n = 45) 12 months post program (n = 17) It is important to note that clients who reported maintaining abstinence or controlled drinking at twelve months, may or may not have maintained those goals during the first six months post Catalyst. Data presented in Figure 36, however, indicates that most (i.e. 7/9) clients who reported lapsing or relapsing in their twelve-month interview, first lapsed or relapsed in the six-month period following Catalyst. As can be seen in Figure 36 below, most clients who either lapsed or relapsed following Catalyst, did so in the first two months, suggesting that this might be a particularly high risk time for clients and additional supports may be beneficial in helping clients to maintain their drinking goals during this time. 75

76 Number of clients Caraniche for the Department of Health Length of time abstinent Figure 36. For clients who lapsed/relapsed, length of time abstinent since discharge Clients who lapsed or relapsed were asked about the reasons why this had occurred. Most clients reported that they had consumed alcohol as a way of coping either with a personal crisis or psychological distress due to mental health concerns. Nearly one quarter of clients who had lapsed or relapsed felt their motivation to resist urges was low, and a further one quarter of clients lapsed in response to social pressure or during a time of celebration, with Christmas cited by several clients as the time of their lapse. Each of these situations are known triggers for lapse and relapse among people with problems related to their alcohol use. Urges / low motivation 23% Social pressure 12% Personal crises 31% Figure 37. Reason for lapse or relapse (n = 26) Mental health 19% Celebrations 15% Clients were asked whether they felt their participation in Catalyst had helped them to avoid lapses or to improve the way they responded to the lapse. This question was asked because one of the identified benefits of the community-based model over residential models was the ability to use lapses that occurred during treatment as a learning opportunity, which would potentially lead to better treatment outcomes, including better responding to lapses, as well as improved capacity to avoid lapsing. Many of the clients interviewed felt that their capacity to avoid lapsing as well as their response to lapses had improved since participating in Catalyst. For instance, one client commented, I don t beat myself up as much now if I have a lapse and I get back on track a bit faster, and another reported, I ve had some lapses but I ve managed to pull out of them faster than before I did Catalyst. 76

77 Number of clients Caraniche for the Department of Health Drug use Clients were asked about their drug use in the time since completing Catalyst. The majority of clients interviewed reported that they were not using other substances prior to Catalyst, however for those who were, their use had either ceased or reduced since completing the program No drug use (NA) Abstinent Reduced No change No drug use (NA) Abstinent Reduced No change 6 months post program (n = 45) 12 months post program (n = 17) Figure 38. Drug use at follow up Health and wellbeing During the program clients are required to set specific goals in relation to four areas of wellbeing; high risk behaviours, physical health, social functioning and mental health. Clients were asked how they have progressed with their individual goals since completing Catalyst and their responses are presented below. It is important to note that some goals do not allow for progressive improvement as they are either achieved or not achieved, for example quitting smoking or starting a new job, and the response item maintained without change largely reflects these types of goals. Management of high risk behaviours and trigger situations In the six-month follow-up interview 93% of clients reported that they had either maintained (13%) or further improved (80%) on positive changes made at discharge in their management of high risk behaviours and trigger situations. Clients tended to report that the skills and knowledge they had learned in the program, in particular through the CBT coping skills units, had helped them to develop a better understanding of the risks and triggers for them, leading to better management of these situation. As one client commented, I wasn t aware of my triggers before the program but now I have a clear picture and I m thinking about the consequences. In the twelve-month follow-up interview clients were asked about changes in their management of high risk behaviours and trigger situations in the previous six-month period. The majority of those interviewed (94%) reported either maintaining improvements or further improving their management of risk behaviours and trigger situations in the previous six months. Those clients that reported a decline in their management of high risk behaviours and triggers since completing Catalyst, (7% at six-month follow up and 6% at 12-month follow up) had relapsed in this time. 77

78 Number of clients Number of clients Caraniche for the Department of Health Improved No change Declined Improved No change Declined 6 months post program (n = 45) 12 months post program (n = 17) Figure 39. High risk behaviour at follow up Physical health Around 90% of clients interviewed at six-months post Catalyst stated that positive changes achieved during Catalyst with regard to their physical health or health behaviours had either improved (75%) or been maintained (16%) in the time since completing the program. At twelve month follow up, around 85% of clients interviewed stated that they had either continued to make improvements (53% ) or maintained improvements (30%) to their physical health or health behaviours in the previous six-months. Many clients felt that the Catalyst program had helped them to implement healthier routines such as better nutrition and regular exercise. Comments from clients about their physical health included, I have more energy and I m eating and sleeping better, I go to the gym every day and I ve lost weight, and For three months now I have been bike riding and jogging. Among the clients that reported a decline in their physical health since completing Catalyst several had relapsed, two had been diagnosed with a serious illness (unrelated to alcohol use) and one reported sleeping problems as a result of ceasing cannabis use (which might actually be regarded as an improvement in health compared to the impacts of regular cannabis use) Improved Maintained without change Declined Improved Maintained without change Declined 6 months post program (n = 45) 12 months post program (n = 17) Figure 40. Physical health at follow up 78

79 Number of clients Caraniche for the Department of Health Social functioning Around 90% of clients interviewed at six-months post Catalyst reported either further improvements (62%) or maintenance of improvements made during Catalyst (25%) in their social functioning since completing the program. At twelve-month follow-up, all clients interviewed said they had either continued to make improvements in their social functioning in the previous six-month period (70%), or maintained earlier improvements (30%). The types of improvements reported by clients included returning to work or study, engaging in volunteer work, and reconnecting with friends and family. For some clients, who reported that they were isolated and housebound prior to Catalyst, regular outings were a significant improvement in their social functioning. At the six-month follow-up interview, 13% of clients felt their social functioning had declined since completing Catalyst. Of these clients, four had relapsed, one reported suffering some mild social anxiety and the other felt reluctant to attend social events due to the fear of lapsing / relapsing Improved Maintained without change Declined Improved Maintained without change Declined 6 months post program (n = 45) 12 months post program (n = 17) Figure 41. Social functioning at follow up Mental health and psychological wellbeing At six month follow up 96% of clients stated that their mental health had either continued to improve (66%)in the time since completing Catalyst, or that improvements made during the program had been maintained (30%). Only two clients (4%) reported that their mental health had declined in the six months since completing Catalyst and both of these clients had relapsed. At twelve-month follow-up, all clients interviewed reported either further improvements to their mental health in the previous six months (64%), or maintenance of earlier improvements (36%). Mental health issues were a significant concern for Catalyst clients, particularly depression and anxiety, and many of the clients interviewed felt that the Catalyst program had been useful in increasing their awareness of their psychological well-being and in providing strategies for managing negative emotions and increasing positive emotions. The Mood Management and Relaxation units were most commonly quoted as assisting clients to improve their mental health. Comments from clients included, I used to bury things but now I m working through them, I name the emotion and I sit with it, I feel more emotionally stable, the depression isn t continual, I can pick myself up and get back on track, and I m managing my stress and anxiety a lot better. 79

80 Number of clients Caraniche for the Department of Health Improved Maintained without change Declined Improved Maintained without change Declined 6 months post program (n = 45) 12 months post program (n = 17) Figure 42. Mental health at follow up Ongoing use of Catalyst skills In both six and twelve-month follow up interviews the evaluators asked clients which skills learned in Catalyst they still remembered and used after completing the program. Almost all clients indicated that they still used the CBT strategies, particularly the ABC model (antecedent, behaviour, consequence) and the four Ds (delay, distract, drink water and deeply breath). Examples of comments from clients include, I use my CBT to cope with things. It s just changed my whole way of thinking, The four Ds work for me every time. In ten to 15 minutes the feeling is gone and I get on with the day, and The ABCs. I used to write them down but now I use them and talk myself through them. A number of clients described using activity scheduling to maintain daily routine and structure, keeping diaries to increase awareness of risks and triggers, and planning ahead for high risk situations such as parties, which were strategies they had learned through Catalyst. For instance, one client reported that the most useful skills they had learned were problem solving and goal setting. I was able to analyse my lapse and why it occurred. I have a better understanding of myself and another client stated that I focus more on achieving tasks whereas before I would bury them in the too hard basket and have a drink. I achieve what I set out to achieve now. The use of strategies for managing stress, anxiety, depression and anger, learned primarily in Mood Management, Anger Management and Relaxation units, but also indirectly through Yoga and Nutrition, were commonly reported by clients as important in maintaining their drinking goals during stressful times and avoiding trigger situations. For instance, one client commented, I m more motivated and I feel more in control. I do some meditation and that helps me to stay in control and another reported, I use the skills for managing my anxiety and depression, so the relaxation and mood management stuff. And keeping things in balance, the holistic approach. Communication skills learned in the CBT coping skills and the Strengthening Relationships units were reported to have had a lasting benefit for many of the clients, both in improving their current relationships and in helping to form new relationships. For instance, one client commented, Communication skills helped a lot because I used to isolate myself and now I m talking to people more. 80

81 9.3.6 Impact on family and significant others Family members and friends are often significantly affected by their loved one s problems with substance use, a concern highlighted by the emphasis on the need for family inclusiveness in the new reforms for the Victorian AOD sector 8. In follow-up interviews, therefore, clients were asked to comment on how their participation in Catalyst had impacted on their families and significant others. Clients described healthier, more communicative relationships with their friends and loved ones, for example one client commented, I m having conversations with my family, which is something I never used to do. It s great having more confidence to talk to them rather than just not answering the phone and another stated, I have realized the impact my drinking was having on my friends and now the people in my life are beginning to re-engage with me. Clients also report reconnecting with loved ones, for instance one client commented, I ve had contact with my son for the first time in 20 years. For some clients, healing family relationships was a primary objective, and a motivating factor, during their treatment in Catalyst. As one client commented, Improving things with my family was a goal for me in Catalyst and that has happened for me. Clients also indicated that their home lives were more harmonious, for instance one client reported, There s been a dramatic change, from constantly fighting every night with my husband and son, to now, it's a joy to come home. My son is doing better in school because he's calmer. There's harmony in the house. We go out as a family more, we try to do something every weekend. Clients also noted improvements in their parenting. For instance, one client commented, They feel safer if I m not using. I m doing the right thing by my kids and another reported, I m more active and involved with my kids. One of the benefits reported by some clients was increased support from their loved ones due to their greater understanding of the reasons behind problematic alcohol use and the recovery process. For instance, one client commented It's made my wife realise that it's not just a weakness, but a drug that I'm addicted to, and she gives me a lot more support now. She was even supportive when I had a lapse, and in recognising the importance of support and understanding for their significant others, one client commented, My husband liked the Strengthening Relationships sessions because it helped him to understand and he felt supported because he wasn't the only one. Now my drinking is an open subject, it's not taboo, and we can talk about it, whereas I used to put it on the back burner and bottle up my feelings about it. Given that families and friends are the primary support systems for clients, and that problems in intimate relationships can be a major trigger for relapse, the positive benefits of improved relationships are an important outcome. Feedback from clients demonstrates that the increased support they have received from significant people in their lives, and their own improved capacity for developing and maintaining positive relationships, has been a key factor in maintaining their drinking goals over time. 8 Department of Health. (2012). New Directions for Alcohol and Drug Treatment: A Roadmap. Victorian State Government. 81

82 Strengths Six and twelve month follow up interviews with clients suggests that many clients are maintaining, or further improving, the positive changes made in the Catalyst program, and that the skills and knowledge gained in Catalyst have been significant factors in helping clients to maintain those goals. The ongoing support offered through Momentum, as well as the opportunity to repeat the Catalyst program, have been beneficial in helping clients to maintain or re-establish their goals. Recommendations Follow up data should continue to be collected to enable ongoing evaluation of the program. Chapter 10. Client Experience The following section presents findings drawn from the Catalyst client satisfaction and completion surveys, and demonstrates clients evaluation of their experience of the Catalyst program. The satisfaction survey was designed by ReGen as part of their internal evaluation process, and the completion survey was designed by Caraniche to address any of the evaluation questions that were not addressed in the satisfaction survey. The surveys look at the level of support provided to clients at all stages of their engagement with the program (i.e. from the time they are admitted to the program to discharge) and clients perspectives on the quality of service provided by Catalyst. Over the course of the evaluation 112 clients provided responses to the satisfaction survey (around 40% of all participants who commenced Catalyst during the pilot phase) and 47 clients responded to the completion survey (around 17% of all participants who commenced Catalyst during the pilot phase). Data from the satisfaction surveys is presented for each year of the program in order to determine whether there were any changes in clients level of satisfaction with various aspects of the program. Evaluation Question XV. Is the service consumer focused? Does the service provide a treatment in which the client is respected and confidentiality and autonomy are maintained? Were clients satisfied with how they were treated during their participation in Catalyst? The majority of clients who responded to the completion survey agreed, and most strongly agreed, that they were treated with respect and dignity by staff of the Catalyst program, as well as by staff from ReGen with whom they have had contact (e.g. intake workers, administrative staff etc.). One client indicated that they were not treated with respect and dignity, however no further comment was provided by the client in the completion survey. 82

83 % of Respondents % of Respondents Caraniche for the Department of Health Agree 11% Disagree 2% Strongly disagree 0% Strongly agree 87% Figure 43. I was treated with respect and dignity (completion survey; N = 47) The majority of clients who responded to the satisfaction survey reported that they felt welcome in the program (see Figure 44). For instance, one client commented, I just didn t know what to expect but once I arrived I found it most welcoming. Those clients that indicated sometimes feeling unwelcome attributed this either to their own personal anxieties about participating in the group, or the behaviour of other participants. For instance, the client who reported they felt somewhat welcome in the program stated, This is a personal issue for me, the group was welcoming and inclusive but I have trouble with other people especially a group I don t know to 2010 (n = 27) 2010 to 2011 (n = 54) Program Year Figure 44. Did you feel welcome in the program? 2011 to 2012 (n = 31) Definitely Mostly Somewhat Not really Not at all The majority of clients who responded to the satisfaction survey said that they felt comfortable being part of the group during their participation in Catalyst, and there were no reportable differences across the three years of the program to 2010 (n = 27) 2010 to 2011 (n = 54) Program Year 2011 to 2012 (n = 31) Figure 45. Did you feel comfortable in the group during the program? Definitely Mostly Somewhat Not really Not at all Of the few who stated they did not feel comfortable, problems between themselves and other clients were indicated. Clashes between clients cannot be avoided in group programs, and all feedback indicates that staff manage such situations appropriately. Did clients feel their confidentiality and autonomy were maintained? Most of the clients that responded to the completion survey felt their confidentiality was maintained by ReGen during their participation in the Catalyst program. One client indicated 83

84 % of Respondents Caraniche for the Department of Health strong dissatisfaction with the extent to which their confidentiality was maintained, however the client reported that this was raised and addressed by staff. Disagree Strongly disagree 0% Agree 2% 27% Strongly agree 71% Figure 46. My confidentiality was maintained (completion survey; N = 47) All clients who responded to the completion survey reported that their treatment goals for Catalyst were mutually agreed upon by themselves and their appointed Catalyst clinician. Clients are required to develop a set of personal treatment goals in the areas of substance use, high risk behaviours, social functioning and physical and psychological health. These goals should have personal meaning for the client, but also be measurable and attainable within the timeframe of the program. Agree 20% Disagree 0% Strongly disagree 0% Strongly agree 80% Figure 47. Were the goals of treatment mutually agreed on? (completion survey; N = 47) Most clients who responded to the satisfaction survey felt they were given enough choice about the units they participated in (see Figure 48 below) to 2010 (n = 27) 2010 to 2011 (n = 54) Program Year 2011 to 2012 (n = 31) Definitely Mostly Figure 48. Were you given enough choice about the parts of the program that were relevant to you? Somewhat Not really Not at all In the first and second years of the program, clients were providing feedback that indicated there was too much choice around which units clients should attend, and many felt that more units should be compulsory. Reasons given for this included a negative impact on group dynamics when non-attendance rates were high, and the view that units such as Mood Management, Anger Management, and Skills Practice, were as beneficial to all clients as the compulsory CBT Coping Skills units. For instance, one client commented, I think some of the activities like the Skills Practice and Mood Management could be compulsory as I think I got more out of these than some of the CBT sessions. As a result of this feedback the Mood Management and Anger Management sessions were made compulsory and Catalyst staff report that the client response to this decision has been positive. 84

85 % of Respondents % of Respondents Caraniche for the Department of Health Were clients generally satisfied with the Catalyst program? Over the course of the pilot program, around three quarters of all clients who responded to the satisfaction survey felt the program met their expectations completely, while the remaining 25% felt the program met most or some of their expectations regarding treatment (see Figure 49 below) to 2010 (n = 27) 2010 to 2011 (n = 54) Program Year Figure 49. Did the program meet your expectations? 2011 to 2012 (n = 31) Definitely Mostly Somewhat Not really Not at all Feedback consistently demonstrates that clients enjoy, engage with and benefit from the Catalyst program. A number of clients indicated that the program exceeded their expectations. For instance, one client commented, I achieved more than I expected, especially with guidance in setting realistic goals. Many clients also felt that the range of units offered was a benefit of the program. For example, one client reported, I was surprised how many components have helped me in all life areas rather than just to do with alcohol which is of utmost importance and benefit to me, while another commented, I wasn t sure what to expect but I have benefited from all aspects of the course. The majority of clients that responded to the satisfaction survey felt the program had helped them to make the changes they wanted to make. Of the few clients that reported some level of dissatisfaction with the program in terms of meeting their goals, some felt the program was not long enough and others felt that more one on one time was needed. For instance, one client commented, (I would have liked) closer follow up on the ITP, more one on one contact time to 2010 (n = 27) 2010 to 2011 (n = 54) 2011 to 2012 (n = 31) Definitely Mostly Somewhat Not really Not at all Program Year Figure 50. Did you feel the program helped you to make the changes you wanted? Clients are also asked in focus groups whether there are any unmet needs and to date no unmet needs have been identified by clients who have attended the focus groups. It should be noted, however, that because focus groups are not attended by all clients, the views of all clients are not represented by this feedback. 85