May, 2011 An Evaluation of the Bruce Grey Hospital-Community Smoking Cessation Program Prepared for Keystone Child, Youth and Family Services & Partners By The Centre for Community Based Research www.communitybasedresearch.ca
Bruce Grey Hospital-Community Smoking Cessation Program Acknowledgements The Bruce Grey Hospital-Community Smoking Cessation Program is a project by Keystone Child, Youth and Family Services, Grey Bruce Public Health Unit, Grey Bruce Health Services, Hanover and District Hospital, South Bruce Grey Health Centre, and community partners. The project is funded by Health Canada For more information about this project please contact: Jennifer Sells, Director of Service Keystone Child, Youth & Family Services 845 2 nd Ave. E. Owen Sound, ON http://www.keystonebrucegrey.org A report written by: Jonathan Lomotey, Ph.D. Lexie Metelka Rebecca Stearns-Smith Centre for Community Based Research 73 King Street W, Suite 300 Kitchener, Ontario, N2G 1A7 www.communitybasedresearch.ca Contact: jonathan@communitybasedresearch.ca 519-741-1318 ext. 232 1
Bruce Grey Hospital-Community Smoking Cessation Program TABLE OF CONTENTS Introduction... 3 Evaluation Purpose and Questions... 4 Methods... 4 Participants... 4 Data gathering... 4 Data analysis... 5 Program Description... 5 Main objectives... 5 Program activities... 5 The Bruce Grey Smoking Cessation Program s Logic... 8 Next Steps in this Evaluation... 8 Qualitative evaluation methods... 9 Quantitative evaluation methods... 9 Conclusion... 10 Appendix 6: About the Centre for Community Based Research... 11 2
Bruce Grey Hospital-Community Smoking Cessation Program Introduction The Bruce Grey Hospital-Community Smoking Cessation Program is offered by Keystone Child, Youth and Family Services and Grey Bruce Public Health in partnership with Grey Bruce Health Services, Hanover and District Hospital, South Bruce Grey Health Centre, and community partners. The program is based on the Ottawa Heart Model s approach to smoking cessation programming: It provides smoking cessation counselling, treatment and supports to patients on admission at partner hospitals. The program also has a unique community component through which it provides ongoing supports to patients upon discharge. The evaluation of Bruce Grey Hospital-Community Smoking Cessation Program was based on the TEIP 1 evaluation tool. The TEIP evaluation tool outlines five major stages in planning and implementing a program evaluation as follows: 1. Laying the ground work which involves a. selecting of an evaluation team b. assessing organizational capacity and resources for the evaluation 2. Laying the foundation for an evaluation a. developing a program logic model b. defining evaluation questions 3. Building a frame a. building an evaluation framework b. documenting an evaluation plan 4. Completing the interior a. Collecting and analyzing data b. Documenting evaluation report 5. Holding an open house a. Disseminating findings This project covers stages 2 and 3 of the TEIP tool 2. Stages 2 and 3 involve developing a program logic model, building an evaluation framework, and developing an evaluation plan for the project. 1 Towards Evidence-Informed Practice 2 Stage one was covered by Keystone Child, Youth and Family Services in identifying and contracting CCBR to conduct the evaluation and providing the evaluation resources. 3
Evaluation Purpose and Questions The purpose of this evaluation was two-fold: 1. To understand the community based smoking cessation program that has evolved over the past three years in Bruce Grey 2. To develop an evaluation framework for the program The main evaluation questions were: 1. How did the program for tobacco cessation start in Bruce Grey, and how has it evolved over the past three years? 2. What are the main activities, processes, and intended outcomes of this program? 3. On what should an outcome evaluation of Bruce Grey Hospital-Community Smoking Cessation Program focus? Methods Participants The participants in this evaluation included managerial staff from Keystone Child, Youth, and Family Services, Grey Bruce Public Health Unit, and a Pharmacist from the Owen Sound Family Health Team. Data gathering Two main data gathering methods were used in this evaluation: document review and a discussion forum. Document review Documents related to the Bruce Grey Hospital-Community Smoking Cessation Program were reviewed to help understand the program s development, activities, processes and intended outcomes. Discussion forum A logic model development session was held with administrative staff from Keystone Child, Youth and Family Services, Grey Bruce Public Health Unit and community partners. This session provided an opportunity to develop a collective understanding of the program s activities, processes, and intended outcomes, and the logic underlying it. The session was facilitated by the evaluators from the Centre for Community Based Research, Kitchener, Ontario. 4
Data analysis Content analysis was performed on the data gathered through the document review and logic model development session to identify the program s main activities, intended outcomes and their indicators, and to understand the logic underlying them. Program Description The Bruce Grey Hospital-Community Smoking Cessation Program provides hospital-based smoking cessation intervention and supports to patients on admission. It involves identification of smokers, and offering them counselling, treatment, and supports to enable them to cope with withdrawal, reduce or quit smoking. The program also has a community based component that provides on-going supports for consenting patients upon discharge. Through this component, community practitioners and service providers continue to provide discharged patients with continued treatment, counselling, and supports. Main objectives The main objectives of the Bruce Grey Hospital-Community Smoking Cessation Program are to increase the number of hospital patients who quit smoking, decrease relapse rates among patients who quit smoking during admission, and ultimately, to decrease tobacco use in Bruce Grey. Program activities The Bruce Grey Hospital-Community Smoking Cessation Program seeks to achieve its main objectives through three sets of activities: 1) system change activities, 2) smoking cessation programming, and 3) ongoing evaluation. The activities under each of these components are described below: System change activities System change activities include developing a network/support system among hospitals, agencies, and community practitioners; communication and publicity regarding the importance of smoking cessation; changing the hospital intake process to incorporate offering patient support in managing or quitting smoking; and changing the hospital information system to incorporate data regarding patient smoking status. The program also involves training nurses and staff, and ongoing evaluation. The training piece relies on pre-existing training for nurses and staff and includes: STARSS and TEACH training for community practitioners, RNAO Smoking Cessation Best Practice Guidelines training for nurses, and Brief Counselling for Tobacco use, using PTCC for hospital staff and community practitioners. 5
Smoking cessation programming Smoking cessation programming has two subcomponents: a hospital subcomponent and a community subcomponent. The activities of the hospital subcomponent are: identification of tobacco users upon admission; provision of treatment in the form of brief advice, counseling, and pharmacotherapy; and provision of self-help materials. The aim of counselling is to assist patients in coping with withdrawal symptoms, help them to manage and/or quit tobacco use, and provide them with information on tobacco cessation, based on their stage of change. Treatment involves pharmacotherapy. The self-help materials are in the form of Quit Kits that help patients to cope with withdrawal and the Canadian Cancer Society s One Step At A Time Booklets. The community subcomponent consists of ongoing smoking cessation counselling, treatment and support by community practitioners and agencies, including Smokers Helpline. Community practitioners and agencies receive referrals from the participating hospitals. Smokers Helpline is a free service, available to everyone in the community, that all patients are encouraged to access. Ongoing evaluation The evaluation component of the program involves conducting follow-up calls to discharged patients who give their consent to be contacted. Through these follow-up calls, information is gathered about patients tobacco management and/or cessation after discharge as well their support needs, and then provide them further information about available supports. The information gathered through these calls is used to further develop and strengthen the program. The program logic model in the next section depicts the Bruce Grey Hospital-Community Smoking Cessation Program s activities and intended outcomes. 6
Figure 1: Bruce Grey Hospital-Community Smoking Cessation Program Logic Model 7
The Bruce Grey Smoking Cessation Program s Logic System change activities are aimed at increasing consistency in smoking cessation programming among partner hospitals, agencies and organizations. The building of a network/support system, information sharing, and communication and publicity are expected to increase collaboration and referrals among hospitals, community practitioners and community organizations; training for nurses and staff is aimed at increasing their understanding of the system change around smoking cessation programming, and increasing their confidence, comfort, and sense of responsibility in providing smoking cessation counselling and support to patients. Smoking cessation programming activities aim at helping patients to cope with withdrawal symptoms, manage and/or quit tobacco use. The activities that are carried out under the Hospital Component, namely counselling, self-help, and pharmacotherapy are expected to increase patients awareness of the benefits of quitting smoking and increase their awareness of community resources that support smoking cessation. Counselling, self-help, pharmacotherapy during admission, and use of community supports are expected to increase patients capacity to cope with withdrawal, manage and or quit smoking. Referrals to community practitioners and agencies for ongoing counselling, treatment and supports is expected to increase the availability and use of supports and help patients in their efforts to quit smoking or reduce relapse rates among patients who quit smoking during admission. Ongoing evaluation is expected to provide continuous feedback into the system to strengthen the smoking cessation program towards achieving its objectives. Ultimately, the activities under all three program components, namely, system change activities, smoking cessation programming, and the ongoing evaluation are expected to increase the number of patients who quit smoking, and lead to an overall decrease in tobacco use in Grey and Bruce. Next Steps in this Evaluation The next steps in this evaluation will cover steps 4 and 5 of the TEIP evaluation tool. These are: 4. Completing the interior a. Collecting and analyzing data b. Documenting evaluation report 5. Holding an open house 8
a. Disseminating findings Step 4 would focus on assessing the achievement of measurable indicators of the program s outcomes: reduction of tobacco use and the increase in tobacco cessation rates. The main evaluation questions to be considered are: 1) To what extent has the Bruce Grey Hospital-Community Smoking Cessation Program increased consistency in smoking cessation programming? 2) To what extent has the Bruce Grey Hospital-Community Smoking Cessation Program achieved its main objectives of helping people reduce or quit smoking? For an effective evaluation of the Bruce Grey Hospital-Community Smoking Cessation Program, we propose the use of sequential mixed methods approach: using both qualitative and quantitative methods for gathering and analyzing data. Evaluation activities may include: Qualitative evaluation methods Key informant Interviews: Key informant interviews to be held with program administrators and facilitators to assess system change around smoking cessation programming. Focus groups: Focus groups to be held with nurses and program staff to assess the program s development, implementation process, as well as its outcomes from their perspectives. Quantitative evaluation methods Hospital and agency data: Program data gathered during intake, programming and follow-up for measuring the following performance indicators identified by the partners: Prevalence/burden study - # of smokers/total admissions X 1 month # Health Care Practitioners trained and aware/# employees on unit (Unit metrics) # identified who received counselling/ # smokers identified on admission (Pt metrics) # identified who received resources and community referral/ # smokers identified on admission (Pt metrics) # of patients who used community resources upon discharge 6 months and 12 months Follow up - # of patients in follow-up who quit at 6 months and 12 months/# patients identified (success metric) Readmission rates of individuals who had been previously admitted for smoking related illnesses. The Participant Survey: A participant survey should be conducted for self-assessment of client outcomes. The survey should gather information about current smoking status, awareness of tobacco dependence, motivation to reduce or quit smoking, sense of access to smoking 9
cessation resources, and patients assessment of the Bruce Grey Hospital-Community Smoking Cessation Program. Conclusion Information gathered in this phase of the evaluation indicates that the Bruce Grey Hospital- Community Smoking Cessation Program is a unique program that combines the Ottawa Heart Model s approach to smoking cessation programming with a community based approach that ensures ongoing support to patients after discharge. Through this program Keystone Child, Youth and Family Services and Grey Bruce Public Health Unit are building partnerships with hospital corporations, community practitioners and community organizations for delivering consistent smoking cessation programming in Bruce Grey. Based on the findings of this phase of the evaluation, we believe the program is capable of generating meaningful data for an outcome evaluation within the next six to eight months. An outcome evaluation of the project should involve the use of both qualitative and quantitative methods in gathering data to answer the following evaluation questions: 1) To what extent has the Bruce Grey Hospital-Community Smoking Cessation Program increased consistency in smoking cessation programming? 2) To what extent has the Bruce Grey Hospital-Community Smoking Cessation Program achieved its main objectives of helping people to reduce or quit smoking? The evaluation should include an analysis of program data gathered at intake, during programming, and at follow-up, as well as information gathered through key informant interviews, focus groups and a participant survey. 10
Appendix 6: About the Centre for Community Based Research The Centre for Community Based Research (CCBR) is an independent, non-profit organization with over 25 years experience in community-based research and program evaluation. Located in Kitchener, Ontario, our projects are local, provincial, national and international in scope. In all Centre work, we use a participatory, action-oriented approach which is well suited to developing relevant and innovative solutions to address important social issues and meet individual, organizational and community needs. Our Mission Our Centre is committed to social change and the development of communities and human services that are responsive and supportive, especially to people with limited access to power and opportunity. Demonstrating leadership through research, education and community involvement, our Centre stimulates the creation of awareness, policies and practices that advance equitable participation and integration of all members of our community. Distinctive Features Our organization is Independent. It is an incorporated non-profit organization guided by a volunteer board of directors, and is not permanently affiliated with any funder or institution. Our approach is Collaborative. We collaborate with academics, service organizations and government to support innovations in human service policy and practice. We see research as an opportunity to give voice to less powerful stakeholders and as a means to foster social cohesion. Stakeholders set the agenda for each research project, and our approach emphasizes ongoing feedback in all directions throughout the process. Our organization is Responsive. We design multi-phase research projects so that our methodologies can be adapted to changing situations. We apply lessons across different projects and communities, and disseminate findings through our newsletters, workshops, and web pages. Our organization is Experienced. It was founded 27 years ago, and has an established reputation for managing projects successfully. Our team includes experienced researchers, facilitators and academic instructors, and we have produced a body of published scholarly and popular writings. Our team is Multidisciplinary. It includes members with graduate degrees in social work, community psychology, developmental and social psychology, women's issues, sociology, and planning. Our team has content expertise in diverse issues including, immigration and cultural diversity, settlement and integration, social housing, family support, community mental health, disability issues, community safety and violence prevention, poverty and international development. For more information about CCBR, visit our web site at: www.communitybasedresearch.ca 11