Certified Recovery Peer Specialist Paraprofessional Training 2017 Course Evaluation Summary Funded by HRSA # 16-193 Behavioral Health Workforce Education and Training for Paraprofessionals Evaluation Executive Summary
Jacksonville University HRSA Grant Recovery Peer Specialist Training Course Sharon T. Wilburn, PhD Melissa Covey, MPH PURPOSE AND DESCRIPTION The purpose of this enhanced Certified Recovery Peer Specialist (CRPS) training was to introduce paraprofessionals to a recovery-oriented and multicultural perspective on behavioral health care and to concurrently increase competency in the field of peer-to-peer recovery support, thus enhancing students marketability. This enhanced training provided a new track for individuals interested in gaining the advocacy, mentoring and recovery support skills to work effectively with individuals and families at risk of mental illness, substance abuse or suicide. Participants were trained to advocate on behalf of the client and serve in the role of a peer mentor in a variety of community-based settings. This training meets the educational requirements for the Florida Certification Board s credential as a Certified Recovery Peer Specialist. In addition to the educational component this certification requires 1) a total of 500 hours of experience providing peer support services and 2) a passing score on the certification exam. This report provides outcome feedback from program documentation and from students who completed this course. KEY FINDINGS 86.7% (52 of 60) enrollees graduated from the CRPS training. Across the board, statistically significant gains (p=.000, at 95% CI) were demonstrated in student's perceived knowledge and confidence in their practical application of CRPS concepts. 93.7% of graduates surveyed intend to take the Florida Certification Board s recovery peer specialist exam. RECRUITMENT, ENROLLMENT and RETENTION 1 CRPS Course Promotion in the Community 2 1 Source: CRPS Program Student Recruitment and Enrollment Records. 2 Sources: Program documentation and external evaluator observation of community events. *APPENDICES AVAILABLE UPON REQUEST
Promotion occurred during a number of community events and through informal communications between the CRPS course key stakeholders and members of the community. Two community presentations were observed by the external evaluator: February 28, 2018: The Opioid Epidemic and The Role of Peer Recovery Specialists hosted by Drug Free Duval. June 1, 2017: Best Practices for Integrating Recovery Peer Specialists into Behavioral Health Care Setting hosted by LSF Health Services and Jacksonville University. o Responses on the June 1, 2017 training evaluation i demonstrated high levels of satisfaction with the presenters, content and activities, and the intent of attendees to integrate recovery peers into their organizations. Applicants LSF received 113 unduplicated, complete applications between December 1, 2016 and June 30, 2017. Review of data collected on application forms found: More applicants were female (n=77, 68.1%); White (n=67, 59.3%) or Black/African-American (N=34, 30.1%); and between the ages of 25-44 years old (n=48, 42.5%) or 45-60 years old (n=45, 39.8%) 111 applicants indicated they had a lived experience that made them eligible to serve as a peer specialist. o 76 (67.3%) had lived experience as an adult in recovery for a minimum of 2 years from a substance abuse or mental health condition. o 29 (25.7%) had lived experience as a family member or caregiver to another individuals who has or is in recovery from a substance abuse or mental health disorder. o 6 (5.3%) had lived experience as a veteran in the armed forces who has been in recovery for 2 years from a substance abuse or mental health condition. Some applicants indicated they had experience in the peer recovery field o 30 (26.6%) had experience as a peer specialist, with 14 (12.4%) currently employed as a peer specialist, and 16 (14.2%) currently volunteering as a peer specialist. o 17 (15.0%) applicants stated they had received an offer of employment, pending successful completion of the Certified Recovery Peer Specialist (CRPS) 40-hour training. 83 (73.5%) stated they had demonstrated leadership and support experience related to mental health and/or substance use conditions. Enrollees Sixty of the 113 applicants were accepted and enrolled in one of three CRPS classes that were offered between January 2017 and October 2017. Review of program documentation on enrollees found: More enrollees were female (n=39, 65.0%); White (n=32, 53.3%) or Black/African-American (n=18, 30.0%); and between the ages of 25-44 years old (n=26, 43.3%) or 45-60 years old (n=22, 36.7%). Retention 52 (86.7%) of the 60 enrollees completed the CRPS training.
PROGRAM FIDELITY EVALUATION Between January 2017 and September 2017, the CRPS training provided three cohorts of students with o Core CRPS Content guided by the Helping Others Heal Peer Support Training Manual (40 hours); o One Love Escalation Workshop (2 hours); o Trauma Informed Care training (2 hours); o Cultural Competency training (2 hours); and o Youth Mental Health First Aid (8 hours). CLASSROOM OBSERVATION The results of 12 classroom observations -four during each cohort s course 3 The facilitators delivered the CRPS core and enhanced content as intended during 11 (93.8%) of the 12 classroom observations. The one observed session during which content was not delivered as intended was due to a sudden, long-term illness of a lead facilitator. One of the two scheduled chapters were reviewed as planned during that session, with the other chapter reviewed as part of the content covered during the next session. On average, facilitators demonstrated each of the 16 instructional characteristics 93.4% of the time. Modeling training skills was the instructional characteristic used least by the facilitators (50.0% of facilitators modeled training skills). Reasons the instructional characteristics were not demonstrated included the characteristics were demonstrated by the co-facilitator during the session, or the content did not require a particular instructional characteristic. The observer noted many occasions during which the students were enthusiastically engaged in learning and class activities. Four separate cohorts were originally scheduled to receive training between January and September 2017. With the departure of one of the lead core CRPS content facilitators in the middle of the second cohort class, the number of classes was reduced to three, with more students enrolled in the third cohort to ensure the program served the required 60 students. 4 Two changes occurred in the training schedule for the third cohort in reaction to school closure due to a hurricane. One session was lengthened to ensure students received the required 40 hours of Core CRPS Content training. One of the enhanced CRPS Content sessions was dropped from the schedule so that the course could end on its scheduled date. 5 SATISFACTION WITH COURSE 5 Distributed as part of the end of course questionnaire, this survey used a four-point agreement scale, where 1 indicates strong disagreement and 4 indicates strong agreement with statements related to students satisfaction with the course, instructors, and course setting. 3 Classroom observations, Appendix B 4 Program documentation 5 CRPS Training Evaluation Survey, section Satisfaction with Course, Appendix C.
Students reported strong satisfaction with the course. Overall group weighted average scores were: Satisfaction with facilitators: 3.87 Satisfaction with training content and activities: 3.78 Satisfaction with training setting/location: 3.68 OUTCOME EVALUATION Two types of surveys were administered to evaluate course outcomes. Retrospective pre-/postsections were administered as part of the end of course survey to collect student s perceptions of their knowledge and confidence in applying that knowledge before they took the course and at the end of the course. 6 A graduate follow-up survey was administered during the October 2017 graduation ceremony to evaluate the persistence of students attitudes towards, intent to pursue, and progress towards taking the Florida Certification Board s recovery peer specialist exam. 7 STUDENT DEMOGRAPHICS ON END OF COURSE SURVEY 7 The demographic data provided by 45 survey participants mirrored class enrollment, with: Females representing 66.7% of respondents, 55.6% identifying themselves as White, and 28.9% as Black/African-American. Age categories were recorded differently from the enrollment data. 84.4% of students were within the 30 to 60 age range. Slightly less than half (46.7%) of respondents held Baccalaureate, graduate, or professional degrees. Slightly less than half (46.7%) of respondents were employed in full time positions. STUDENT DEMOGRAPHICS ON GRADUATE FOLLOW-UP SURVEY 8 Comparing the demographics of the 37 graduates who completed the CRPS Graduate Follow- Up Survey with the total population of students that had participated in the training: A slightly higher proportion (73.0%) of the graduates who completed the survey were female. The proportion of graduates who identified as being White (54.1%) or Black/African-American (29.7%) mirrored the total population of students. 83.8% of graduate respondents were 30 years or older, which is similar to the age range of the total population of students. Slightly less than half (45.9%) of graduate respondents held Baccalaureate, graduate, or professional degrees. 38.9% of graduate respondents were employed in full time positions. STUDENT PERCEPTIONS OF CONTENT KNOWLEDGE AND PRACTICAL APPLICATION BY SUBSCALE 9 6 CRPS Training Evaluation Survey, section CRPS Training Outcome Questionnaire, Appendix D. 7 8 9 CRPS Training Evaluation Survey, section CRPS Training Outcome Questionnaire, Appendix D.
Survey questions related to the chapter objectives listed in the Helping Others Heal Peer Support Training Manual (Core CRPS Content) were answered by 45 students. Survey questions related to the objectives of the enhanced content sections (Enhanced CRPS Content) were answered by between 22 to 45 students Across the board, statistically significant gains (p=.000, 95% CI) were demonstrated in student's level of agreement with statements about their perceived knowledge and practical application. On average, group subscale scores increased by one categorical level, with pretest averages ranging from 2.6 (midway between "disagree" and agree") to 3.6 (midway between "agree" and "strongly agree") and post-test averages ranging from 3.0 ("agree") to 3.8 (slightly lower than "strongly agree"). Appendix B (a separate document) provides statistical results for each content subscale. Change in group average scores between before taking the course and then after taking the course for each subscale: o Core CRPS Content Knowledge increased from 2.8 to 3.8 and Practical Application increased from 2.8 to 3.7. o Enhanced CRPS Content, Cultural Competency Knowledge increased from 2.7 to 3.7 and Practical Application increased from 2.7 to 3.6. o Enhanced CRPS Content, Relationship Violence Knowledge increased from 3.0 to 3.6 and Practical Application increased from 2.8 to 3.6. o Enhanced CRPS Content, Trauma Knowledge increased from 2.7 to 3.7 and Practical Application increased from 2.6 to 3.6. o Enhanced CRPS Content, Youth Mental Health Knowledge increased from 2.6 to 3.6 and Practical Application increased from 2.8 to 3.6. CAREER ATTITUDES AND INTENT Distributed as part of the end of course 10 and graduate follow up 11 questionnaires, this survey uses a four-point agreement scale, where 1 indicates strong disagreement and 4 indicates strong agreement with statements related to their attitudes toward the CRPS field and their intent to pursue a career as a CRPS. Attitudes towards CRPS Field: 11 93.7% of students agreed or strongly agreed that the training prepared them to take the exam (43 out of 44 responses). Students showed a strong positive attitude towards the CRPS position (group-weighted average of 3.84 across attitude statements). 40.0% of the first cohort of students demonstrated a misconception about the role of the CRPS on a mental wellness team. This misconception was substantially reduced by the end of the grant program, with only 14.0% of completed end of course surveys from the three cohorts disagreeing or strongly disagreeing with the statement, A person who meets with a Certified Recovery Peer Specialist may also be referred to a Counselor. Intent to Pursue Career as CRPS: Immediately after completing coursework 11 and reaffirmed by graduates during their graduation ceremony in October 2017, 12 participants had strong intent to obtain the necessary 10 CRPS Training Evaluation Survey, section Career Attitudes and Intent, Appendix E. 11
500 hours of experience and take the Florida Certification Board s recovery peer specialist exam. Immediately after completing coursework, 93.2% (41 of 44) of students agreed or strongly agreed they already had or planned to get the 500 hours of experience required by the Florida Certification Board. 11 By October 2017, 45.9% (17 of 37) of graduates indicated they had 500 hours of experience, 2.7% (1 of 37) had more than 250 hours, and 48.6% (18 of 37) had less than 250 hours. 12 o 77.8% (14 of 18) of graduates specified how they planned to get the experience; o 5.6% (1 of 18) was concerned about having time to get the hours; and o 16.7% (3 of 18) indicated they were not sure how they would get experience. Those three were all members of the third cohort class that had finished coursework the week before. Immediately after completing coursework, 93.0% (40 of 42) of students agreed or strongly agreed they had already taken or planned to take the Florida Certification Board s recovery peer specialist exam. 11 By October 2017, 93.7% (36 of 37) of graduates still intended to take the exam. 12 o 20.6% (7 of 34) within the next month; o 23.5% (8 of 34) within three months; o 35.3% (12 of 34) intended to take the exam within six months; o 14.7% (5 of 34) within the next year; and o 5.9% (2 of 34) intended to take the exam at some point later than a year. Intended Work Settings 12 By October 2017, 18.9% (7 of 37) of graduates currently worked as a recovery peer specialist. When asked to identify all settings in which they intended to work as a CRPS, students chose a Drug Addiction Treatment Practice/Facility or Community Agency (each, n=16, 42.2%), Other Mental Health Treatment (n=11, 29.7%), Youth Treatment Mental Health Practice/Facility (n=9, 24.3%), Hospital (including Veterans; n=9, 24.3%) and a Rural Mental Health or Addiction Treatment setting (N=6, 16.2%) Barriers to Becoming a CRPS 13 Barriers identified by graduates included Time Constraints (5 of 24, 20.8%); Legal Background / need exemption (4 of 24, 16.7%); Financial (2 of 24, 8.3%); Limited recovery peer opportunities / other job options (2 of 24, 8.3%). 45.5% (10 of 22) graduates stated they did not experience barriers. Most students with barriers were actively seeking resolution to those barriers: o 13.6% (3 of 22) graduates with barriers indicated their barriers had been resolved; and o 40.9% (9 of 22) graduates described a plan to resolve or were in the process of resolving barriers. STUDENT SUGGESTIONS FOR COURSE IMPROVEMENT 12 13
The following summarizes responses to this question from all CRPS training evaluation surveys administered during January 2017 through October 2017. Please note that some students responded to this question at multiple points in time. In all 71 responses that provided suggestions for course improvement were collected from the Core CRPS Content and Enhanced CRPS Content end of course surveys and from the Graduate Follow-Up survey. A little more than half (54.9%, 39 of 71) of the responses indicated participants felt no change was needed or that the class was of good quality. 14.1% (n=10) of the responses dealt with the course schedule. 7 of these responses recommended longer sessions to make the course shorter while 2 stated more time was needed to learn the material. I would do longer classes and shorter length of time, e.g. 5 x 8 hr. days, 10 x 4 hr. days 9.9% (n=7) of the responses dealt with course activities. More hands on activities the one we did were very helpful More roleplay Practice stories earlier in the training. 8.5% (n=6) dealt with course content. Focus more on content/less on discussions (personal) Have more curriculum aimed at Family. An instructor with family experience. Include a 2-day training on how to prepare your exemptions 7.0% (n=5) recommended more connection to opportunities to gain experience or work in the field. By getting persons from the Community that are willing to hire as soon as we finish our training Give examples of where to acquire hours. Have a client who has worked with a peer specialist speak to the class LESSON LEARNED FROM EVALUATION ACTIVITIES Student attendance during the Enhanced CRPS Content sessions dropped dramatically during each cohort. For future courses of this type, consider modifying survey administration so knowledge and practical application outcome data are collected specifically for each Enhanced CRPS Content session. CONCLUSION Overall, the data indicate the program is successful in increasing the knowledge and confidence of students who want to pursue a position as a Certified Recovery Peer Specialist. Class instructors should continue to do a great job!.