Evidence-Based Practice Fidelity Site Visit Tools This product was supported by Florida Department of Children and Families Substance Abuse and Mental Health Program Office funding.
Evidence-Based Practice (EBP) Fidelity Site Visit Tools 1. EBP Fidelity Site Visit Agenda 2. EBP Fidelity Site Visit Provider Checklist 3. EBP Fidelity Survey 4. Agency Self-Assessment Summary 5. Client Review Checklist 6. Self-Assessment Review Tool 7. Exit Interview Form 8. Agency Review of EBP Site Visit 9. EBP Site Visit Peer Mentor Feedback 2
FL Department of Children & Families Evidence-Based Practice Fidelity Initiative Tool Review Site Visit Agenda 9:00 AM Welcome and Overview Introductions Peer Mentor Review of Process Peer Mentor EBP Presentation Agency Staff Q&A 10:00 AM Program and Client Record Review Peer Mentors LUNCH Working lunch 3:00 PM Team Meeting Findings & Comments Peer Mentor Team 4:00 PM Exit Interview Findings & Comments Peer Mentor Team Recommendations of Priorities Technical Assistance Other Feedback This product was supported by Florida Department of Children and Families Substance Abuse and Mental Health Program Office funding. 3
Evidence-Based Practice (EBP) Fidelity Site Visit Provider Checklist The following activities should be completed by the provider prior to the one day EBP site visit. Complete a self-review of an evidence-based practice in use at your agency using the Evidence-Based Practice Fidelity Self-Assessment Survey. Schedule a site visit through FADAA Invite your local SAMH Supervisor to attend the exit interview. Inform staff involved of site visit process and tasks Designate a point of contact. Inform staff of site visit responsibilities Provide site visit location and hotel information to FADAA Evaluate twelve (12) client records for clients who have received treatment with EBP by completing the Client Review Checklist. Based on the results of the Fidelity Tool and the Client Review Checklist, identify areas at your agency that are considered strengths as well as areas that need improvement or would benefit from technical assistance. Provide a place for the peer mentor team to work. Prepare brief presentation for site visit Provide 12 client records which have been reviewed. Have materials and source documents within easy access for the peer mentor team. Schedule follow-up site visit with peer mentor team at the exit interview meeting Complete site visit evaluation and forward to FADAA 4
AGENCY SELF-ASSESSMENT SUMMARY Provider Agency Name EPB Program Date Based upon the combined results of your self-assessment using the EBP Assessment Survey and the Client Record Review, please respond to the following: Identify the strengths (up to 5) of your EBP. 1. 2. 3. 4. 5. What are your opportunities for improvement? 1. 2. 3. 4. 5. Do you have any needs for technical assistance? If yes, please identify. 1. 2. 3. 5
Client Review Checklist Provider Name: Provider ID Number: EBP Name: Rater s Name: Identifier: Review Date: Time Begin: Time Ended: Admission Date: Discharge Date: Completed EBP: Active EBP: INSTRUCTIONS: Please complete a separate checklist for every client reviewed. Use the written information from the clinical record reviewed and any assessment reports to complete this checklist. Remember that services that are not documented in writing are presumed not to exist. Comments should be made for each item that is answered with a No or N/A. If necessary, comments can be extended to the back of these sheets. Please print your comments and remember that they are an aid for mentoring the program. You will need to go back to your comments to assist the program in the future. Please note that this checklist applies only to clients who are in the evidence- based practice (EBP) program being reviewed. 6
1 Admission Criteria/Intake Process 1.1 Is the information in the clinical record consistent with the organization s written admission criteria? 1.2 Does the clinical record indicate the recommended level of care is based on the American Society of Addiction Medicine (ASAM) criteria? 7
2 Assessments (The table below contains a list of items that should be included in the psychosocial assessment for admission into the EBP. Indicate the presence of each item below. Provide comments for all No and Item # 2.1 Emotional or mental health 2.2 Level of substance abuse impairment 2.3 Family history 2.4 Client s substance abuse history 2.5 Education and employment history 2.6 Social history and functioning 2.7 Past or current sexual trauma 2.8 Other trauma (not sexual) 2.8 Client s leisure and recreational activities 2.9 Cultural influences 2.10 Spiritual or values orientation 2.11 Legal history 2.12 Client s perceptions of strengths and abilities 2.13 Clinical summary (Use the box provided below to record comments. Provide comments for any No or 8
2.14 Does the clinical record include the client s medical history? 2.15 Does the clinical record include the client s physical examination? 9
2.16 Does the clinical record document an assessment of the client s HIV risk? 2.17 Does the clinical record include a diagnosis? 10
3 Treatment Planning 3.1 Is the treatment plan based on the assessment findings? 3.2 Is the level of severity of the client s presenting problem identified in the client s record? 11
3.3 Was the client s involvement in treatment planning documented in the clinical record? 3.4 Does the treatment plan in the clinical record specify individualized client goals? 12
3.5 Does the treatment plan in the clinical record specify measurable objectives? 3.6 Does the treatment plan in the clinical record specify time frames for completion of client objectives? 13
3.7 Does the treatment plan indicate the type of services to be provided? 3.8 Does the treatment plan indicate the frequency of services to be provided? 14
3.9 Is the need for referrals documented in the clinical record? 3.10 Are recommended referrals documented on the client s treatment plan? 15
3.11 Does the clinical record indicate any follow-up to referrals made to the client? 16
4 Documentation of Implementation of EBP Treatment Services 4.1 Does the clinical record indicate the client s participation in the EBP therapeutic services? 17
4.2 Does the clinical record document dates and times for therapeutic services? 4.3 Are the client s progress notes present in the clinical record? 18
4.4 Does the clinical record show substance use testing and findings? 4.5 Is the number of days of client abstinence documented? 19
4.6 Does the record show the client s participation in individual counseling? 4.7 Does the record show the client s participation in group counseling? 20
4.8 Does the record describe the client s response to treatment? 5 Discharge and Continuing Care Planning 21
5 Discharge and Continuing Care Planning 5.1 Does the clinical record indicate that a discharge planning session was conducted? 22
5.2 Is there an aftercare plan in the client s record? 5.3 Does the aftercare plan outline the goals to be accomplished during aftercare? 23
5.4 Does the aftercare plan outline the process for monitoring client progress during aftercare? 5.5 Is there a relapse prevention plan in the client s record? 24
5.6 Does the clinical record document discussion with the client regarding social support resources? 5.7 Does the clinical record document discussion with the client regarding post discharge follow-up? 25
6 Indications of Treatment Outcomes 6.1 Does the clinical record describe the method used to assess the client s substance use after completion of treatment? 6.2 Are the results of substance use screening documented as an outcome of treatment? 6.3 Does the clinical record document completion of treatment goals? 26
Peer Mentor Review of Fidelity Self-Assessment Tool Reviewer: Date of Review: Organization Reviewed (Item #1): Provider ID (Item #2): Subcontractor Name (Item #3): Subcontractor Provider ID (Item #4): Name of EBP (Item #15): Item Number Check for N/A Verified (Y/N) Source/s of Information Comments 14 16 17 18 19 27
Item Number Check for N/A Verified (Y/N) Source/s of Information Comments 20 21 22 23 24 25 26 27 28 29 28
Item Number Check for N/A Verified (Y/N) Source/s of Information Comments 30 31 32 33 34 35 36 37 38 39 29
Item Number Check for N/A Verified (Y/N) Source/s of Information Comments 40 41 42 43 44 45 46 47 48 49 30
Item Number Check for N/A Verified (Y/N) Source/s of Information Comments 50 51 52 53 54 55 56 57 58 59 31
Item Number Check for N/A Verified (Y/N) Source/s of Information Comments 60 61 62 63 64 65 66 67 32
Identified Strengths: Identified Challenges: 33
Evidence-Based Practice Peer Mentoring Visit Exit Interview Form EBP Program Name Provider Name Visit Date Print Name 1. 2. 3. 4. Print Name 1. 2. 3. 4. 5. 6. Identified Strengths of Program: Peer Mentor Team Title EBP Program Staff Title Visit Review Questions 34
Challenges for Improvement: Suggestions: Additional Comments 35
Agency Review of Peer Monitoring Visit EPB PROGRAM NAME PROVIDER AGENCY NAME DATE OF VISIT Visit Review Items Please indicate your agreement or disagreement with each of the following statements about the peer mentoring program or the peer mentor visit by checking the selected column to the right of each item. Strongly Disagree Disagree Neutral Agree Strongly Agree 1. The peer mentor program is more cooperative than evaluative. 2. The review team was courteous and friendly. 3. The review identified the evidence-based practice program s most important strengths. 4. The review identified the evidence-based practice program s most important weaknesses. 5. There was sufficient time to conduct the onsite review. 6. The peer mentors were knowledgeable about evidence practice programs. 7. Instructions for the review forms and visit were sufficient and clear. 8. The review process focused on the most important issues. 36
9. The completed exit interview form provided a clear summary of the findings of the review. 10. Overall, we are pleased with the peer mentoring program. Positive Aspect of Peer Mentoring Program and/or Visit Issues and Challenges for Improvement Suggestions for Improvement 37
Additional Comments 38
FL Department of Children & Families EBP Fidelity Initiative Tool Review Site Visit Peer Mentor Feedback Agency: Date: Please rate the questions using the following scale: 5= Excellent 4= Above average 3= Average 2=Fair 1= Poor Rating 1. Did you have adequate information to conduct the site visit? 2. Were you adequately prepared to conduct the site visit? 3. Were you provided information in a timely manner? 4. How would you rate the site visit process overall? Please provide your comments to the following questions: 5. How comfortable were you with your level of participation and providing feedback to the provider? Do you feel your input was valued? 6. What were the positive aspects of the site visit? 7. What changes/additions would you suggest to the process? 8. Do you have any additional comments? 39
40