FASD 10- Year Strategic Plan Year 7 FASD Service Network Evaluation. Edmonton and Area Fetal Alcohol Network. July 22, 2014

Transcription

1 FASD 10- Year Strategic Plan Year 7 FASD Service Network Evaluation Edmonton and Area Fetal Alcohol Network July 22, 2014 Hélène Wirzba Wirzba Consulting Inc. Christopher Cameron Objective Research and Evaluation Inc.

2 Acknowledgements The Year- 7 FASD Service Network Evaluation would not have been possible without the guidance, support and help of many individuals and organizations. We extend our thanks to Alberta Human Services, Disability Services Division, FASD Unit, and to the Prairie Central FASD Network for their managerial and financial support. We have appreciated the Evaluation Steering Committee s guidance over the project period. We express our gratitude to each of the twelve Alberta FASD Networks, Network Coordinators, Leadership Team members, and FASD service providers all over Alberta, for taking the time to share their knowledge and insight, to participate in the focus group discussions, and to collect the required evaluation data. The thoughts, quotes and stories shared by children, youth, men and women whose lives were changed because of the Alberta FASD Service Networks are extremely valuable, as they illustrate and corroborate other findings. It is our hope that the evaluation findings and recommendations in this report will contribute to improving FASD services across Alberta. Hélène Wirzba & Christopher Cameron May 2014

3 Table of Contents Acknowledgements Executive Summary... i 1.1. Service Network... i 1.2. Prevention... ii 1.3. Assessment and Diagnosis... iii 1.4. Support Services... iv 1.5. Awareness... v 1.6. Conclusion... vi 2. Introduction Alberta FASD 10- Year Strategic Plan and FASD Service Networks FASD- CMC Strategic and Operational Plan 2013/ Development of Alberta FASD Service Network Evaluation Frameworks Year 7 FASD Network Evaluation Team and Steering Committee... 2 The Evaluation Team... 2 Role of Evaluation Steering Committee Evaluation Methodology and Tools... 2 Focus Groups and Narrative Reports... 2 Online Reporting System (ORS)... 2 Surveys and Templates Limitations Service Network Outcomes Logic Model FASD Service Network Outputs Review of Year 5 Compliance Guideline Recommendations Outcome #1: Collaboration... 9 Narrative Report and Focus Group Findings... 9 Survey Findings Outcome #2: Stakeholder Engagement Narrative Report and Focus Group Findings Survey Findings Outcome #3: Training Narrative Report and Focus Group Findings Stakeholder Survey Findings Outcome #4: Practice Informed by Research and Evaluation Narrative Report and Focus Group Findings Stakeholder Survey Findings Discussion related to Service Network Outcomes FASD Indicated and Targeted Prevention Logic Model Outputs Outcome #1: Reduction in Alcohol and Substance Abuse ORS Findings... 19

4 PCAP Data Outcome Reporting Tool Overview Outcome #2: Increase in Birth Control and Family Planning Methods ORS Findings PCAP Data Outcome Reporting Tool Outcome #3: Access to Information and Supports Advocate- Client Relationship Inventory Overview Outcome #4: Improvement in Well Being Advocate- Client Relationship Inventory ORS Findings Outcome #5: Decrease in Placement Disruption Outcome #6: Client Satisfaction, Involvement and Retention Other Prevention Outcomes Discussion Related to FASD Indicated and Targeted Prevention FASD Assessment and Diagnosis Logic Model Outputs Outcome #1: Understanding of FASD Outcome #2: Knowledge of Supports and Services in the Community Outcome #3: Referral to Recommended Supports Outcome #4: Client Satisfaction, Involvement and Retention Other Outcomes: Discussion Related to Assessment and Diagnosis FASD Support Services Logic Model Outputs Focus groups and Narrative Report Findings ORS Findings Outcome #1: Individuals with an FASD Services Based on their Needs Outcome #2: Individuals with an FASD have Knowledge and Access to Community Resources Survey Findings Discussion / Summary Statement Outcome #3: Individuals with an FASD Experience Improvement in Well- Being Survey Findings ORS Findings Support Services to Children, 0-12 years old ORS Findings Support Services for Youth, Ages 13 to ORS Findings - Support Services for Young Adults, Ages 18 to ORS Findings - Support Services to adults, 25+ years old Outcome #4: Caregivers of Individuals with an FASD have Knowledge of and Access to Community Resources Outcome #5: Caregivers of Individuals with an FASD have access to respite care, peer and professional supports Outcome #6: Caregivers of Individuals with an FASD Improved well- Being Survey Findings ORS Findings Support for caregivers Outcome #7: Client Satisfaction, Involvement and Retention Other Outcomes Discussion Related to Support Services... 54

5 7. FASD Universal Awareness Logic Model Outputs Outcome # 1: Increased Knowledge about the Effects of Alcohol Use Outcome # 2: Increased Understanding of FASD Outcome # 3: Changes in Attitude about / Stigma Attached to FASD Outcome # 4: Knowledge about Caring for / Supporting someone with FASD Outcome # 5: Client Satisfaction, Involvement and Retention Outcome #8: Staff Training, Supervision and Retention Other Outcomes Discussion Related to Universal Awareness Appendices Focus Group Participants Documents reviewed... 64

6 1. Executive Summary The Alberta Year- 7 FASD Service Network Evaluation took place in and was based on the FASD Service Network Evaluation Frameworks developed in The deliverables included a Provincial Service Network Evaluation Report as well as Service Network- specific evaluation reports. This document summarizes findings and recommendations specific to the Edmonton FASD Service Network (EFAN) Service Network Since the Year- 5 evaluation, EFAN moved from an informal leadership structure to becoming a Society with a Governance Board. There have been few changes in the governance structure as a result of the incorporation, except for the additional responsibilities related to the incorporation. EFAN continued to have an excellent alignment with the non- service delivery aspects of the FASD Service Network Program Guidelines. They have addressed all the recommendations made during the year- 5 evaluation. They have successfully transitioned from their previous informal governance structure to becoming a registered Society. EFAN operates according to four values: collaboration, responsiveness, diversity and accountability. The Network has increased and broadened its membership to over 50 members in the past two years. EFAN funds less than half of all FASD programs offered in Edmonton, and only about 40 percent of EFAN s current members received funding through EFAN, demonstrating that the Network continued to be successful in promoting collaboration and stakeholder engagements. EFAN hosted monthly half- day meetings with its members, which were well attended. Responses to the Stakeholder Survey questions related to collaboration were positive. Respondents demonstrated a belief in their Leadership Team s ability to govern and lead their Network. Additionally, survey respondents believed that their Network was responsive to regional needs and commitment to offering FASD services across the lifespan. Collaboration with Aboriginal and other cultural communities appeared to be a commitment that the Network was making. Focus group participants gave multiple examples of how EFAN promoted stakeholder engagement. Stakeholder survey respondents confirmed the information, indicating that their Networks fostered collaboration, actively listened to member s opinions, and valued their input. They said that their Network was responsive to the needs of diverse populations and facilitated the development of new partnerships with others in their region with a stake in the FASD community. Finally, they strongly believed that their Network was proactive increasing awareness of FASD in their region. Information available from focus groups, narrative reports and the stakeholder survey demonstrated that EFAN successfully promoted FASD training in its target area, for persons affected by FASD, professionals and the general public. Survey respondents believed that their Network facilitated meaningful information sharing, that training and educational resources offered by their Network were based on research and other leading practices, and that their participation in training offered through their Network had increased their understanding of FASD and that their ability to provide proper care and support to persons affected by FASD had improved. Findings from the focus groups, the narrative report and the stakeholder survey confirmed that EFAN was aware of community issues relevant to FASD and that community needs assessments, research, and evaluation findings were used to guide their Networks work. Some of EFAN s members were actively engaged in research activities. July 23, 2014 i

7 EFAN s stakeholder responses had higher agreement rates for all questions related to collaboration, stakeholder engagement, training, and evaluation & research than the combined responses of all Networks stakeholders. Based on these findings, there is ample evidence that EFAN continues to have a strong governance structure, and is successfully meeting all Service Network outcomes. There is no specific recommendation for EFAN s Governance Model Prevention In , 58 women accessed Parent Child Assistance Prevention (PCAP) programming through EFAN funding. A total of 33 new client files were created in ORS for PCAP clients between June 2012 and October Even though PCAP is a 3- Year program, ORS data suggests that only 77% of the original client files remained active in the subsequent reporting period (4-6 months), 44% in the 6-9 months reporting period, and 36% in the months reporting period. It is unknown whether clients exited the program early, or whether there were problems with ORS data entry. The PCAP Outcome Reporting Tool developed for the Year- 7 evaluation had data for 36 PCAP clients, of which 17 were in their first year, 8 in their second and 11 in their 3 rd year of the program. Recommendation: EFAN should discuss the apparent low retention rates of PCAP clients with its service providers, and if confirmed, discuss how retention rates could be increased. According to ORS data, EFAN s pregnant PCAP clients appeared more likely to consume alcohol and/or drugs while pregnant than all Alberta PCAP clients but there was a decrease in alcohol consumption between the first and second reporting period. The percentage of women who decreased or eliminated alcohol use during the first quarter in the program was similar for EFAN as for consolidated results. There was no apparent decrease in the percentage of women using drugs between the first and second reporting period. The percentage of women who decreased or eliminated drug use during the first quarter in the program was similar for EFAN as for all FASD Networks. According to the PCAP Outcome Reporting Tool, EFAN s PCAP clients were slightly less likely to have abstained from drugs and alcohol for at least one month than all FASD Network PCAP clients. Based on ORS data, the percentage of non- pregnant women successfully alcohol- exposed pregnancies by having an effective birth control method and/or not drinking alcohol was 89% during their first 3 months in the program, 85% for women who had been in the program for 4 to 6 months, and 100 percent for women in the program for 7 to 12 months. This result is based on a very small number of women and may not be representative of all women ever enrolled with PCAP in EFAN s target area. Based on the PCAP Outcome Reporting tool, of the known 34 non- pregnant women, only 14 in total (41%) were using an effective birth control method. The percentage of non- pregnant women who were effectively using a birth control method was very low in the 1 st and 2 nd year of the program (17% and 33%), but increased to 80% in the 3 rd year. Of the 34 non- pregnant women not using an effective birth control method, 1 abstained from alcohol for at least month, which suggests that only 15 out of the 34 women were effectively preventing new FASD births when the PCAP Data Outcome Reporting Tool was used. This result is lower than the 71% found when looking at provincial consolidated data. July 23, 2014 ii

8 There is a discrepancy in the findings between the two evaluation tools on the percentage of PCAP clients, with regards to the percentage of women effectively preventing alcohol- exposed pregnancies. Recommendation #1: The low rates in use of birth control in the first two years of the program and the low rate of women effectively preventing FASD births should be investigated further. The most common presenting issues for women in the EFAN PCAP program were similar as for the provincial sample. Housing, addiction, mental health, adaptive life skills and finances were presenting issues for over 45% of the women when their files were first opened in ORS. As with the provincial consolidated data, for many issues, there appears to be an initial increase in the prevalence of issues followed by a decrease over time. 63% of EFAN PCAP clients with a target child had custody of such children, and 63% of the children lived with their mothers. The rates are similar to the provincial consolidated rates. Overall, EFAN s PCAP programs appear to meet the Service Network Prevention Outcomes. More information is needed to explain lower than provincial retention rates and effective prevention of FASD birth rates Assessment and Diagnosis EFAN funds FASD assessment and diagnosis services in partnership with Alberta Health Services at the Glenrose Children s FASD Clinic. The Network funding provides one additional assessment per month at the Glenrose clinic, or a total of 12 assessments per year. Only 8 post- clinic surveys were completed for EFAN as part of the Year- 7 Service Network Evaluation, 7 by caregivers, and 1 by a client. Because of the small sample size, the survey findings should be interpreted with caution. Recommendation #2: EFAN should consider offering post- clinic surveys on an on- going basis, in order to increase the validity of their clinic evaluation findings. Survey results suggest that both parents/caregivers and clients had an improved understanding of FASD after the clinic day; and that the level of understanding in relation to strengths and weaknesses and thinking and learning patterns had improved for both parents/caregivers and clients after the clinic day as well. All parents/caregivers and clients had a positive, favorable impression of the diagnostic process after their assessment clinic experience. The majority of parents/caregivers and clients indicated that they understood FASD better after their clinic experience. The majority of parents/caregivers and all clients indicated that they had received new information about services and supports, and that they planned to connect with the services and supports that were recommended to them by assessment clinic staff. The majority of parents/caregivers and clients were satisfied with their assessment clinic experience and indicated that they received what they wanted from the process. They believed that their diagnosis was easily understood and that they were confident in the assessment process findings. They unanimously stated that they were treated with dignity and respect by assessment clinic staff. Overall, findings suggest that EFAN s funded FASD assessment and diagnostic services positively contribute towards the provincial outcomes, at a similar level as the consolidated provincial findings. However, EFAN funds only a small number of assessments, and results need to be validated with a larger number of surveys. July 23, 2014 iii

9 1.4. Support Services In , EFAN provided funding for support services to 8 different agencies in Edmonton, reaching out to 319 unique clients. As part of the Year- 7 Service Network evaluation process, EFAN s support service providers returned 99 client surveys, 30 surveys filled out by caregivers on behalf of services received by their dependents, and 37 surveys filled out by caregivers who received support services. ORS data confirmed that EFAN provided support services to clients of all age groups and to caregivers, that almost all clients received services for at least 2 reporting quarters, and the great majority for longer periods. More data over a longer period of time will confirm the good retention rates. Clients and caregivers reported that agencies helped them in a number of ways: taking care of themselves and their health, dealing with relationship issues, and support with school & employment were the most common areas of help. Clients also reported that agencies helped them deal with addiction and legal issues. EFAN s responses were similar to provincial responses. The majority of parents/caregivers and clients who participated in support services indicated that they had a better understanding of their strengths and weaknesses and knew what they needed to change about their behavior to take better care of themselves as a result of the support services. Presenting issues and outcomes for individuals receiving support services showed similar patterns for EFAN- funded support clients as provincial results. The three most frequent presenting issues for children receiving EFAN- funded support services were: behavior (43%), education (50%), and social skills (43%). The four most frequent presenting issues for youth years old receiving EFAN- funded support services were: social skills (66%), adaptive/life skills (50%), education (30%), and behavior (26%). The four most common presenting issues for young adults, ages 18 to 24, were: housing (45%), adaptive skills / live skills (53%), social skills (48%), and behavior (39%). The most common presenting issues for adults, ages 25+ were: mental health (66%), finances (52%), housing (50%), and employment (43%). There were inconsistencies in ORS data entry for the caregiver groups, and data could therefore not be analyzed. For most issues, there appeared to be an initial increase in the percentage of clients experiencing the issue in subsequent periods, with the beginning of a decrease in the 4 th or 5 th quarter. This pattern was also found in the provincial consolidated sample. The majority of parents/caregivers receiving support from support services indicated that they had increased knowledge of how FASD affects their child and that they had received information that made them feel more capable of caring for their child. Results were similar to the provincial sample. The majority of parents/caregivers receiving support from support services indicated that they had received assistance accessing help in the community for their child, that their support network had increased, and that their access to respite care had increased since accessing support services. Results were similar to the provincial sample, except that EFAN s caregivers were more likely than all caregivers to have gained new access to respite care. Respite care is an area of service that is not systematically offered by all Networks. July 23, 2014 iv

10 The majority of parents/caregivers receiving support from support services indicated that their family life had improved. Half of the parents/caregivers indicated that their stress levels had decreased as a result of the support that they have received. The majority of parents/caregivers also reported an increased sense of hope for their child. Results were similar to consolidated provincial responses. All parents/caregivers and almost all clients who participated in support services were happy with the services they received and that they received the support that they wanted. All parents/caregivers and clients indicated that they were treated with dignity and respect. Over half of the caregivers had attended at least one FASD training event, a higher percentage than in the provincial sample. Overall, EFAN s Support Services appear to meet the Support Services outcomes. There is no specific recommendation for EFAN s funded support services Awareness EFAN made an intentional investment in using social media to connect with stakeholders and disseminate information. EFAN had a website, a blog, a Facebook and a Twitter account. EFAN s members have learned to use the social medial tools to access a wealth of information about FASD, events and resources. EFAN was responsible for 46% of all resources developed and reported during the Service Network Year- 7 Evaluation. They developed a variety of resources, including brochures, posters, information sheets, a calendar, key chains and other promotional items. The purpose of their resource development included preventative education (52%), targeted community initiatives (29%) and professional development (24%). Resources were targeted at the general public (100%), individuals affected by FASD ((57%) and professionals (71%). Often the resources had more than one purpose or targeted audience. None of the resources were evaluated (compared to 7% on a provincial level), and 95% were available to other Networks. EFAN was responsible for 7 events of all events (6%) reported during the Service Network Year- 7 Evaluation. Most of them (86%) were professional development events. Only 29% of the events were evaluated. EFAN was responsible for 7 events of all events (6%) reported during the Service Network Year- 7 Evaluation. Only 29% of the events were evaluated (compared to 56% on a provincial level), resulting in 40 post- event surveys. EFAN members have access to full- time educators through one of their members, through another source of funding. This may explain the small number of events reported, and that only 40 post- event surveys returned. Recommendation #3: EFAN should consider the feasibility of strengthening their evaluation of events and resources. Survey results were very positive, with generally higher satisfaction rates as consolidated provincial results: 97% of respondents indicated that they learned something new about FASD and that they had a better understanding of the disability after attending an event. All individuals who participated in an FASD event stated that they felt more compassionate and that they intended to be more supportive of individuals affected by FASD. 93% indicated that the information obtained would help them care for or support someone with FASD better. 98% indicated that they were satisfied with the event and that it was a good use of their time. In the Network stakeholder survey, 80% of respondents indicated that the training and education provided by the Network helped them increase their understanding of FASD have July 23, 2014 v

11 made them more compassionate and supportive of individuals affected by FASD (14% of respondents did not know). Evaluation results suggest that EFAN is a leader among all Networks in resource development, and that there is a high rate of satisfaction with the FASD training offered through the Network Conclusion The Year- 7 Evaluation took place after a recent change in governance structure, where EFAN became a Society, and changed its leadership structure. Evaluation results suggest that EFAN s transition has been successful, and has not affected the Network s ability to sustain community collaboration and foster stakeholder involvement. Client and system outcomes were most often met at higher levels than the provincial consolidated levels. The report includes several recommendations specific to EFAN. July 23, 2014 vi

12 2. Introduction 2.1. Alberta FASD 10- Year Strategic Plan and FASD Service Networks In 2003, the Government of Alberta launched a cross- ministry initiative to develop a comprehensive coordinated response to FASD, which resulted in the implementation of Alberta s FASD 10- Year Strategic Plan ( ). During the first five years of the Plan ( ), the foundation was laid for Alberta s model of FASD prevention and service delivery, the FASD Service Network Program. Eleven FASD Service Networks serve geographical regions throughout the Province and one Network serves Métis Settlements across Alberta. Under the direction of Alberta s FASD Cross- Ministry Committee (FASD- CMC), Services Networks developed and implemented an FASD prevention and service delivery model that is client focused and result based, enabling individuals with FASD, their families and caregivers, to access information and services through a single door, the FASD Service Network in their region. The year 5 Evaluation established baseline performance measures against outcomes and targets, providing a foundation for subsequent evaluations FASD- CMC Strategic and Operational Plan 2013/2014 The FASD Strategic and Operational Plan is an outcomes- based management plan designed to refine the goals under each of the strategic pillars. It presents a life- cycle approach that emphasizes results, learning and adaptation. For each of the strategic pillars: awareness, prevention, assessment & diagnosis, supports for individuals & caregivers, FASD learning organization, the plan identifies a related goal, systems and client outcomes, actions and leads. Each outcome in the plan is measurable through key performance indicators related to the provincial FASD goals and the Government of Alberta Results Based Budgeting (RBB) process Development of Alberta FASD Service Network Evaluation Frameworks The Year- 5 evaluation made seven overall recommendations, some of which were specific to FASD Service Networks. The third recommendation was to clarify outcomes, i.e. to articulate clear, measurable outcomes for individuals affected by FASD (client outcomes) and for the FASD service delivery model (system outcomes) that demonstrate system effectiveness and efficiency. The fourth recommendation was to develop a data collection model, which identifies indicators and develop data collection templates, and provides funding to Service Networks supporting continuous, reliable data collection. This led to the development of five evaluation frameworks, one for each pillar in Each of the frameworks include a logic model, client and system outcomes, indicators, measurement tools, output and outcome measurement plans. The frameworks were used to guide the Service Network evaluation of the Service Network Year- 7 evaluation 23 July

13 2.4. Year 7 FASD Network Evaluation Team and Steering Committee The Evaluation Team The FASD- CMC contracted Wirzba Consulting Inc. (Hélène Wirzba) in partnership with Objective Research and Evaluation Inc. (Christopher Cameron), as the evaluation team. Hélène Wirzba and Christopher Cameron were the consultants contracted for the development of the FASD- Service Network evaluation frameworks. Role of Evaluation Steering Committee The Year 7 FASD Service Network Evaluation Steering Committee included representatives from FASD Service Network and the FASD- CMC. The committee was responsible for the implementation of the Network Year 7 Evaluation and met on a monthly basis, between July 2013 and June In particular, they: Provided recommendations regarding policies, standards and any issue that arose during the evaluation implementation; Provided feedback on additional evaluation tools and resources that were developed (such as focus group guidelines, evaluation report templates ); and Linked with other organizations and committees to share resources, ideas and concerns, and keep abreast of new information related to the FASD- CMC 7- Year evaluation Evaluation Methodology and Tools The evaluation methodology is described in each of the five evaluation frameworks. The deliverables included a provincial report with consolidated information from all 12 Service Networks, and Network- specific reports. Data sources for the Year- 7 Service Network evaluation included the following: Focus Groups and Narrative Reports The data presented is based on focus groups with each of the 12 Service Network leadership teams, as well as a review of the Service Network narrative reports, and other supporting documents, such as by- laws, policies and procedures, strategic planning documents, websites... Ten out of 12 focus groups took place over teleconference. The remaining 2 focus groups were face- to- face meetings. Each group met for 60 to 90 minutes. Focus groups were well attended: 68 leadership representatives had the opportunity to contribute to the focus groups. Each focus group had at least 3 representatives. Online Reporting System (ORS) The FASD Online Reporting System (FASD- ORS) is an online reporting system that generates reports to describe clients being served by the Alberta FASD Service Network Program. It was first introduced in April 2012 and is managed by the FASD Unit, Disability Services Division, Alberta Human Services. The data elements include client demographic information, presenting issues and outcome statements to measure the difference being made in the lives of individuals accessing services in the categories of prevention, assessment/diagnosis, and support for those affected by FASD across the lifespan. FASD service workers post data in ORS for each of their clients on a quarterly basis (reporting quarters). 23 July

14 The data captured and reviewed as part of the Year 7 Service Network Evaluation comes from the 12 FASD Service Networks across the province and only includes information about clients funded directly through the FASD Service Network Program. This was the first time ORS data was analyzed with the purpose of generating outcome data. Several limitations were identified during the process. For example answers to many of the data entry fields are currently optional resulting in many blank field. All of the identified issues have been flagged, and will be addressed in the planned minor and major ORS enhancements. ORS data was mostly analyzed for the provincial report. Service- Network specific reports do not include a complete analysis of ORS data specific to their funded clients. It is expected that such analysis will be available in the future. Demographic information For each of the service pillars, the demographic information presented in the Year 7 Service Network report is based on the FASD Service Network Program data report (v1) produced by the Alberta FASD Unit in October 2013, and is limited to unique clients served between April 1, 2012 and March 31, 2013 (ORS Reporting Quarters Year 1 Quarter 1 to Year 1 Quarter 4). Assessment and diagnostic services The information about types of FASD diagnosis, secondary diagnosis, presenting issues, and recommendations, is based on all unique clients for whom the diagnosis process was completed between April 2012 and September 2013 (ORS Reporting Quarter Year 1 Quarter 1 to Year 2 Quarter 2). Support and prevention services For the purpose of the outcome analysis for support and prevention services, we created cohorts of newly admitted clients in each of the ORS reporting quarters, excluding the first one (April to June 2012) and examined the changes of presenting issues and related outcome statements in subsequent reporting periods for clients for whom there was data in subsequent reporting periods (2nd, 3rd, 4th 5th and 6th quarter of services). Sample sizes were created, as per the following table. The sample size for clients in their first quarter (0-3 months of services) is the largest. Subsequent quarters have lower sample sizes because some files were either inactive or closed, and because the ORS system is still very new. (For example, all clients newly admitted in Jul- Sep 2013 had only one reporting period, and those admitted in April to June 2013 had only 2 reporting periods.) 23 July

15 Length in program Clients with 0-3 months services Clients with 4-6 months services Clients with 7-9 months services Clients with months services Clients with months services New to ORS Yr1 Qt2 New to ORS in Jun- Sep 2012 Still in ORS in Oct- Dec 2012 Still in ORS in Jan- Mar 2013 Still in ORS in Apr- Jun 2013 Still in ORS in Jul- Sep 2013 New to ORS Yr1Qt3 New to ORS in Oct- Dec 2012 Still in ORS in Jan- Mar 2013 Still in ORS in Apr- Jun 2013 Still in ORS in Jul- Sep 2013 New to ORS Yr1Qt4 New to ORS in Jan- Mar 2013 Still in ORS in Apr- Jun 2013 Still in ORS in Jul- Sep 2013 New to ORS Yr2Q1 New to ORS in Apr- Jun 2013 Still in ORS in Jul- Sep 2013 New to ORS Yr2Qt2 New to ORS in Jul- Sep 2013 We analyzed the changes in preventing issues and associated outcomes for clients receiving prevention and/or support services. There are limitations in the analysis of presenting issues and outcome statements because: We had to exclude all clients admitted to ORS in the period of April- June 2012 in the outcome data analysis, because ORS did not have any field to track the date clients first received services, and we were not able to determine the length of FASD services received for these clients files. We made the assumption that all new client files, starting in the second ORS reporting period (July to September 2012) were also new clients for FASD service providers. ORS did not track presenting issues and related outcome statements at admission; the information entered for new clients includes services provided in that quarter. ORS did not accurately track reasons for file closure. It was impossible to determine the percentage of clients for whom outcomes were met. It is possible that clients who stayed in the program for several subsequent quarters have higher needs than the ones who leave the program early. It is also possible that clients who left were transient, were jailed, or even died. For support services to clients with FASD, the sample size was large enough to analyze and present results by age groups, and by types of clients in the provincial report. For the prevention pillar, we tracked information about pregnancy status, pregnancy outcomes, family planning, alcohol and drug use for all clients newly admitted between April 2012 and September 2013 (ORS Reporting Quarter Year 1 Quarter 2 to Year 2 Quarter 2). ORS did not track any information about the wellbeing and custody status of clients children. Also for non- pregnant women, information about alcohol and drug consumption was recorded only if the women did not use an appropriate family planning method. Surveys and Templates Each Service Network was responsible to coordinate the administration of the surveys, and the completion of event and resource development templates in partnership with their funded service providers, in their geographical area, according to the established protocols. Data collection took place between July 1 and December 31, The consultants compiled and analyzed the data using the Fluid Survey TM online survey tool. 23 July

16 2.6. Limitations Only a portion Alberta s FASD support services are delivered through funding from Service Networks. At the time of this evaluation, all Networks were aware of other organizations / agencies involved in FASD support: Child and Family Services had FASD supports for children with FASD and their foster parents; School Boards provided FASD supports to students and their families in the schools through the WRAP and TRY program. Several Networks mentioned community funding for FASD supports through Alberta Seniors and Community Supports (now Alberta Senior Services). Data presented in this section was mostly retrieved from narrative reports, as the scope of the evaluation did not include any interaction with front- line service providers. Some of the key performance indicators were not measurable through this evaluation, because neither ORS nor narrative reports kept track of information required. The focus group discussion guidelines did not have any specific question related to the delivery of front- line FASD services. The qualitative information is based on information included in narrative reports, observations made during the focus group discussions and survey comments. 23 July

17 3. Service Network Outcomes 3.1. Logic Model FASD Service Network 23 July

18 3.2. Outputs Mission Statement: EFAN works to enhance the capacity of our community to prevent Fetal Alcohol Spectrum Disorder and support those impacted by FASD through coordinated planning, collaboration, education, service delivery and advocacy. The Edmonton and area Fetal Alcohol Network (EFAN) is a collaborative venture made up of individuals from community agencies, governmental departments and concerned citizens, which recognize the significant opportunity to achieve valuable social impacts through a grassroots partnership. EFAN s service areas cover Edmonton, Fort Saskatchewan, St. Albert, Spruce Grove, Leduc, and Sherwood Park. The Network offers a wide variety of programs and services that cater to those impacted by FASD. We work alongside Catholic Social Services, Bosco Homes, Bissel Centre, Glenrose Hospital, Elves Special Needs Society, Métis Nation of Alberta, CASA, Leduc County FCSS, and Elizabeth Fry Society to enhance services available to people affected by FASD. The Edmonton and Area Fetal Alcohol Network (EFAN) existed in some form since 1997, long before the inception of the FASD Service Network Program. It was founded when community members with an interest in the issues surrounding FASD came together to support one another, and was known as the Region 6 Steering Committee. The first staff member was recruited in 2001, through a contract with Catholic Social Services. The current Network Coordinator jointed the Steering Committee as a staff member in EFAN was formally created in EFAN is one of the original seven FASD Networks funded through the Alberta FASD Service Network Program. It was a shift for EFAN to transition from being a steering committee to becoming fund- granting organization. The governance structure changed to include a Leadership Team responsible for administrating the funds according to provincial Network guidelines and a Network Coordinator was hired to support the fund granting. The broader membership was still responsible for decision- making. Since the year- 5 evaluation the Network membership and Support and Service Team had discussions on whether becoming a Society was a direction that individuals and agencies wanted to take. Granting bodies suggested that they might feel uncomfortable providing funds to groups that did not have legal entity status. In order to safeguard grant funding received from the FASD service network program and other government sources, EFAN incorporated as a Society in August There have been few changes in the governance structure as a result of the incorporation, except for the following: the Network Coordinator had to take on additional responsibilities related to the Corporate Registry requirements; and the Network Coordinator, together with the Executive Board, had to handles the business of the Network, such as managing contractual agreements Review of Year 5 Compliance Guideline Recommendations The FASD Cross- Ministry Service Network Program Guidelines define standards and instructions on Service Networks governance structure, planning, funding decisions, financial management and performance reporting. In March 2011, the FASD- CMC undertook a review on how Networks were doing with respect to implementing the non- 23 July

19 service delivery aspects of the program guidelines. Each Network received a report outlining their compliance status and listing specific recommendations. The Year- 7 evaluation included a review of the recommendations and the current compliance status, based on information provided in the narrative reports, the focus group discussions, and additional documentation. The findings of the guideline review process indicated that: EFAN s performance with respect to the implementation of the non- service delivery aspect of the program guidelines was excellent. EFAN s year- 5 evaluation report stated that EFAN has been successful in integrating the requirements of the FASD Service Network Program (SNP) into their existing operations and leveraging the expertise and infrastructure of the local FASD community to expand and enhance programs and services for individuals and caregivers affected or suspected of FASD and improving outcomes for the individuals served by FASD- SNP funding. EFAN has addressed all recommendations made in the guideline review process and the evaluation report, as indicated in the following table. Recommendations Action Taken Source of Information From Guideline Review 1. Articulate and document succession planning policies and procedures 2. Ensure that authority to approve the business plan is clearly documented as a responsibility of the Network Coordinator From Evaluation Report 3. Continue to build and strengthen collaborations in Alberta s FASD research community, acting as a natural interface between front- line service providers and clients, and researchers and develop mechanisms to use findings to influence provincial planning for FASD. 4. Develop succession plans for the Network Leadership team members and Coordinator in order to EFAN s revised 2013 policy and procedure manual includes new policies on succession plan policy, emergency succession plan policy both policies deal with changes in network leadership team, especially co- chairs. EFAN s revised policy 2013 clarifies the fact that the Coordinator is responsible for developing all reports, including but not limited to quarterly reports, annual reports, business plans, and that reports will be approved by the Network on an as required basis. The policy manual includes accountability charts. EFAN continues to build and strengthen collaboration in Alberta s FASD research community. The Network is active on the provincial evaluation- working group. Member agencies are involved in research projects. Publications are disseminated and impact service providers Same as 1 and 2 Review of policy manual on EFAN s website EFAN annual report Review of policy manual on EFAN s website EFAN annual report EFAN annual report EFAN annual report July

20 Recommendations Action Taken Source of Information mitigate risks when faced with attrition or turnover. 5. Continue to explore membership gaps and avenues for coordination of services among Network members and other organizations in the community even where obvious connections to FASD prevention, management and services do not exist. 6. Continue to measure client outcomes as a mechanism to communicate the positive impact of the Network EFAN sent open invitations to community members and agencies that have an interest in FASD and may be willing to participate, as well as to government departments that are represented at CMC. Prospective membership is a monthly standing agenda for working groups and during the annual strategic planning session. They have hosted special events, open meetings and advertised to raise EFAN s profile in the community. EFAN currently has 53 members, of which 11 newly joined in EFAN is involved in Network specific and provincial evaluation initiatives. Program specific appraisals are developed and utilized on a consistent basis. EFAN annual report Focus group report. EFAN currently has an excellent alignment with the non- service delivery aspects of the FASD Service Network Program Guidelines. They have addressed all the recommendations made during the year- 5 evaluation. They have successfully transitioned from their previous informal governance structure to becoming a registered Society Outcome #1: Collaboration The planning and delivery of regional programs and services associated with FASD is accomplished through a collaborative approach. Narrative Report and Focus Group Findings Collaboration: EFAN provides a forum for agencies and members to work together in an intersection of common objectives as defined by the membership. The collaborative relationships between Network members allow of the accomplishment and revision of the objectives and mission stated in the Terms of Reference. EFAN s Policies and Procedures 2013 EFAN operates according to four values: collaboration, responsiveness, diversity and accountability. One of the 8 goals listed in the policy and procedure manual is to: promote community connections and collaborations between organizations and community members, municipally, provincially, nationally and internationally. EFAN has increased and broadened its membership to over 50 members in the past two years. After EFAN s incorporation as a Society, some government employees were not sure any more whether they could be Network members and fully participate in meetings and committees. Some chose to attend as ex- officio members. EFAN values the participation of government representatives, and would like to continue partnering with a broad cross- ministry representation. 23 July

21 EFAN funds less than half of all FASD programs offered in Edmonton, and only about 40 % of EFAN s members receive funding through EFAN. Several of the EFAN members sit at provincial FASD Councils. EFAN has representation at all Alberta FASD Councils, except for PCAP, but one staff of an EFAN- funded agency sits on PCAP council. Sitting on the Councils allows EFAN members to quickly get access to information and to disseminate the information to all EFAN members. EFAN is the hub of all FASD information for Edmonton. EFAN has the capacity to disseminate information quickly and widely to its members. Survey Findings Stakeholder Survey Overview A total of 52 individuals completed the Stakeholder Survey. On average, each Network submitted 29 surveys. EFAN had a relative high number of stakeholders filling out the stakeholder survey. These individuals were asked what best describes their association with their Network. Some individuals identified more than one association, which led to a total of 73 responses. 23 (32%) of these responses were for the Network member option; 14 (19%) of these responses were for the Network leadership team option; 3 (4%) were for the parent/caregiver/friend of a child or adult living with FASD option; and 33 (45%) were for the professional involved in providing FASD services to the general public option. Question 3. The Leadership Team has the necessary skills to govern the Network. 4. The Leadership Team is doing a good job governing the Network. 5. The Network's ability to manage contracts is satisfactory. 6. The Network has appropriate policies and procedures in place to manage contracts. 7. The Network s membership represents regional FASD stakeholders. 8. The Network is aware of current FASD needs in the region. 9. The Network is responsive to the FASD needs in the region. 10. The Network believes that FASD services should be available across the lifespan. 11. The Network supports collaborative initiatives with Aboriginal communities. 12. The Network supports collaborative initiatives with other cultural communities. Strongly Agree / Agree Neutral Strongly Disagree / Disagree Don t Know / N/A Total Responses EFAN 46 (90%) 2 (4%) 2 (4%) 1 (2%) 51 All Networks 79% 6% 5% 10% 283 EFAN 44 (86%) 5 (10%) 1 (2%) 1 (2%) 51 All Networks 75% 9% 6% 10% 280 EFAN 38 (75%) 4 (8%) - 9 (17%) 51 All Networks 68% 5% 4% 24% 281 EFAN 37 (73%) 2 (4%) 1 (2%) 11 (21%) 51 All Networks 61% 5% 3% 31% 280 EFAN 41 (80%) 4 (8%) 1 (2%) 5 (10%) 51 All Networks 74% 7% 5% 14% 279 EFAN 45 (88%) 3 (6%) 1 (2%) 2 (4%) 51 All Networks 86% 4% 7% 4% 282 EFAN 44 (86%) 2 (4%) 2 (4%) 3 (6%) 51 All Networks 80% 5% 8% 7% 279 EFAN 47 (92%) 2 (4%) 1 (2%) 1 (2%) 51 All Networks 90% 2% 3% 6% 278 EFAN 40 (78%) 3 (6%) 1 (2%) 7 (14%) 51 All Networks 73% 6% 3% 18% 281 EFAN 28 (55%) 10 (20%) 2 (4%) 11 (22%) 51 All Networks 54% 11% 6% 30% July

22 Responses to the Stakeholder Survey questions related to collaboration were positive, with higher agreement rates than the provincial consolidated statements. Respondents demonstrated a belief in their Leadership Team s ability to govern and lead their Network. Additionally, survey respondents were favorable in their support of statements related to their Network s responsiveness to regional needs and commitment to offering FASD services across the lifespan. Collaboration with Aboriginal and other cultural communities appeared to be a commitment that the Network is making Outcome #2: Stakeholder Engagement Mechanisms are in place to facilitate and encourage stakeholder engagement in strategic planning, and to provide stakeholder opportunities for networking and information sharing. Narrative Report and Focus Group Findings EFAN facilitates monthly half- day meetings for its members. The first hour of the meeting is for funded members only, and deals with issues related to Network funding. The second half of the morning is open to all members. It starts with a plenary where information relevant to all members and includes agency updates. Following that, members divide in 3 different working sub- committees: child & youth, adults with FASD, prevention. Because of the large number of participants, EFAN was sometimes short on space and had to look for alternative meeting space. According to focus group participants, the EFAN Network meetings offered great opportunities to collaborate, to meet people face to face, and to learn what each organization is doing. The information shared at the face- to- face meeting enabled front- line workers to better advocate on behalf of their clients. The meetings allowed for easy dissemination of information, and get the word out in the community about events, workshops, and new funding source, and for smoother referrals between agencies. EFAN benefits from having a large membership when it does its yearly strategic planning. Because of EFAN, there have been new partnerships and collaborations, such as a recent collaboration between the Glenrose FASD clinic and CASA, a community- focused provider of mental health services for infants, children, adolescents and their families, located in Edmonton and serving central and northern Alberta. CASA is providing a support group for families going through the waitlist at Glenrose. They are providing some re- integration / education for families. Focus group participants stated that members work better because of EFAN. Working with each other allowed them to refer clients from agencies with long waiting lists to others who had current openings. EFAN has enabled program connection. Members were more aware of each other s programs and resources and were using resources more effectively. Members learned about new resources that could benefit their clients, and were therefore able to serve their clients in a better way. The EFAN monthly meetings were an opportunity for all agencies to broaden their scope about FASD, to look at the big picture. For example agencies working with children with FASD gained understanding of how FASD affects adults. The working sub- committees offered an opportunity to jointly engage in concrete work. Recently, one of the working groups jointly designed a 12- month calendar with monthly 23 July

23 bytes of information about FASD. It was one way for EFAN to produce a high- impact low- cost resource. The calendar was launched for the 2013 FASD day. The Bissel Centre obtained funding for a FASD community educator. Through the Network the community educator was able to schedule presentations for EFAN members, agency staff members, and agency clients. EFAN was instrumental in facilitating the developing of resources for adult employment. Members needed to collaborate, be more creative, and innovate in order to impact employment for adult clients with FASD. This included professional development, supports for caregivers, mentoring, sharing of information. EFAN helped members to link all the available resources and information. EFAN s monthly meetings were also an opportunity to access new resources, such as the Barrier Reduction Fund. The fund is available to individuals, ages 6 to 24 with an FASD clinical diagnosis and another diagnosis. Members are invited to submit one- time applications on behalf of their clients. The funding is used to help clients reduce at least one barrier they face; there are accountability measures after 2 and 4 months. Survey Findings Stakeholder Survey Question 14. Network meetings are well attended. 15. The Leadership Team is listening to member's opinions. 16. The Network values stakeholder input. 17. Network leaders are respectful of members' culture and beliefs. 18. Regional FASD needs are discussed at strategic planning meetings. 19. The Network is responsive to the needs of a diverse population. 20. The Network facilitates collaboration between members. 21. In the past year, the Network has assisted in building new partnerships in the region. 22. The Network is proactive in increasing awareness of FASD in the region. 23. Network leaders participate in relevant other community networks. To a Great Extent / To Some Extent Very Little / Not at All Don t Know / N/A Total Responses EFAN 42 (82%) - 9 (18%) 51 All Networks 59% 5% 37% 272 EFAN 44 (86%) - 7 (14%) 51 All Networks 64% 4% 32% 272 EFAN 41 (80%) 2 (4%) 8 (16%) 51 All Networks 71% 4% 25% 271 EFAN 44 (86%) 1 (2%) 6 (12%) 51 All Networks 78% 1% 21% 272 EFAN 39 (76%) 1 (2%) 11 (22%) 51 All Networks 60% 2% 38% 272 EFAN 45 (88%) 2 (4%) 4 (8%) 51 All Networks 80% 2% 18% 270 EFAN 44 (86%) 1 (2%) 6 (12%) 51 All Networks 75% 5% 20% 270 EFAN 32 (63%) 3 (6%) 16 (31%) 51 All Networks 65% 6% 29% 270 EFAN 50 (98%) - 1 (2%) 51 All Networks 90% 4% 6% 272 EFAN 43 (84%) 1 (2%) 7 (14%) 51 All Networks 69% 3% 27% 266 Focus group participants gave multiple examples of how EFAN promoted stakeholder engagement. Stakeholder survey respondents indicated that their Networks fostered collaboration, actively listened to member s opinions, and valued their input. Further, respondents said that their Network was responsive to the needs of diverse populations and facilitated the development of new partnerships with others in their region with a stake in the FASD community. Finally, respondents strongly believed that their Network was 23 July

24 proactive increasing awareness of FASD in their region. EFAN s stakeholder responses had higher agreement rates for all questions related to stakeholder engagement than the combined responses of all Networks stakeholders Outcome #3: Training Network stakeholders have knowledge of and access to training and educational resources that are based on research and leading practice. Narrative Report and Focus Group Findings EFAN s first goal, according to its policy and procedure manual was to encourage the development of a wide range of community information and resources related to FASD. According to focus group participants: the most important thing that EFAN does is disseminating information to everybody. In addition to circulating resources developed by others, EFAN created its own resources, such as monthly FASD tip sheets that were available on their website. EFAN encouraged members to attend FASD conferences, in Alberta and beyond. EFAN enabled members to increase their professional expertise because of the sharing and collaboration that occurred during meetings. It enabled members to make the best use of available resources in a coordinated and cost- effective way. The Network Coordinator was available to offer training on FASD for staff, parents and caregivers, focusing on strategies to work with persons with FASD. Training sessions were opened to the community: extended families, neighbors, and teachers. Clients and caregivers had access to EFAN resources. Training sessions, online FASD resources, and FASD communities of practice helped decrease the sense of isolation families face. Member agencies could see their client families grow, thanks to training resources. The role of EFAN in directly providing training has decreased over the past years, because of an increase in its administrative duties. The Bissel Centre, one of EFAN s members, trough funding from Child and Family Services, Alberta, was able to hire 1.5 FTE FASD Educators who currently provide education and trainings to the community EFAN would like to provide more training opportunities, for all new staff, for parents and caregivers, and volunteers. Agencies had limited budgets for professional development, and often conferences have prohibitive costs. For example, very few agencies had the funds to register for the recent FASD conferences in Edmonton. It was only because EFAN received a complementary registration through the Solicitor General that one person could attend the FASD Legal Conference. For the First International FASD Prevention Conference, EFAN could send a person because she was a presenter and her registration fee was waived. In 2012, EFAN sponsored a few students of the University of Alberta to participate in an FASD workshop with Dr. Gabor Maté. 23 July

25 Stakeholder Survey Findings Question 25. The Network facilitates sharing of FASD information with members. 26. The Network facilitates sharing of FASD information with partners. 27. The Network facilitates sharing of FASD information with the general public. 28. Training and educational resources offered through the Network are based on research and leading practice. 29. Network leaders are knowledgeable in their area of work. 30. Network leaders are knowledgeable about FASD best practices. 31. As a result of training and educational resources offered through the Network, I have an increased understanding of FASD. 32. Training and educational resources offered through the Network have helped me be more compassionate and understanding towards individuals and families living with FASD. 33. Training and educational resources offered through the Network have helped me improve the way I care for and support persons affected by FASD. Definitel y / Probably Not Sure Definitely Not / Probably Not Don t Know / N/A Total Responses EFAN 46 (90%) 1 (2%) 2 (4%) 2 (4%) 51 All Networks 88% 3% 2% 7% 265 EFAN 46 (90%) 1 (2%) 1 (2%) 3 (6%) 51 All Networks 88% 2% 2% 7% 264 EFAN 46 (94%) 2 (4%) 1 (2%) - 49 All Networks 87% 5% 4% 5% 260 EFAN 38 (74%) 7 (14%) 1 (2%) 5 (10%) 51 All Networks 83% 5% 1% 10% 265 EFAN 49 (96%) - 1 (2%) 1 (2%) 51 All Networks 91% 3% 1% 5% 266 EFAN 49 (96%) - 1 (2%) 1 (2%) 51 All Networks 88% 5% 1% 6% 265 EFAN 42 (82%) 1 (2%) 2 (4%) 6 (12%) 51 All Networks 81% 2% 5% 12% 266 EFAN 42 (82%) - 3 (6%) 6 (12%) 51 All Networks 79% 3% 6% 12% 265 EFAN 41 (80%) - 3 (6%) 7 (14%) 51 All Networks 77% 2% 5% 17% 265 Overall, responses to the Stakeholder Survey questions related to the training outcome were positive. Respondents strongly believed that their Network facilitated meaningful information sharing with its members, partners, and the general public. Respondents stated that training and educational resources offered by their Network were based on research and other leading practices. Respondents said that their participation in training offered through their Network had increased their understanding of FASD and that their ability to provide proper care and support to persons affected by FASD had improved. Finally, the knowledge and expertise of Network leadership was rated highly with respondents expressing strong support for their leadership s capacity in this area. 23 July

26 3.7. Outcome #4: Practice Informed by Research and Evaluation Research findings, including those from monitoring and evaluation systems are used to inform FASD strategic planning, and FASD services and programs. Narrative Report and Focus Group Findings Because EFAN s members sat on the provincial FASD Councils, they were made aware of best practices, evaluation findings and new research, and could easily share the information with others on the Network. In recent years, EFAN has increasingly used social media to disseminate FASD information, through a website, a blog, a Facebook and a Twitter account. Agencies appreciated the fact that the FASD Learning Series offered through the government website was available free of charge, and could be viewed from people s offices or homes at any time. The Learning Series was used as a resource to learn about research and best practice. Occasionally EFAN invited Edmonton researchers to present research findings at their meetings. Smaller member agencies appreciated the fact that by sitting at the EFAN table, they heard first hand about research that happened in larger agencies, and how findings contributed to improving practice. Like other Networks, EFAN contributed to the provincial FASD on- line- reporting system, with the hope of improving data gathering. There have always been contacts between Edmonton FASD researchers and EFAN members offering FASD services. EFAN itself did not engage in research and did not access any research funding. EFAN was originally getting requests to help connect researchers with agencies. It happened less in later years, because agencies had on- going connections with research institutions. Larger agencies, such as Catholic Social Services, were more likely to engage in research than smaller ones, but research findings were shared with everybody. Agencies had learned over time how to screen requests to participate in research projects. They believed that the focus should be on meeting their clients needs, not the researcher s needs. They have also learned to ask for financial resources to cover their costs in participating in research activities. Large agencies, such as Catholic Social Services, who had a long- term working relationship with researchers have benefited from having students helping with their own community- based research questions, needs assessments, and evaluation. Stakeholder Survey Findings Question 35. Evaluation findings are used to guide the work of the Network. 36. Research findings are used to guide the work of the Network. Strongly Agree / Agree Neutral Strongly Disagree / Disagree Don t Know / N/A Total Responses EFAN 35 (68%) 4 (8%) - 12 (24%) 51 All Networks 61% 5% 2% 31% 261 EFAN 34 (66%) 6 (12%) - 11 (22%) 51 All Networks 62% 9% 4% 26% The Network is EFAN 47 (92%) 1 (2%) - 3 (6%) July

27 Question aware of community issues relevant to FASD. 38. Community needs assessments are used to guide the work of the Network. Strongly Agree / Agree Neutral Strongly Disagree / Disagree Don t Know / N/A Total Responses All Networks 82% 4% 5% 10% 262 EFAN 38 (74%) 2 (4%) - 11 (22%) 51 All Networks 60% 7% 5% 28% 261 Findings from the focus groups, the narrative report and the stakeholder survey confirmed that EFAN was aware of community issues relevant to FASD and that community needs assessments, research, and evaluation findings were used to guide their Networks work. Some of EFAN s members were actively engaged in research activities Discussion related to Service Network Outcomes Since the Year- 5 evaluation, EFAN moved from an informal leadership structure to becoming a Society with a Governance Board. There have been few changes in the governance structure as a result of the incorporation, except for the additional responsibilities related to the incorporation. EFAN continues to have an excellent alignment with the non- service delivery aspects of the FASD Service Network Program Guidelines. They have addressed all the recommendations made during the year- 5 evaluation. They have successfully transitioned from their previous informal governance structure to becoming a registered Society. EFAN operates according to four values: collaboration, responsiveness, diversity and accountability. The Network has increased and broadened its membership to over 50 members in the past two years. EFAN funds less than half of all FASD programs offered in Edmonton, and only about 40 percent of EFAN s members receive funding through EFAN, demonstrating that the Network was successful in promoting collaboration and stakeholder engagements. EFAN hosts monthly half- day meetings with its members, which are well attended. Responses to the Stakeholder Survey questions related to collaboration were positive. Respondents demonstrated a belief in their Leadership Team s ability to govern and lead their Network. Additionally, survey respondents believed that their Network was responsive to regional needs and commitment to offering FASD services across the lifespan. Collaboration with Aboriginal and other cultural communities appeared to be a commitment that the Network is making. Focus group participants gave multiple examples of how EFAN promoted stakeholder engagement. Stakeholder survey respondents indicated that their Networks fostered collaboration, actively listened to member s opinions, and valued their input. Further, respondents said that their Network was responsive to the needs of diverse populations and facilitated the development of new partnerships with others in their region with a stake in the FASD community. Finally, respondents strongly believed that their Network was proactive increasing awareness of FASD in their region. Information available from focus groups, narrative reports and the stakeholder survey demonstrated that EFAN successfully promoted FASD training in its target area, for persons affected by FASD, professionals and the general public. Survey respondents believed that their Network facilitated meaningful information sharing, that training and educational resources offered by their Network were based on research and other leading practices, and 23 July

28 that their participation in training offered through their Network had increased their understanding of FASD and that their ability to provide proper care and support to persons affected by FASD had improved. Findings from the focus groups, the narrative report and the stakeholder survey confirmed that EFAN was aware of community issues relevant to FASD and that community needs assessments, research, and evaluation findings were used to guide their Networks work. Some of EFAN s members were actively engaged in research activities. EFAN s stakeholder responses had higher agreement rates for all questions related to collaboration, stakeholder engagement, training, and evaluation & research than the combined responses of all Networks stakeholders. EFAN continues to have a strong governance structure, and is successfully meeting all Service Network outcomes. 23 July

29 4. FASD Indicated and Targeted Prevention 4.1. Logic Model 23 July

30 4.2. Outputs EFAN offers FASD prevention services through the Bosco Homes PCAP program for at risk Aboriginal women and the Catholic Social Services First Steps program. The First Steps program served 28 unique clients and Bosco home 30 unique clients in , through the funding provided by EFAN. The PCAP program funded through Catholic Social Services has multiple funders (EFAN, Edmonton and Area child and family services authority, Signs of Hope fundraising campaign and other provincial FASD funding). Alberta Health Services also provides services to women with substance abuse problems who are or may become pregnant through their enhanced services for women program. EFAN s PCAP program created 33 new client files in ORS (prevention pillar) between July 2012 and September Of those, 24 files had data for at least two consecutive quarters, 12 for 3 consecutive quarters, 4 for 4 consecutive quarters, and 1 for 5 consecutive quarters. Even though PCAP is a 3- year program, ORS data suggests that there were either data entry problems or that a large percentage of clients left the program prior to completing the 3 years. 31 PCAP files could be tracked over 2 reporting periods (6 months); only 24 files (77% of original number) still had data in the 2 nd reporting period. 28 could be tracked over a period of 3 reporting periods (9 months); only 12 files (43% of original number) still had data in the 3 rd reporting period. 11 could be tracked over 4 reporting periods (12 months); only 4 (36%) still had data in the 4 th reporting period. The Alberta PCAP manual suggests approximately 33% of clients will not complete the PCAP program. EFAN ORS data suggests that a lower retention rate. It is unclear whether clients exited the program early, or whether there were problems with ORS data entry. 13 of the 33 women (39%) were known to be pregnant during the quarter their file was newly created; 55% were not pregnant, and the pregnancy status was unknown or not reported for the remaining 6%. Of the women still enrolled in the program during the following quarter (4-6 months in the program), 17% were still pregnant, 54% were not pregnant, and the pregnant status was not reported or unknown for the remaining 29%. The pregnancy outcome was reported for 5 pregnant women only. All of them had a live birth. Three children had a normal birth outcome and 1 had health complications Outcome #1: Reduction in Alcohol and Substance Abuse Clients reduce alcohol and substance abuse. ORS Findings Findings are based on a small sample size: 13 women were pregnant at the time of admission in the PCAP program (0-3 months), 4 were pregnant in the 2 nd quarter in the program (4-6 months) and 1 in the 3 rd quarter (7-9 months) 23 July

31 Percentage of Pregnant Women Consuming Alcohol by Length in PCAP Program, According to ORS Findings Length in PCAP program EFAN Consolidated Alberta Results (n=47) 0-3 months 4-6 months 7 to 9 months 13 pregnant women of whom 11 consumed alcohol (84%) 66 pregnant women of whom 47 were consuming alcohol (70%) 4 pregnant women of whom 2 consumed alcohol (50%) 26 women of whom 11 were consuming alcohol (42%) Change in alcohol consumption during 1 st quarter: EFAN Pregnant women who drank and increased use: 9% Pregnant women who drank and decreased use: 36% Pregnant women who drank and used at the same level: 27% Pregnant women who eliminated alcohol use: 27% Consolidated Alberta FASD Network Results (n=47) 1 pregnant woman who consumed alcohol 11 women of whom 2 were consuming alcohol (18%) Pregnant women who drank and increased use: 11.5% Pregnant women who drank and decreased use: 44.3% Pregnant women who drank and used at the same level: 16.3% Pregnant women who eliminated alcohol use: 27.9% According to ORS data, EFAN s pregnant PCAP clients appeared more likely to consume alcohol while pregnant than all Alberta PCAP clients, but there was a substantial decrease in alcohol consumption between the first and second reporting period. The percentage of women who decreased or eliminated alcohol use during the first quarter in the program was similar for EFAN as for consolidated results. Percentage of Pregnant Women Using Drugs by Length in PCAP Program, According to ORS Findings Length in PCAP program EFAN Consolidated Alberta Results (n=47) 0-3 months 4-6 months 7 to 9 months 13 pregnant women of whom 9 were using drugs (69%) 66 women of whom 37 were using drugs (54%) 4 pregnant women of whom 3 used drugs (75%) 26 women of whom 11 were using drugs (42%) 1 pregnant woman who used drugs 11 women of whom 2 were using drugs (18%) Change in drug use for pregnant women during 1 st quarter: EFAN Pregnant women who used drugs and increased use: 0% Pregnant women who used drugs and decreased use: 44% Pregnant women who used drugs and used at the same level: 33% Pregnant women who eliminated drug use: 22% Consolidated Alberta FASD Network Results (n=47) Pregnant women who used drugs and increased use: 10.5% Pregnant women who used drugs and decreased use: 42.1% Pregnant women who used drugs and used at the same level: 24.6% Pregnant women who eliminated their use: 22.8% EFAN s PCAP clients appeared more likely to use drugs while pregnant than all FASD Networks PCAP clients. There was no apparent decrease in the percentage of women using drugs between the first and second reporting period. The percentage of women who 23 July

32 decreased or eliminated drug use during the first quarter in the program was similar for EFAN as for all FASD Networks. PCAP Data Outcome Reporting Tool Overview In an effort to triangulate ORS data and obtain a snapshot of the women currently enrolled in PCAP programs a time- limited PCAP data outcome reporting tool was developed for this evaluation. This tool was used to collect information about the client s length in program; abstinence of alcohol and drug use for one month during the reporting period; whether the client was pregnant over the past six months; whether the client is regularly using birth control; who has legal custody of the target child, and where that target child is living at six months. EFAN PCAP service providers provided data for 36 current PCAP clients: 12 clients were in their first 6 months of the program; 5 clients had been in the program for 7-12 months; 8 clients had been in the program for months; 0 for 19 to 24 months; 1 for 25 to 30 months; and 10 for 31 to 36 months. The distribution of clients between reporting periods was uneven, and this can be expected, based on the fact that when programs started, they admitted a larger number of clients simultaneously. However, there is a large drop of clients between the year 1 and 2, which could be further explored with service providers (in light of a similar drop in ORS files). Client Drug Abstinence (for at least one month), According to PCAP Outcome Data Reporting Tool EFAN Consolidated Data (all FASD Networks) Abstains from drugs 42% 50% Does not abstain from drugs 42% 36% Don t Know 16% 13% Total 36 clients 285 clients Client Alcohol Abstinence (for at least one month), According to PCAP Outcome Data Reporting Tool EFAN Consolidated Data (all FASD Networks) Abstains from alcohol 39% 43% Does not abstain from alcohol 42% 43% Don t Know 19% 14% Total 36 clients 285 clients EFAN s PCAP clients were slightly less likely to have abstained from drugs and alcohol for at least one month than all FASD Network PCAP clients. 23 July

33 4.4. Outcome #2: Increase in Birth Control and Family Planning Methods Use of birth control and family planning methods increase. ORS Findings FASD can be prevented by either preventing new pregnancies through an effective family planning method or by stopping alcohol consumption if there is no effective family planning. The percentage of non- pregnant women successfully preventing new FASD births is the total of those having an effective FP planning method and those without effective family planning but not drinking alcohol. Based on consolidated data from the PCAP Outcome- Reporting Tool for all Networks, this increased from 68.3% during the first 3 months in the program to 73% in their second and third quarters of the program, and to 90% in the fourth quarter. ORS had EFAN data for 18 non- pregnant women in their first quarter of the program; of which 13 were still in the program in a second quarter, 10 in a 3 rd quarter, 4 in the 4 th quarter, and 1 in the 5 th quarter. 67% of non- pregnant women used an effective family planning method during their first 3 months in the program, 69% in the 2 nd quarter, and 40% in the 3 rd quarter, 0% in the 4 th and 5 th quarter. Of the women not pregnant and not using effective family planning, 33% used alcohol during their first 3 months in the program, 50% in the second quarter, 14% in the 3 rd quarter, 0% in the 4 th quarter, and 100% in the last quarter Of the women not pregnant and not using effective family planning, 17% used drugs during their first 3 months in the program, 50% in the 2 nd quarter, 14% in the 3 rd quarter, 0% in the 4 th and 100% in the 5 th quarter. The percentage of non- pregnant women successfully preventing new FASD births by having an effective birth control method and/or not drinking alcohol was 89% during their first 3 months in the program, 85% for women who had been in the program for 4 to 6 months, and 100 percent for women in the program for 7 to 12 months. This result is based on a very small number of women and may not be representative of all women ever enrolled with PCAP in EFAN s target area. PCAP Data Outcome Reporting Tool % of Non- pregnant PCAP Clients Currently Effectively Preventing new FASD Births Based on PCAP Outcome Reporting Tool EFAN Consolidated Alberta Results Total PCAP clients Total PCAP clients currently pregnant 1 13 Total PCAP clients whose pregnancy status was unknown 1 80 Total non- pregnant PCAP clients % of known non- pregnant PCAP clients using effective birth 41% 54% control method % of non- pregnant PCAP clients, not using an effective birth 3% 17% control method, but not drinking any alcohol % of non- pregnant PCAP clients currently effectively preventing new FASD births 47% 71% 23 July

34 EFAN has data on 36 current PCAP clients, of which 1 was currently pregnant, and pregnancy status was unknown for 1 client. Of the known 34 non- pregnant women, 14 (41%) were using an effective birth control method. The percentage of women effectively using a birth control method was very low for non- pregnant women in their first year of the program (3 out of 15 women, 17%), increased for non- pregnant women in their 2 nd year (3 out of 9 women, 33%), and in their 3 rd year (8 out of 10 women, 80%). Of the 34 non- pregnant women not using an effective birth control method, 1 abstained from alcohol for at least month, which suggests that only 15 out of the 34 women (44%) were effectively preventing new FASD births when the PCAP Data Outcome Reporting Tool was used. This result much is lower than the 71% found when looking at provincial consolidated data Outcome #3: Access to Information and Supports Clients have increased knowledge of and access to community supports and services. Advocate- Client Relationship Inventory Overview The PCAP Advocate- Client Relationship Inventory is a standardized survey that was developed by PCAP researchers at the University of Washington. This survey is used to assess how clients feel about their experience with the PCAP mentor. Further, the survey is used to assess PCAP clients satisfaction with their PCAP program. It was used in the context of this evaluation to provide this type of information. Advocate- Client Relationship Inventory Question Strongly Agree / Agree Neither Agree nor Disagree Strongly Disagree / Disagree Total Responses 1. Helps me understand. EFAN 21 (88%) 2 (8%) 1 (4%) 24 All Networks 90% 8% 2% Understands my situation. EFAN 18 (75%) 5 (21%) 1 (4%) 24 All Networks 88% 9% 4% My work together with my advocate helps my ongoing development and the development of my baby. 11. Understands if I tell her what I want to do. 18. Gives me printed information after she explains it to me. EFAN 19 (83%) 3 (13%) 1 (4%) 23 All Networks 86% 11% 4% 153 EFAN 24 (100%) All Networks 93% 6% 1% 190 EFAN 22 (92%) 2 (8%) - 24 All Networks 78% 14% 8% Understands me. EFAN 24 (100%) All Networks 91% 7% 2% Praises me when I reach a goal. EFAN 24 (100%) All Networks 93% 4% 3% 190 Even though the advocate- client relationship inventory does not include any specific question related to knowledge and access to community supports and services, the PCAP model is a case management model, and the clients responses suggested that women received information and were able to meet their goals (which includes accessing 23 July

35 community supports). Most clients who participated in PCAP program felt that their advocate understood them and their situation and provided them with the information and support required to make healthy choices for them and their children. EFAN data suggested a similar level of agreement with the statements than the provincial consolidated data Outcome #4: Improvement in Well Being Clients experience improvement in their wellbeing (health, mental health, housing, income education, legal support). Advocate- Client Relationship Inventory Question Strongly Agree / Agree Neither Agree nor Disagree Strongly Disagree / Disagree Total Responses 2. Helps me keep a positive EFAN 24 (100%) outlook. All Networks 92% 7% 1% Brings out the best in EFAN 24 (100%) me. All Networks 87% 11% 1% Helps me learn how to EFAN 23 (96%) 1 (4%) - 24 solve my problems. All Networks 85% 11% 4% Encourages me to make EFAN 24 (100%) my own decisions. All Networks 92% 6% 4% Helps my family get EFAN 20 (84%) 2 (8%) 2 (8%) 24 along better. All Networks 70% 21% 9% Does not ask me to do EFAN 23 (96%) 1 (4%) - 24 anything I cannot do. All Networks 86% 11% 3% Helps me develop my EFAN 23 (96%) 1 (4%) - 24 role within my family. All Networks 77% 15% 8% Helps me develop as a EFAN 22 (92%) 2 (8%) - 24 member of my family. All Networks 78% 16% 7% Respects my EFAN 24 (100%) independence. All Networks 93% 4% 3% Accepts my ways. EFAN 23 (96%) 1 (4%) - 24 All Networks 93% 4% 3% Motivates me to protect EFAN 23 (100%) my baby's health. All Networks 93% 6% 1% Cares about what EFAN 23 (96%) 1 (4%) - 24 happens to me. All Networks 95% 5% 0% Is sensitive to how I EFAN 22 (92%) 2 (8%) - 24 feel. All Networks 89% 9% 2% Praises me for eating EFAN 16 (67%) 8 (33%) - 24 healthy food. All Networks 74% 22% 4% Shares with me. EFAN 22 (92%) 2 (8%) - 24 All Networks 83% 13% 4% Encourages me to EFAN 24 (96%) succeed in daily life. All Networks 93% 6% 1% Respects my family's EFAN 23 (96%) 1 (4%) - 24 ways of doing things. All Networks 86% 6% 4% Work we do together builds on my strengths. EFAN 23 (96%) 1 (4%) - 24 All Networks 92% 7% 1% July

36 26. Trust advocate to look after my best interests. 27. Advocate tells me about herself. EFAN 24 (100%) All Networks 96% 2% 2% 190 EFAN 22 (92%) 2 (8%) - 24 All Networks 73% 18% 9% 190 The majority of clients who participated in PCAP programming had a positive impression of their advocate. This suggests that advocates help clients improve their wellbeing. ORS Findings Presenting issues have been reviewed for all 36 women with a new file in the ORS PCAP prevention pillar, regardless of their pregnancy status, between April 1, 2012 and September The data currently entered in ORS allows for an analysis of issues for only 5 quarters. However the sample sizes for the 4 th and 5 th quarter (4 and 1 client files) were small and may not be representative. Quarter 5 data had been excluded. Issues for Clients in Prevention Programs during the 1 st quarter in PCAP Program Presenting Issue Frequency of clients with the issue at admission (EFAN) Frequency of clients with the issue at admission (All Networks) Housing 66.67% 60.19% Addiction 63.64% 66.24% Health 24.24% 33.76% Mental Health 54.55% 45.22% Behavior 15.15% 21.02% Employment 18.18% 24.20% Education 39.39% 28.98% Legal 18.18% 21.43% Adaptive / Life Skills 45.45% 37.58% Social Skills 30.30% 22.93% Finances 48.48% 44.27% Custody 27.27% 34.08% Family Violence 27.27% 26.11% 23 July

37 Changes in presenting issues by length in prevention program (EFAN) The most common presenting issues for women in the EFAN PCAP program were similar as for the provincial sample. Housing, addiction, mental health, adaptive life skills and finances were presenting issues for over 45% of the women when their files were first opened in ORS. As with the provincial consolidated data, for many issues, there appears to be an initial increase in the prevalence of issues followed by a decrease over time Outcome #5: Decrease in Placement Disruption There is a decrease in placement disruption and child welfare involvement. PCAP Outcome Data Reporting Tool PCAP s Clients Custody and Residence of Children based on PCAP Outcome- Reporting Tool EFAN Consolidated Alberta Results Total PCAP clients Total PCAP clients with a target child % PCAP clients with a target child 92% 85% % of women with a target child who had legal custody of that 63% 60% child % of women with a target child whose child lived with them 63% 58% 63% of EFAN PCAP clients with a target child had custody of such children, and 63% of the women with a target child had their child living with them. The rates are similar to the provincial consolidated rates Outcome #6: Client Satisfaction, Involvement and Retention Client satisfaction, involvement and retention. 23 July

38 As presented in section 4.6, data from the advocate client relationship inventory suggests that clients are satisfied about their relationship with PCAP advocates Other Prevention Outcomes # 7: Staff Training, Supervision and Retention. #8: Interdisciplinary team collaboration and informed approaches. # 9: Program Accessibility and Flexibility. Service Networks were not required to report specifically on PCAP staff training, supervision and retention, interdisciplinary team collaboration, informed practices, and program accessibility & flexibility Discussion Related to FASD Indicated and Targeted Prevention In , 58 women accessed Parent Child Assistance Prevention (PCAP) programming through EFAN funding. A total of 33 new client files were created in ORS for PCAP clients between June 2012 and October Even though PCAP is a 3- Year program, ORS data suggests that only 77% of the original client files remained active in the subsequent reporting period (4-6 months), 44% in the 6-9 months reporting period, and 36% in the months reporting period following the creation of the original files. It is unknown whether clients exited the program early, or whether there were problems with ORS data entry. In the PCAP outcome tool developed for the Year- 7 evaluation, of the 36 clients for whom data was reported, the distribution of clients was also suggested a drop of clients during the first and second year, beyond accepted rates. Recommendation: EFAN should discuss the apparent low retention rates of PCAP clients with its service providers, and if confirmed, discuss how retention rates could be increased. According to ORS data, EFAN s pregnant PCAP clients appeared more likely to consume alcohol and/or drugs while pregnant than all Alberta PCAP clients. There was a substantial decrease in alcohol consumption between the first and second reporting period. The percentage of women who decreased or eliminated alcohol use during the first quarter in the program was similar for EFAN as for consolidated results. There was no apparent decrease in the percentage of women using drugs between the first and second reporting period. The percentage of women who decreased or eliminated drug use during the first quarter in the program was similar for EFAN as for all FASD Networks. EFAN s PCAP clients were slightly less likely to have abstained from drugs and alcohol for at least one month than all FASD Network PCAP clients. Based on ORS data, the percentage of non- pregnant women successfully preventing new FASD births by having an effective birth control method and/or not drinking alcohol was 89% during their first 3 months in the program, 85% for women who had been in the program for 4 to 6 months, and 100 percent for women in the program for 7 to 12 months. This result is based on a very small number of women and may not be representative of all women ever enrolled with PCAP in EFAN s target area. 23 July

39 EFAN submitted data on 36 current PCAP clients through the PCAP Outcome Reporting tool. Of the known 34 non- pregnant women, 14 (41%) were using an effective birth control method. The percentage of non- pregnant women effectively using a birth control method was very low in the 1 st and 2 nd year of the program (17% and 33%), but increased to 80% in the 3 rd year. Of the 34 non- pregnant women not using an effective birth control method, 1 abstained from alcohol for at least month, which suggests that only 15 out of the 34 women were effectively prevention new FASD births when the PCAP Data Outcome Reporting Tool as used. This result much is lower than the 71% found when looking at provincial consolidated data. There is a discrepancy between findings from ORS and the PCAP outcome- reporting tool. Recommendation: The low rates in use of birth control in the first two years of the program and the low rate of women effectively preventing FASD births should be investigated further. The most common presenting issues for women in the EFAN PCAP program were similar as for the provincial sample. Housing, addiction, mental health, adaptive life skills and finances were presenting issues for over 45% of the women when their files were first opened in ORS. As with the provincial consolidated data, for many issues, there appears to be an initial increase in the prevalence of issues followed by a decrease over time. 63% of EFAN PCAP clients with a target child had custody of such children, and 63% of the children lived with their mothers. The rates are similar to the provincial consolidated rates. Overall, EFAN s PCAP programs appear to meet the Service Network Prevention Outcomes. More information is needed to explain lower than provincial retention rates and effective prevention of FASD birth rates. 23 July

40 5. FASD Assessment and Diagnosis 5.1. Logic Model 23 July

41 5.2. Outputs EFAN funds FASD assessment and diagnosis services in partnership with Alberta Health Services at the Glenrose Children s FASD Clinic. The Network funding provides one additional assessment per month at the Glenrose clinic, or a total of 12 assessments per year. Edmonton has several other FASD assessment and diagnosis clinic sites not funded through EFAN. The Glenrose Rehabilitation program has an adult FASD clinic; and the Canadian FASD Diagnostic and Training Centre has a child, youth and adult mobile clinic team Outcome #1: Understanding of FASD Clients and caregivers have increased understanding of how FASD affects them. Caregivers of clients participating in the assessment and diagnosis process at FASD clinics were asked to complete a post- clinic caregiver survey at the conclusion of the child/dependent s process on Clinic Day. Caregivers were asked to provide feedback on how the assessment and diagnosis process helped them from a knowledge standpoint. Further, caregivers were asked to describe how their child/dependent felt the process went and what kind of an impact the diagnosis will have on their life in the future. In total, 7 of these surveys were submitted through EFAN. Six caregivers (86%) had children ages 6-12 years old, and 1 caregiver s dependent (14%) was in the years old category. Post- Clinic Client Survey Overview Clients that were over 13 years of age and deemed to be capable of survey completion by assessment and diagnosis clinic staff were asked to complete a post- clinic client survey upon completion of their process. Clients were asked to provide feedback on their experience with the clinic process and to describe how the diagnosis will help them in the future. Only 1 of these surveys was submitted through EFAN. The client that completed the survey was 25 years old or older. Because of the small sample size (7 surveys completed by caregivers, and 1 by a client), the findings should be interpreted with caution. Recommendation: EFAN should consider offering post- clinic surveys on an on- going basis, in order to increase the validity of their evaluation data. Findings - Post Clinic Caregiver Survey Post- clinic Caregiver Survey 9. As a result of our interactions with the clinic before, during, and after the clinic day, I understand FASD better. 10. I understand my child / dependent's strengths and weaknesses better. EFAN All Networks EFAN All Networks Completely agree / somewhat agree Neither agree nor disagree Completely disagree / somewhat disagree Did not know / NA 82 (97%) 3 (3%) (99%) 1 (1%) Total 23 July

42 Post- clinic Caregiver Survey 11. I understand better how my child / dependent thinks and learns since she/he received the assessment and diagnosis. EFAN All Networks Completely agree / somewhat agree Neither agree nor disagree Completely disagree / somewhat disagree Did not know / NA 83 (98%) 2 (2%) Total Post Clinic Caregiver Survey Yes No Don t know Question 18: Does your child / dependent feel that she/he understands her/his condition better because of her/his involvement with the clinic? Total Responses EFAN 2 (29%) 3 (43%) 2 (29%) 7 All Networks 43 (51%) 10 (11%) 32 (38%) 85 The responses of the parents/ caregivers regarding their children s / dependents feelings were the same, regardless of the children s / dependents ages. I am so happy that we had this assessment done. It has helped me understand how much damage has been done to my child s brain from alcohol. - Parent/caregiver Findings: Post Clinic Client Survey Post Clinic Client Survey Yes No Maybe Did not know / NA 7: Do you understand FASD better after coming to our clinic? 8: Do you understand your condition better after coming to our clinic? 9: Do you understand your strengths and weaknesses better because you came to our clinic? 10: Do you have a better understanding on how you think and learn after coming to our clinic? EFAN 1 (100%) All Networks 34 (85%) 3 (8%) 3 (8%) 0 (0%) EFAN 1 (100%) All Networks 34 (85%) 1 (3%) 4 (10%) 1 (3%) EFAN 1 (100%) All Networks 32 (80%) 2 (5%) 4 (13%) 1 (3%) EFAN 1 (100%) All Networks 36 (90%) 2 (5%) 1 (3%) 1 (3%) The responses of the surveys submitted through EFAN are similar to the provincial consolidated responses. Survey results suggest that both parents/caregivers and clients had an improved understanding of FASD after the clinic day. 23 July

43 Survey results also suggested that the level of understanding in relation to strengths and weaknesses and thinking and learning patterns had improved for both parents/caregivers and clients after the clinic day as well. All parents/caregivers and clients had a positive, favorable impression of the diagnostic process after their assessment clinic experience. The majority of parents/caregivers and clients indicated that they understood FASD better after their clinic experience Outcome #2: Knowledge of Supports and Services in the Community Clients and caregivers have increased knowledge of supports and services available to aid them in the community. Findings: Post Clinic Caregiver Survey Post Clinic Caregiver Survey Question 12: I am confident that the recommendations made by the clinic team will be useful to support my child / dependent. Completely Agree / Somewhat Agree Neither Agree nor Disagree Completely Disagree / Somewhat Disagree Don t Know / N/A Total Responses EFAN 7 (100%) All Networks 72(85%) 12 (14%) 1 (1%) 0 85 Post Clinic Caregiver Survey Yes No Don t know Question 19: Did your child/dependent feel that the clinic helped her/him learn about new services and supports in the community? Total Responses EFAN 1 (14%) 4 (57%) 2 (29%) 7 All Networks 36(42%) 11(13%) 38 (45%) 85 Findings: Post Clinic Client Survey Post Clinic Client Survey Yes No Maybe Don t know NA Question 11: Did someone at our clinic tell you about new programs, organizations, or people that might be able to help you? EFAN 1 (100%) All Networks 36(42%) 1 (3%) 2 (4%) 1 (3%) All parents / caregivers indicated that they were completely confident that the recommendations made by the clinic team would be useful to support their child / dependent. Only 14% of the caregivers felt that the clinic helped their their child/dependent learn about new services in the community. It is possible that because the children were relatively young, this question was not relevant to their age group. The clinic gave us an abundance of information that was all really great. At first I felt overwhelmed with the diagnosis, but I now feel very confident my child will succeed with help. - Parent/caregiver 23 July

44 5.5. Outcome #3: Referral to Recommended Supports Clients and caregivers are referred to recommended supports, based on assessments. Post Clinic Caregiver Survey Post Clinic Caregiver Survey 13. I have been given new information about what help is available for me and my child/dependent in the community, since she/he received the assessment and diagnosis. 14. Someone has helped in accessing supports and/or other resources recommended for my child/dependent. 15. I plan to access support in the future. Completely Agree / Somewhat Agree Neither Agree nor Disagree Completely Disagree / Somewhat Disagree Don t Know / N/A Total Responses EFAN 6 (86%) - 1 (14%) - 7 All Networks 75 (89%) 2 (2%) 3 (4%) 5 (6%) 85 EFAN 6 (86%) - 1 (14%) - 7 All Networks 69 (81%) 4 (5%) 4 (5%) 8 (9%) 85 EFAN 6 (86%) - 1 (14%) - 7 All Networks 79 (93%) - 1 (1%) 5 (6%) 85 Post Clinic Client Survey Post Clinic Client Survey Yes No Maybe Don t know NA Question 12: Did someone at our clinic help you get the supports and services that were recommended for you? Question 13: Do you plan to get the supports and services that were recommended for you? EFAN 1 (100%) All Networks 32 (80%) 4 (10%) 1 (3%) 3 (8%) EFAN 1 (100%) All Networks 32 (80%) 0 (0%) 7 (13%) 1 (3%) As parents of child with FASD the clinic was critical to us getting a proper diagnosis and therefore be able to share information with their school. This information will aid the school in supporting our child better. Having access to bi- weekly support from the worker has helped to manage some of my child s learning challenges. - Parent/caregiver 23 July

45 The majority of parents/caregivers and all clients indicated that they received new information about services and supports, and that they planned to connect with the services and supports that were recommended to them by assessment clinic staff Outcome #4: Client Satisfaction, Involvement and Retention Client satisfaction, involvement and retention Findings: Post Clinic Caregiver Survey Question 4. According to you, how easy was the explanation of the assessment and diagnosis process? 5. How easy was it for you to understand the diagnosis? Easy to Understand Somewhat Complicated to Understand Too Complicated to Understand Don t Know / N/A EFAN 6 (86%) 1 (14%) All Networks Total Responses 73 (86%) 12 (14%) EFAN 5 (71%) 2 (29%) All Networks 69 (82%) 16 (19%) Post Clinic Caregiver Survey A lot Some Very little Question 6: How much confidence do you have in the results of the evaluation? None Don t know or NA EFAN 7 (100%) All Networks 74 (87%) 10 (12%) 1 (1%) 0 (0%) 0 (0%) Post Clinic Caregiver Survey Question 7: Overall, how satisfied were you with the assessment and diagnosis services my child / dependent received? Very satisfied or satisfied Neutral Dissatisfied or very dissatisfied Don t know or NA Total Responses EFAN 7 (100%) All Networks 81 (95%) 4 (5%) 0 (0%) 0 (0%) 85 Post Clinic Caregiver Survey Question 8: I received what I hoped to gain from the clinic. Completely Agree / Somewhat Agree Neither Agree nor Disagree Completely Disagree / Somewhat Disagree Don t Know / N/A Total Responses EFAN 7 (100%) All Networks 83 (98%) 1 (1%) 0 (0%) 1 (1%) 23 July

46 Post Clinic Caregiver Survey Question 16: What was your child / dependent's overall experience about the clinic day? Very good or good Indifferent Bad or very bad Don t Know / N/A Total Responses EFAN 7 (100%) All Networks 77 (92%) 1 (1%) 0 (0%) 6 (7%) 85 Post Clinic Caregiver Survey Yes No Don t know or NA Question 17: Did your child / dependent feel that the clinic staff treated her/him with dignity and respect? Question 20: Did your child / dependent feel that the assessment will help her/him in the future? Total responses EFAN 7 (100%) All Networks 80 (94%) 0 (0%) 5 (6%) 85 EFAN 4 (57%) 3 (43%) - 7 All Networks 39 (46%) 6 (7%) 40 (47%) 85 Findings: Post Clinic Client Survey Post Clinic Client Survey Yes No Maybe Don t know Question 4: Did you understand what would be happening at the clinic? Question 5: Did you understand your diagnosis? Question 6: Are you happy with the services you received? Question 14: Do you think that the information you got from our clinic will help you in the future? Question 15: Did the people in our clinic treat you with respect? Total Responses EFAN 1 (100%) All Networks 34 (85%) 2 (5%) 3 (8%) 1 (3%) 40 EFAN 1 (100%) All Networks 33 (83%) 2 (5%) 3 (8%) 1 (3%) 40 EFAN 1 (100%) All Networks 37 (93%) 1 (3%) 0 (0%) 2 (5%) 40 EFAN 1 (100%) All Networks 30 (75%) 0 (0%) 8 (20%) 2 (5%) 40 EFAN 1 (100%) All Networks 40 (100%) 0 (0%) 0 (0%) 0 (0%) 40 My child was worried about the outcome of course. They were fearful of hearing the worst. However, the information from the assessment was presented in a manner that made my child feel more positive about themselves and more empowered about their future. Thank you. - Parent/caregiver One hundred percent professional. Couldn t have asked for a better service. Thank you. - Client 23 July

47 The majority of parents/caregivers and clients were satisfied with their assessment clinic experience and indicated that they received what they wanted from the process. Parents/caregivers and clients said that their diagnosis was easily understood and that they were confident in the assessment process findings. Finally, parents/caregivers and clients unanimously stated that they felt they were treated with dignity and respect by assessment clinic staff Other Outcomes: #7: Staff Training, Supervision and Retention. # 8: Interdisciplinary team collaboration and informed approaches. # 9: Program Accessibility and Flexibility. Service Networks were not required to report specifically on PCAP staff training, supervision and retention, interdisciplinary team collaboration, informed practices, and program accessibility & flexibility Discussion Related to Assessment and Diagnosis EFAN funds FASD assessment and diagnosis services in partnership with Alberta Health Services at the Glenrose Children s FASD Clinic. The Network funding provides one additional assessment per month at the Glenrose clinic, or a total of 12 assessments per year. Only 8 post- clinic surveys were completed for EFAN as part of the Year- 7 Service Network Evaluation, 7 by caregivers, and 1 by a client. Because of the small sample size, the survey findings should be interpreted with caution. Recommendation: EFAN should consider offering post- clinic surveys on an on- going basis, in order to increase the validity of their clinic evaluation findings. Survey results suggest that both parents/caregivers and clients had an improved understanding of FASD after the clinic day; and that the level of understanding in relation to strengths and weaknesses and thinking and learning patterns had improved for both parents/caregivers and clients after the clinic day as well. All parents/caregivers and clients had a positive, favorable impression of the diagnostic process after their assessment clinic experience. The majority of parents/caregivers and clients indicated that they understood FASD better after their clinic experience. The majority of parents/caregivers and all clients indicated that they received new information about services and supports, and that they planned to connect with the services and supports that were recommended to them by assessment clinic staff. The majority of parents/caregivers and clients were satisfied with their assessment clinic experience and indicated that they received what they wanted from the process. They believed that their diagnosis was easily understood and that they were confident in the 23 July

48 assessment process. They unanimously stated that they felt they were treated with dignity and respect by assessment clinic staff. Overall, findings suggest that EFAN s funded FASD assessment and diagnostic services positively contribute towards the provincial outcomes, at a similar level as the consolidated provincial findings. However, EFAN funds only a small number of assessments, and results need to be validated with a larger number. 23 July

49 6. FASD Support Services 6.1. Logic Model 23 July

50 6.2. Outputs Focus groups and Narrative Report Findings In , EFAN provided funding for support services to 8 different agencies in Edmonton: Elizabeth Fry Society: the Girls Empowered and Strong program is a support group for teenage girls, ages 13 to 17, impacted by FASD, who want to address the many issues facing them. The goal of the program is to provide the girls with the skills and knowledge they need in order to make healthy lifestyle choices, thereby preventing further involvement with the legal system. The program offers workshops on various topics and summer programming. In the program reached 45 unique clients. Catholic Social Services: the McDaniel Youth Program is a mentorship program for teenagers ages 14 to 19, impacted by FASD. The program provides one- on- one mentorship to youth to help them access community supports and make a successful transition into adulthood. It also includes group activities. The program works in partnership with other community agencies, include a mobile addiction counselor. Mentors work with youth for a period of up to 3 years. In , the program served 41 unique clients. Bissel Centre, FASD Services: adult mentorship program. The adult advocate provides one- on- one long- term support to adults who are affected by FASD and male partners of PCAP participants impacted by FASD. The program helps clients access community supports and addresses secondary disabilities associated with FASD. The mentor works with program participants for a period of up to 5 years. In , the program had 16 unique clients. Bosco Homes: the Open Arms Program provides culturally sensitive mentorship services to mothers impacted by FASD who are actively parenting children ages 6 to 18. It provides hands on assistance to access services and supports. In , 38 mothers accessed the services. Métis Nation of Alberta: the Mentorship services to Métis youth and families impacted by FASD provides support services to Métis children, youth and adults through culturally one- to- one mentorship. In , the program reached out to 34 individuals. Bridges, Leduc Country: the Bridges FASD mentor program is a support program for adults (18 years or older) living within Leduc County who have either strong indicators, or a diagnosis of FASD. It provides support for 3 years to help stabilize the lives of affected individuals and their children, and aims to teach and model appropriate daily life management and skills. In , the program served 32 unique individuals. Kids Kottage / Elves: Elves Special Needs Society and the Kids Kottage Foundation have formed a cooperative association to provide the FASD Respite care program which supports families with children ages 3 to 8 years old, affected by FASD. Offering a variety of educational and recreational activities, children are cared for in a safe and structured environment. The program runs from September to June, and provides up to a maximum of 6 hours of respite per week for each child, with a maximum of 20 participants in attendance per session. In , parents were also invited to participate in a weekly support group / training session. In , 87 unique clients received respite care. CASA: the Family Reno s program provides sensory processing and behavior management strategies to families whose children are impacted by FASD and a mental 23 July

51 health disorder, in order to improve the parent- child relationship and family functioning. 26 clients received services in Many of the EFAN- funded agencies have multiple funders for the FASD programs. Additional support services available in the Edmonton area include programs such as the Wellness, Resiliency and Partnership (WRAP) program for junior and senior high schools with FASD which is offered in 14 school sites; or the Alberta Health Services Forensic Psychiatry program for youth. ORS Findings EFAN s service providers created 269 new ORS files in the support pillar between April 2012 and September EFAN provides support services to clients of all age groups; EFAN also provides support to caregivers. ORS sample size by age group. Age Group Data for 1 reporting period Data for 2 reporting period Data for 3 reporting period Data for 4 reporting period Data for 4 reporting period years old Caregivers Total years old years old years old or more We reviewed 60 unique files of children, ages % of children, age 0-12, with at least 2 reporting periods, still had data entered during the second reporting period. There was an 11% drop- off during the 3 rd reporting period for children with at least 3 reporting periods. We reviewed 50 unique files of youth, ages % of youth, age 13-17, with at least 2 reporting periods, still had data entered during the second reporting period. There was an 18% drop- off during the 3 rd reporting period for youth with at least 3 reporting periods. We reviewed 38 unique files of young adults, ages 18 to % of young adults, with at least 2 reporting periods, still had data entered during the second reporting period. There was a 21% drop- off during the 3 rd reporting period for youth with at least 3 reporting periods. We reviewed 44 unique files of adults, ages % of young adults, with at least 2 reporting periods, still had data entered during the second reporting period. There was a 21% drop- off during the 3 rd reporting period for adults with at least 3 reporting periods. We reviewed 77 unique files of caregivers. There was a 2% drop- off in the number of ORS caregiver files with at least 2 reporting period, and another 16% drop off during the 3 rd reporting period for caregivers. ORS data confirmed that EFAN provided support services to clients of all age groups and to caregivers, that almost all clients received services for at least 2 reporting quarters, and the great majority for longer periods. 23 July

52 6.3. Outcome #1: Individuals with an FASD Services Based on their Needs Individuals diagnosed with and/or suspected of FASD have access to coordinated services based on their needs. Service Networks were not required to report specifically on how their support services were structured Outcome #2: Individuals with an FASD have Knowledge and Access to Community Resources Individuals diagnosed with and/or suspected of FASD have knowledge of and access to community resources. Survey Findings Caregiver Responding on Behalf of the Client Support Services Survey Overview Parents/caregivers of clients receiving support services were asked to complete a survey on behalf of their child/dependent. All parents/caregivers, regardless of the child/dependent s age, were asked to complete one survey during the course of the data collection period (i.e. July through December 2013). Parents/caregivers were asked to provide feedback on their child/dependent s experience with the agency and staff providing support services to them and to describe how these services will help them in the future. 30 of these surveys were submitted. The survey breakdown by age of client served by a support service agency was as follows: Age of Client N (%) 3 to 5 years old 15 (50%) 6 to 12 years old 14 (47%) 13 to 17 years old - 18 years old or older 1 (3%) Client Support Services Survey Overview Clients that were over 13 years of age and deemed to be capable of survey completion by support service staff were asked to complete one survey during the course of the data collection period (i.e. July through December 2013). Clients were asked to provide feedback on their experience with the agency and staff providing support services to them and to describe how these services will help them in the future. 99 of these surveys were submitted. The survey breakdown by age of client who completed the survey was as follows: Age of Client N (%) 13 to 17 years old 34 (34%) 18 years old or older 65 (66%) Caregiver as the Individual Receiving Support Survey Overview Parents/caregivers who were receiving support services themselves were asked to complete one survey during the course of the data collection period (i.e. July through December 2013). Parents/caregivers were asked to provide feedback on their experience 23 July

53 with the agency and staff providing support services to them and to describe how these services will help them in the future. 37 of these surveys were submitted. The survey breakdown by age of client served by a support service agency was as follows: Age of Client N (%) 3 to 5 years old 14 (38%) 6 to 12 years old 14 (38%) 13 to 17 years old 3 (8%) 18 years old or older 6 (16%) Caregiver on Behalf of Client Support Services Survey Caregiver on Behalf of Client Survey Question 6: The agency helped my child/dependent in the following areas: Areas of Support EFAN (% by total number of responses) Consolidated Provincial Findings (All Networks) (n = 161) Taking care of herself/himself 30 (100%) 55% Going back to school 4 (13%) 14% Doing better in school 18 (60%) 58% Finding a job - 8% Doing better at her/his job - 4% Finding a new place to live - 8% Helping her/him take better care of the place she / he lives 8 (27%) 14% Taking care of her/his money 1 (3%) 10% Accessing a new source of income 1 (3%) 9% Taking care of her/his health (physical health, dental health, mental health ) 12 (40%) 33% Dealing with alcohol and/or drugs - 14% Dealing with legal issues - 9% Her / his relationships with family and friends 28 (93%) 64% Other 3 (10%) 13% Total Responses 105 Examples of other areas of support include domestic violence, childcare, parenting, physical activities, self- esteem, social skills, and disability support. Caregiver on behalf of client surveys Questions 10. I know where to look for help if my child/dependent needs help. 11. The supports we received helped us find and use other programs and services that can assist us. Completely Agree / Somewhat Agree Neither Agree nor Disagree Completely Disagree / Somewhat Disagree Don t Know / N/A Total Responses EFAN 30 (100%) All Networks 96% 1% 6% 1% 183 EFAN 30 (100%) All Networks 84% 4% 11% 1% July

54 Client Support Services Survey Client Survey Question 6: The agency helped my child/dependent in the following areas: Areas of Support N (% by total number of responses) Consolidated Provincial Findings (All Networks) (n = 161) Taking care of herself/himself 61 (62%) 68% Going back to school 25 (25%) 20% Doing better in school 25 (25%) 20% Finding a job 39 (39%) 30% Doing better at her/his job 13 (13%) 12% Finding a new place to live 35 (35%) 29% Helping her/him take better care of the place she / he lives 32 (32%) 26% Taking care of her/his money 30 (30%) 34% Accessing a new source of income 22 (22%) 32% Taking care of her/his health (physical health, dental health, mental health ) 39 (39%) 50% Dealing with alcohol and/or drugs 46 (46%) 39% Dealing with legal issues 33 (33%) 33% Her / his relationships with family and friends 45 (45%) 51% Other 9 (9%) 8% Total Responses 454 Examples of other areas of support include transportation, childcare, parenting, connecting with FASD assessment and diagnosis, and daily living and organizational skills. Client Survey Question Yes No Maybe Don t Know / N/A 10. Do you know where to look for help if you need help? 11. Did someone at our agency tell you about new programs, organizations, or people that can help you? 12. Did someone at our agency help you get new supports and services? Total Responses EFAN 86 (87%) 1 (1%) 9 (9%) 3 (3%) 99 All Networks 86% 2% 10% 2% 348 EFAN 78 (78%) 7 (7%) 8 (8%) 6 (6%) 99 All Networks 79% 8% 8% 6% 348 EFAN 69 (69%) 7 (7%) 9 (9%) 14 (14%) 99 All Networks 76% 7% 8% 9% 347 Discussion / Summary Statement EFAN s responses are similar to the provincial consolidated responses. Clients who participated in support services received support for a myriad of issues. Taking care of themselves and support with family relationships were two of the most common issues supported. All parents/caregivers and a significant proportion of clients indicated that support services staff have taught them where to look for help and have connected them with new supports and services in the community. 23 July

55 6.5. Outcome #3: Individuals with an FASD Experience Improvement in Well- Being Individuals diagnosed with and/or suspected of FASD experience improvement in their wellbeing. Survey Findings Caregiver on Behalf of Client Support Services Survey Caregiver on Behalf of Client Question 8. As a result of working with this program I know more about my child/ dependent s strengths and what she/he is good at. 9. As a result of working with this program we have learned things we need to change or do differently. 12. We are better able to take care of ourselves since we started getting services from the agency. Completely Agree / Somewhat Agree Neither Agree nor Disagree Completely Disagree / Somewhat Disagree Don t Know / N/A Total Responses EFAN 29 (97%) 1 (3%) All Networks 153 (84%) 15 (8%) 14 (8%) 1 (1%) 183 EFAN 30 (100%) All Networks 164 (90%) 4 (2%) 12 (7%) 3 (2%) 183 EFAN 30 (100%) All Networks 156 (85%) 10 (6%) 15 (8%) 2 (1%) 183 Client Support Services Survey Client Survey Question Yes No Maybe Don t Know / N/A Total Responses 8. Do you know more about EFAN 72 (72%) 4 (4%) 12 (12%) 11 (11%) 99 your strengths and weaknesses after coming to our agency? All Networks 74% 6% 13% 8% Have you learned about things you need to change or do differently after coming to our agency? 13. Are you able to better take care of yourself since you started coming to our agency? EFAN 78 (78%) 4 (4%) 7 (7%) 10 (10%) 99 All Networks 75% 3% 10% 11% 183 EFAN 77 (77%) 1 (1%) 13 (13%) 8 (8%) 99 All Networks 76% 2% 15% 7% 183 The majority of parents/caregivers and clients who participated in support services indicated that they have a better understanding of their strengths and weaknesses and know what they need to change about their behavior to take better care of themselves as a result of their participation. 23 July

56 ORS Findings Support Services to Children, 0-12 years old Presenting Issues of Children 0-12 Years Old (60 children) 1 Issues for 0-12 year- old children in Support Programs during their 1 st quarter in the program Presenting Issue Frequency of clients with the issue at admission (EFAN n= 60) Frequency of clients with the issue at admission (All Networks) (n = 255) Housing 15.00% 11.76% Addiction 0% 0.78% Health 3.33% 20.78% Mental Health 45% 26.67% Behavior 43.33% 38.82% Employment 0% 1.18% Education 50.00% 42.75% Legal 3.33% 1.96% Adaptive / Life Skills 11.67% 34.12% Social skills 43.33% 52.16% Finance 1.67% 2.35% Custody of children 15.00% 5.49% Family violence 1.67% 2.35% The three most frequent presenting issues are the same for children receiving EFAN- funded support services as in the provincial sample: behavior (43%), education (50%), and social skills (43%). Except for mental health, the EFAN sample does not appear to show a decrease in frequency of presenting issue for children who have been receiving support services over several quarters. This finding is in line with provincial findings, which show the start of a decrease in the second year of service only. 1 There are only 9 clients in quarter 4. Data needs to be interpreted with caution. 23 July

57 ORS Findings Support Services for Youth, Ages 13 to 17 Presenting Issues of children years old (50 youth) 2 Presenting issues for Youth year- old in Support Programs during their 1 st quarter in the program Presenting Issue Frequency of clients with the issue at admission (EFAN n= 50) Frequency of clients with the issue at admission (All Networks) (n = 314) Housing 22% 13.06% Addiction 6% 5.73% Health 6% 10.19% Mental Health 12% 13.38% Behavior 26% 24.84% Employment 6% 7.96% Education 30% 25.80% Legal 10% 5.10% Adaptive / Life Skills 50% 30.57% Social skills 66% 38.85% Finance 4% 3.82% Custody of children 18% 5.10% Family violence 6% 1.59% The four most frequent presenting issues are the same for youth years old receiving EFAN- funded support services as in the provincial sample: social skills (66%), adaptive/life skills (50%), education (30%), behavior (26%). For most of the issues, the percentages are higher in the EFAN sample than in the provincial consolidated sample. EFAN youth also have a higher rate of custody of children issue (18%) than the provincial sample. Over time, there appears to be an increase in the percentage of youth with most presenting issues. The provincial sample shows similar trends; it is likely that many issues faced by youth are recognized only after building a relationship with youth. More data over a longer period of time is needed to confirm the findings. 2 There are only 5 clients in quarter 4. Data needs to be interpreted with caution. 23 July

58 ORS Findings - Support Services for Young Adults, Ages 18 to 24 Most Common Presenting issues of young adults, years old (38 clients) 3 Issues for young adults, years old in Support Programs during their 1 st quarter in the program Presenting Issue Frequency of clients with the issue at admission (EFAN n= 38) Frequency of clients with the issue at admission (All Networks) (n = 314) Housing 44.74% 48.39% Addiction 18.42% 34.31% Health 21.05% 31.96% Mental Health 34.21% 37.24% Behavior 39.47% 32.55% Employment 34.21% 48.39% Education 36.84% 28.74% Legal 15.79% 26.69% Adaptive / Life Skills 52.63% 58.94% Social skills 47.37% 41.94% Finance 28.95% 53.96% Custody of children 18.42% 11.14% Family violence 13.16% 10.56% All 13 issues listed in ORS are relevant to young adults to a certain extent. Each of the issues was present for at least 10% of them during the first quarter. The most common presenting issues at admission were: housing (45%), adaptive skills / live skills (53%), social skills (48%), mental health (34%), employment (34%), education (37%) and behavior (39%). The percentage of issues for EFAN- funded clients is very similar to the provincial percentage, except for addiction, health and legal issues, which appeared to be less frequent for EFAN than for all the Network clients. There appears to be an initial increase in the prevalence for most issues followed by a decrease in subsequent quarters, which is consistent with the provincial patterns. 3 There are only 7 clients in quarter 4; data needs to be interpreted with caution. 23 July

59 ORS Findings - Support Services to adults, 25+ years old Most Common Presenting issues of adults, 25+ (44 clients) Issues for adults, 25+ in Support Programs during their 1 st quarter in the program Presenting Issue Frequency of clients with the issue at admission (EFAN n= 44) Frequency of clients with the issue at admission (All Networks) (n = 652) Housing 50.00% 28.07% Addiction 38.64% 25.61% Health 38.64% 19.63% Mental Health 65.91% 33.74% Behavior 27.27% 25.61% Employment 43.18% 25.00% Education 27.27% 14.26% Legal 36.36% 17.94% Adaptive Skills / Life Skills 34.09% 32.67% Social Skills 38.64% 22.85% Finance 52.27% 32.52% Custody of children 27.27% 13.96% Family violence 11.36% 7.36% All 13 issues listed in ORS are relevant to adults to a certain extent. Each of the issues was present for at least 10% of them during the first quarter. The most common presenting issues at admission were: mental health (66%), finances (52%), housing (50%), and employment (43%). There are similarities between the EFAN- sample and the provincial sample, but for each of the presenting issues, the percentage of EFAN- funded clients is higher than the percentage in the provincial sample. For most issues, except for mental health, addiction and behavior, there appears to be a decrease over time. It is likely that adults who continue receiving services over a longer period of time are the ones with the most severe issues and the least supportive environment, thus positive outcomes may be understated. 23 July

60 6.6. Outcome #4: Caregivers of Individuals with an FASD have Knowledge of and Access to Community Resources Caregivers of individuals diagnosed with and/or suspected of FASD have knowledge about FASD, about how to care for a person with FASD; and about available supports. Survey Findings Caregiver Receiving Support Services Survey Caregiver Receiving Support Question 6. The supports I received from the agency help me increase my understanding of FASD and how it affects my child / dependent. 7. The supports I received from the agency helped me feel more capable to parent/care for my child / dependent. 11. I have been given new information about what help is available for my child/dependent in the community while working with the agency. Completely Agree / Somewhat Agree Neither Agree nor Disagree Completely Disagree / Somewhat Disagree Don t Know / N/A Total Responses EFAN 37 (100%) Provincial 96% 3% 1% EFAN 36 (97%) 1 (3%) Provincial 92% 6% 3% EFAN 36 (97%) (3%) 37 Provincial 89% 3% 5% The majority of parents/caregivers receiving support from support services indicated that they have increased knowledge of how FASD affects their child and that they have received information that made them feel more capable of caring for their child. Results were similar to the provincial sample Outcome #5: Caregivers of Individuals with an FASD have access to respite care, peer and professional supports Caregivers of individuals diagnosed with and/or suspected of FASD have access to respite care, peer and professional supports. Survey Findings Caregiver Receiving Support Services Survey Caregiver Receiving Support Questions 12. I have received assistance in accessing help in the community for my child/dependent while working with the agency. Completely Agree / Somewhat Agree Neither Agree nor Disagree Completely Disagree / Somewhat Disagree Don t Know / N/A Total Responses EFAN 36 (97%) 1 (3%) Provincial 89% 6% 9% 1% My access to respite EFAN 27 (73%) 7 (19%) 1 (3%) 2 (5%) July

61 Caregiver Receiving Support Questions care for my child/dependent has increased since working with the agency. 14. My support network has increased since working with agency. Completely Agree / Somewhat Agree Neither Agree nor Disagree Completely Disagree / Somewhat Disagree Don t Know / N/A Total Responses Provincial 47% 20% 12% 21% 196 EFAN 35 (95%) 2 (5%) Provincial 91% 4% 3% 2% 195 The majority of parents/caregivers receiving support from support services indicated that they had received assistance accessing help in the community for their child, that their support network had increased, and that their access to respite care had increased since accessing support services. Results were similar to the provincial sample, except for a greater response to gaining new access to respite care Outcome #6: Caregivers of Individuals with an FASD Improved well- Being Caregivers of individuals diagnosed with and/or suspected of FASD experience decreased levels of stress and increased ability to cope. Survey Findings Caregiver Receiving Support Services Survey Caregiver Receiving Support Question 8. My family life has improved since receiving supports and services from the agency. 9. My level of stress has decreased since accessing services from the agency. 10. My sense of hope for my child/dependent with FASD has increased since accessing services from the agency. Completely Agree / Somewhat Agree Neither Agree nor Disagree Completely Disagree / Somewhat Disagree Don t Know / N/A Total Responses EFAN 34 (92%) 3 (8%) Provincial 89% 8% 3% 1% 196 EFAN 35 (95%) 2 (5%) Provincial 83% 11% 5% 2% 196 EFAN 36 (97%) 1 (3%) Provincial 92% 4% 3% 2% 196 The majority of parents/caregivers receiving support from support services indicated that their family life had improved. Half of the parents/caregivers indicated that their stress levels had decreased as a result of the support that they have received. The majority of parents/caregivers also reported an increased sense of hope for their child. Results were similar to consolidated provincial responses. 23 July

62 ORS Findings Support for caregivers Most Common Presenting issues of parents / caregivers Issues for parents / caregivers in Support Programs during their 1 st quarter in the program Presenting Issue Frequency of clients with the issue at admission (EFAN n= 64) Frequency of clients with the issue at admission (All Networks) (n = 335) Housing 3.90% 2.99% Addiction 1.30% 2.99% Health 1.30% 4.18% Mental Health 5.19% 12.24% Behavior 48.05% 25.97% Employment 2.60% 4.18% Education 2.60% 9.25% Legal 2.60% 2.09% Adaptive Skills / Life Skills 7.79% 14.93% Social skills 15.58% 8.96% Finance 1.30% 5.67% Custody of children 11.69% 5.67% Family violence 0% 1.19% Issues for caregivers were difficult to analyze because of inconsistency in ORS data entry: new ORS files created between July to September 2012 all had very high rates of behavior issues (in all 5 reporting quarters) and social skills (in the 4 th and 5 th reporting quarter) of 95 to 100%. For all other cohorts of clients, the rates of behavior and social skill issues were similar to the provincial rates. For other issues, rates were similar to provincial findings. 23 July

63 6.9. Outcome #7: Client Satisfaction, Involvement and Retention Client satisfaction, involvement and retention. Survey Findings Caregiver on Behalf of Client Support Services Survey Caregivers on Behalf of Client Question 4. Overall, I am happy with the support services my child / dependent and I have received so far. 5. People who work at the agency treat us with dignity and respect. 7. I am involved in planning the services my child / dependent receives. 13. My child / dependent received the kind of services we wanted. Completely Agree / Somewhat Agree Neither Agree nor Disagree Completely Disagree / Somewhat Disagree Don t Know / N/A Total Responses EFAN 30 (100%) Provincial 91% 2% 8% EFAN 30 (100%) Provincial 100% EFAN 27 (90%) 3 (10%) Provincial 86% 3% 11% 1% 183 EFAN 30 (100%) Provincial 86% 3% 9% 3% 183 Question 14: Have you ever attended any of the following FASD events or training opportunities? (Caregivers of clients receiving services) EFAN (n=30) All Networks (n=196) FASD Day event 8 (27%) 51(28%) FASD training through videoconference 0 (0%) 13 (7%) FASD training organized through their support agency 24 (80%) 55 (30%) FASD conference. 1 (3%) 22 (12%) Client Support Services Survey Client Survey Question 4. Are you happy with the services you received at our agency? 5. Do the people at our agency treat you with respect? 7. Are you involved in planning the services you receive from our agency? 14. Did you receive the kind of services you wanted from our agency? Yes No Maybe Don t Know / N/A Total Responses EFAN 92 (92%) 1 (%) 1 (1%) 5 (5%) 99 Provincial 93% 1% 2% 4% 348 EFAN 98 (99%) 1 (1%) Provincial 99% 0% 1% EFAN 85 (85%) 5 (5%) 3 (3%) 6 (6%) 99 Provincial 81% 3% 7% 9% 348 EFAN 85 (85%) 1 (1%) 7 (7%) 6 (6%) 99 Provincial 87% 1% 7% 6% July

64 Question 15: Have you ever attended any of the following FASD events or training opportunities? (Clients receiving support services) EFAN (n=99) All Networks (n=348) FASD Day event 6 (6%) 43 (12%) FASD training through videoconference 0 2 (1%) FASD training organized through their support agency 0 14 (4%) FASD conference. 2 (2%) 7 (2%) Caregiver Receiving Support Services Survey Caregiver Receiving Support Question 4. Overall, I am satisfied with the services I received from the agency. 5. Agency staff treat me with dignity and respect. Completely Agree / Somewhat Agree Neither Agree nor Disagree Completely Disagree / Somewhat Disagree Don t Know / N/A Total Responses EFAN 37 (100%) Provincial 98% 1% 1% EFAN 37 (100%) Provincial 96% 4% 0% Question 14: Have you ever attended any of the following FASD events or training opportunities? (Caregivers receiving support services) EFAN (n=37) All Networks (n=196) FASD Day event 7 (19%) 54 (28%) FASD training through videoconference 6 (16%) 25 (13%) FASD training organized through their support agency 19 (51%) 71 (36%) FASD conference. 5 (14%) 40 (20%) Caregiver Receiving Support Question 15. Were you involved in planning the services you received? 16. Did you get the kind of services you needed? Yes, Definitely / Yes, Generally Neither Agree nor Disagree No, Not Really / No, Not at All Don t Know / N/A Total Responses EFAN 34 (92%) 1 (3%) 2 (5%) - 37 Provincial 78% 3% 18% 1% 196 EFAN 36 (97%) 1 (3%) Provincial 95% 3% 3% All parents/caregivers and almost all clients who participated in support services were happy with the services they received and that they received the support that they wanted. All parents/caregivers and clients indicated that they were treated with dignity and respect. Over half of the caregivers had attended at least one training event, a higher percentage than in the provincial sample Other Outcomes #8: Staff Training, Supervision and Retention #9: Interdisciplinary team collaboration and informed approaches. 23 July

65 #10: Program Accessibility and Flexibility. Service Networks were not required to report specifically on Service Providers system outcomes Discussion Related to Support Services In , EFAN provided funding for support services to 8 different agencies in Edmonton, reaching out to 319 unique clients. As part of the Year- 7 Service Network evaluation process, EFAN s support service providers returned 99 client surveys, 30 surveys filled out by caregivers on behalf of services received by their dependents, and 37 surveys filled out by caregivers who received support services. ORS data confirmed that EFAN provided support services to clients of all age groups and to caregivers, that almost all clients received services for at least 2 reporting quarters, and the great majority for longer periods. More data over a longer period of time will confirm the good retention rates. Clients and caregivers reported that agencies helped them in a number of ways: taking care of themselves and their health, dealing with relationship issues, and support with school & employment were the most common areas of help. Clients also reported that agencies helped them deal with addiction and legal issues. EFAN s responses were similar to provincial responses. The majority of parents/caregivers and clients who participated in support services indicated that they had a better understanding of their strengths and weaknesses and knew what they needed to change about their behavior to take better care of themselves as a result of the support services. Presenting issues and outcomes for individuals receiving support services showed similar patterns for EFAN- funded support clients as provincial results. The three most frequent presenting issues for children receiving EFAN- funded support services were: behavior (43%), education (50%), and social skills (43%). The four most frequent presenting issues for youth years old receiving EFAN- funded support services were: social skills (66%), adaptive/life skills (50%), education (30%), and behavior (26%). The four most common presenting issues for young adults, ages 18 to 24, were: housing (45%), adaptive skills / live skills (53%), social skills (48%), and behavior (39%). The most common presenting issues for adults, ages 25+ were: mental health (66%), finances (52%), housing (50%), and employment (43%). There were inconsistencies in ORS data entry for the caregiver groups, and data could therefore not be analyzed. For most issues, there appeared to be an initial increase in the percentage of clients experiencing the issue in subsequent periods, with the beginning of a decrease in the 4 th or 5 th quarter. This pattern was also found in the provincial consolidated sample. The majority of parents/caregivers receiving support from support services indicated that they had increased knowledge of how FASD affects their child and that they had received 23 July

66 information that made them feel more capable of caring for their child. Results were similar to the provincial sample. The majority of parents/caregivers receiving support from support services indicated that they had received assistance accessing help in the community for their child, that their support network had increased, and that their access to respite care had increased since accessing support services. Results were similar to the provincial sample, except that EFAN s caregivers were more likely than all caregivers to have gained new access to respite care. Respite care is an area of service that is not systematically offered by all Networks. The majority of parents/caregivers receiving support from support services indicated that their family life had improved. Half of the parents/caregivers indicated that their stress levels had decreased as a result of the support that they have received. The majority of parents/caregivers also reported an increased sense of hope for their child. Results were similar to consolidated provincial responses. All parents/caregivers and almost all clients who participated in support services were happy with the services they received and that they received the support that they wanted. All parents/caregivers and clients indicated that they were treated with dignity and respect. Over half of the caregivers had attended at least one FASD training event, a higher percentage than in the provincial sample. Overall, EFAN s Support Services appear to meet the Support Services outcomes. There is no specific recommendation for EFAN s funded support services. 23 July

67 7. FASD Universal Awareness 7.1. Logic Model 23 July