New Directions, a psycho-educational program for adolescents with eating disorders: Stage two in a multi-step program evaluation

Transcription

1 New Directions, a psycho-educational program for adolescents with eating disorders: Stage two in a multi-step program evaluation Authors: Mary Kaye Lucier MSW, RSW, Executive Director, Bulimia Anorexia Nervosa Association (BANA) Rosanne Menna PhD, C Psych., University of Windsor; Department of Psychology Date: September 30, 2008

2 Table of Contents Executive Summary 3 Project Summary 7 Methodology 8 Results 11 Conclusion & Recommendations/Next Steps 17 Knowledge Exchange Plan 19 Schedule B 21 Required Signatures 23 2

3 Executive Summary a) Brief description of the program/project to be evaluated: New Directions is a ten week-1.5 hour per session, psycho-educational group invention program for adolescents aged years, who have been diagnosed with an eating disorder. The goal of the New Directions program is to increase insight and awareness of four related factors contributing to eating disorders: (i) eating disorder symptoms; (ii) factors contributing to the development of eating disorders; (iii) the physical and mental effects of eating disorders; and (iv) develop a plan for recovery. The program was designed to interrupt symptoms related to eating disorders such as extreme dieting, body dissatisfaction and unhealthy weight control methods. Referral into New Directions is based on the results of a specialized structured eating disorder diagnostic assessment. Following the specialized assessment, each referral is presented at the weekly rounds, for a diagnosis. The diagnosis is based on the DSM IV criteria for eating disorders. Each person is provided with a treatment plan. Those with the diagnosis of an eating disorder are invited to attend the New Directions group. Following the completion of New Directions, the participants are reassessed and presented for a re-diagnosis and re-evaluation of the treatment plan. b) Evaluation questions and purpose for the evaluation: The evaluation questions were: (1)What are the changes in eating disorder symptoms, psychological functioning and eating behaviour at least one year following the completion of New Directions? (2) What are the factors related to recidivism and variables which may lead to a greater understanding of recovery, relapse, and re-occurrence? (3) What are the steps in planning for an expanded methodology to compare outcomes of the New Directions pre and post intervention? The purpose for the evaluation was to: (1) Track client data over the course of the New Directions program through pre, post and follow-up assessments; (2) Enhance BANA s capacity to conduct outcome research; (3) Develop a plan for a more complex evaluation of client outcomes of New Directions program; (4) Describe the degree of satisfaction with the program; (5) Disseminate the project findings. c) Brief Description of the Methods Used (e.g. data collection using psychometric tools, surveys, interviews, etc.): The specialized eating disorder assessment consists of a diagnostic interview, followed by the administration of psychometric tests that measure eating disorder psychopathology (i.e. symptoms, body dissatisfaction, and size acceptance), related psychological functioning (i.e., self-esteem, depression) and demographic information. The diagnosis is based on the DSM IV criteria for eating disorders, and is under the 3

4 supervision of either a clinical psychologist or physician. Three assessments were conducted, each comprising of the above measures. Assessments were completed before the groups began (at initial intake; pre-test), following completion of the New Directions program (post-test), and at follow-up March 08 to June 08 (post-post treatment). Thirty six clients who completed the program between were sent an invitation letter to participate in the follow-up program evaluation assessment. Nine adolescents agreed to participate in the follow-up study. Six completed the followup assessment. The diagnostic interview and questionaires were administeted on the volunteer participants and were subject to agency policy and procedures. The purpose for the program evaluation was explained at the time of each visit and was presented on the agency "Consent to Treatment and Research Form". Participants had the option to withdraw their consent at any time. All of the diagnostic assessments were conducted by trained clinicians, in the field of eating disorders, and were employees of BANA. Policies and procedures exist within the agency that reflect the most current and up to date legislation on the collection, retrieval and storage of personal and health information. The data entry personnel were required to sign confidentiality statements, prior to data entry. Steps Involved in the Research Project: 1. A letter of introduction was sent to participants who had completed the New Directions program; 2. Participants were invited to participate in the follow-up study; 3. Data was collected from a specialized diagnostic assessment interview of eating disorder behaviours, and self report questionnaires that measure attitudes, feelings, thoughts and behaviours related to the symptomatology of eating disorders, and related psychopathology; 4. Participation in agency research component for the purposes of data collection and data analysis, is voluntary; 5. Participants were reminded that participation in no way affected their ability to participate in the programs offered at the agency; d) Summary of Main Findings Demographic Description: Nine participants agreed to participate in the follow-up testing, however 6 participants completed the assessment protocol. The participants were female, with fifty percent from divorced or single parent families. The average age for the participants was (SD= 1.9 years) years. All were Caucasian, and middle class. All spoke English. BMI scores increased from intake to post and follow-up assessment. Changes in Psychological Functioning: At the follow-up assessment participants reported less suicidality. Current and past thoughts of suicide and suicide attempts dropped from intake to post and follow-up assessment. 4

5 At intake most of the participants of New Directions reported no form of drug or alcohol use, one third reported using alcohol or cannabis socially. At follow-up assessment all participants reported alcohol use and one third reported cannabis use. All participants reported using alcohol socially. Participants reported no history of abuse at intake, post or follow-up assessment. There was a general indication of change in self-harm behaviours reported. At the initial intake, approximately two thirds (67%) of the participants reported self-harm behaviours, and the remaining one third (33%) of the participants reported no self-harming behaviours. At post and follow-up assessment, approximately 83% of the participants reported no self-harm behaviours, and 17% reported some form of self-harm behaviours. At intake, post and follow-up assessment participants endorsed cutting as the self-harm behaviour. Changes in Eating Disorder Symptoms: Overall, the participants who completed New Directions reported a general reduction in eating disorder behaviours. At follow-up, participants reported fewer binge episodes, and vomit days. Participants reported less dieting and fasting and fewer attempts to lose weight after the program. Changes in Diagnosis From Post Follow-up assessment: For the purposes of this study, there were three possible diagnostic categories: Anorexia Nervosa (AN), those with a Body Mass Index (BMI) of less than 85% of BMI 20, severe dieting behaviours and body image disturbance; Bulimia Nervosa (BN), those who exhibited binge and/or purge behaviours and body image disturbance; and those with not otherwise specified eating disorders (EDNOS). At initial intake approximately 17% were AN, 50% were BN, and 17% were EDNOS). At post assessment, approximately 17% were AN-partial remission, 70% with BN, and 17% EDNOS. At follow-up one third were BN-partial remission, 17% BN, 33% with EDNOS and 17% could not be classified due to lack of information. At follow-up assessment approximately 17 % of the categories of diagnoses did not change. There was some movement towards partial remission for approximately one third of the participants. Level of satisfaction with the program: At the follow up interview, the following chart represents the average satisfaction rating with the program: 5

6 The following chart represents the average rating for quality of the services received: The following chart indicates the average rating of the group content: 6

7 7

8 Summary of Implications of Findings The recovery rate from eating disorder treatment programs is based on the criteria found in the DSM IV. Generally around 1/3 usually complete their prescribed treatment. Those who completed more than five New Directions sessions reported fewer suicidal thoughts, plans and attempts, and self-harm behaviours at follow-up assessment assessment. BMI scores increased from post to follow-up assessment and a general reduction in eating disorder behaviours were notes. The findings imply some positive change in behaviours at follow-up assessment. 8

9 Summary of Recommendations Presentations and workshops about the program and research findings will be developed for presentation at agencies across Ontario and Canada. An executive summary of recommendations to the Ontario Network of Eating Disorder Service Providers regarding treatment outcomes and program evaluation will be prepared. Study findings will also be available on the BANA website, University of Windsor REB website, and Provincial Centre of Excellence for Child and Youth Mental Health at CHEO website. Project Summary a) Purpose of evaluation and questions asked: The literature on effective treatment for eating disorders indicates that psycho-education in the treatment of eating disorders is the first step among a variety of integrated methods of treatment. A psycho-education approach is viewed as a less intrusive method, and is usually conducted in an outpatient setting. Subsequent treatment following psycho-education may be recommended pending the outcome of the completion of the program. The overall goal of this project was to evaluate the sustained effectiveness of the New Directions program following the completion of the program. The objective of this evaluation was to analyze data collected on participants of the New Directions program from September 1, March 31, Expected outcomes of the program were sustained increased self-esteem, sustained reduction in eating disorder symptoms, sustained improvements in depression and sustained normalized eating behaviour. Ultimately, the goal is to track clients recover rates. The purpose for the evaluation was to: To create a snapshot of the participants in the New Directions program at post - post treatment, by analysing data collected on participants; To enhance the agency s capacity for more complex research on client outcomes; To disseminate project findings; To create an evidence-based program for adolescents with an eating disorder that is accessible through a training manual suitable for delivery in other agencies. To assess client satisfaction with agency programs. Research questions: (1)What are the changes in eating disorder symptoms, psychological functioning and eating behaviour at the follow-up assessment after participating in the New Directions program? (2) What are the factors related to recidivism and variables which may lead to a greater understanding of recovery, relapse, and re-occurrence? (3) What is the plan for an expanded methodology to evaluate the outcomes of New Direction program? 9

10 b) Description of project to be evaluated New Directions is a ten week-1.5 hour per session, psycho-education group invention program for adolescents aged years, who have been diagnosed with an eating disorder. The goal of the New Directions program is to increase insight and awareness of four related factors contributing to eating disorders: (i) eating disorder symptoms; (ii) factors contributing to the development of eating disorders; (iii) the physical and mental effects of eating disorders; and (iv) developing a plan for recovery. The program was designed to interrupt symptoms related to eating disorders such as extreme dieting, body dissatisfaction and unhealthy weight control methods c) Identification of the target population for the program/relevant stakeholders Studies have shown that adolescence represents a period of risk for the development of eating disorders. Eating disorders affect 5% of the female population between the ages of 12 and 24 years. The mortality rate is the highest of all the psychiatric disorders, and is estimated at 10%. Early intervention strategies in a group setting have been suggested to be useful in shortening the course of the eating disorders and reducing their severity. BANA offers treatment and prevention services for eating disorders in Chatham and Windsor. BANA s mission is to provide specialized treatment, education, and support services for individuals affected by eating disorders. The agency employs a multi-disciplinary team approach consisting of social work, psychology, medical, and dietitian services. New Directions is the adolescent program. d) Review of related research The research on outcome for eating disorder treatment lists depression, obsessivecompulsive disorder, personality disorder, suicide attempts and completion, sexual abuse and substance abuse as predictors of treatment failure, in persons with eating disorders especially for persons who are in the restricting type of eating disorders, i.e. AN, EDNOS-ANR. (Berkman, Lohr & Bulik, 2007). Most of the treatment outcome studies for eating disorders relate to cognitive behavioural therapy and family therapy for adolescents (Bulik, Berkman, Brownley, Sedway, & Lohr, 2007; Brownley, Berkman, Sedway, Lohr, & Bulik,2007).The primary focus of the research treatment outcome studies, generally centred around AN, was restoration of weight. The secondary focus of the research generally was focussed on psychological functioning. Most of the studies found that behaviour based interventions were best in weight restoration. Drug therapies were found to be the least effective form of treatment (Bulik et al.,2007).no known research has been conducted on the outcome of psycho-educational treatment for adolescents (Bulik et al). Methodology a) Design The specialized eating disorder diagnostic assessment consists of a diagnostic interview, followed by the administration of psychometric tests that measure both eating disorder psychopathology, and related psychological functioning. Following the 10

11 specialized structured eating disorder diagnostic assessment, a set of satisfaction surveys were administered to each study participant. All of the participants were presented at the clinical rounds for a diagnosis. The diagnosis is based on the DSM IV criteria for eating disorders, and is under the supervision of either the clinical psychologist for persons aged 16 and older, or the physician for persons under the age of 16 years. The 36 adolescents who entered and completed at least one session of the New Directions group were invited to participate in a follow-up assessment interview. Of those 36 adolescents, who were referred into the New Directions program, nine agreed to complete a follow-up assessment, which included a standardized, structured eating disorder diagnostic interview, and 3 questionnaires. b) Data Collection Demographic Questionnaire. Adolescents were asked to provide their age, height, weight, language spoken, ethnicity, and parent s occupation. Weight and height were used to determine body mass index (BMI; calculated using the following formula: weight (kg)/height (m) 2 ). Diagnostic Interview. Adolescents were interviewed about their typical day of eating, binge, vomiting, fasting episodes, laxative, diuretics, and diet pill use, exercise, body image, body satisfaction, abuse history, alcohol and drug use, suicidal thoughts and behaviour and self-harm behaviour. Eating Disorders Inventory-2 (EDI: Garner, 1991). The EDI-2 is a 91 item self-report measure of eating disorder symptoms. In completing the measure participants answer a number of questions about their shape, weight, and eating on a 6-point scale ranging from never to always. The EDI Drive for Thinness, Body Dissatisfaction, and Bulimia subscales were used in this project to describe the severity of participants eating disorder pathology. Extensive psychometric support for this instrument and norms for adolescents are available in the treatment manual (Garner, 1991). Child Depression Inventory (CDI; Kovacs, 1983). The CDI is a 27 item self-report measure of depression. Each item consists of three statements graded in order of increasing severity from 0 to 2. For each item participants respond by marking the sentence that best describes them during the past 2 weeks. The measure has high levels of internal consistency, re-test reliability and convergent validity. Rosenberg Self-Esteem Questionnaire (RSE: Rosenberg; 1979). This measure consists of 10-items addressing global positive or negative attitudes toward the self. Items are of the following type; I feel that I have a number of good qualities; All in all, I am inclined to feel that I am a failure. Participants rated on a five-point scale (1=strongly disagree, 4=strongly agree) the extent to which each statement is felt or experienced. A total is computed by summing the ratings on all items. The possible scores on selfesteem range from 1 (low self-esteem) to 40 (high self-esteem). The instrument has been used widely and has good reliability and validity. The forms and questionaires were subject to agency policy and procedures. The data collected for follow-up study was explained at the time of the follow-up visit and was presented on the agency "Consent to Treatment and Research Form". 11

12 Opportunities for debriefing occured at the time the consent was given. At the time consent was given, participants were provided with copies of the consent form, and the opportunity to ask questions about the research component of the agency program. At the beginning of the specialized diagnositc assessment interview, each participant was informed that the data collected for their file would be used for the purposes of program evaluation. Each particpant was also informed that they have a right to withdraw from the project at any time. Each participant was informed that the data collected for the purposes of program evaluation would be kept confidential, and would not be used for other purposes without their prior consent. Participants participation in the agency research component for the purposes of data collection and data analysis, is voluntary. Participants were reminded that participation in no way affects their ability to participate in the programminig offered at the agency. Each participant who agreed to participate in the program evaluation, was provided with a research data entry number, and thereby, the identifying information in the database is unknown to the researcher. c) Sources of information and data report Adolescents completed self-report questionnaires and a structured interview. The questionnaires and diagnostic interviews were administered by trained qualified clinicians in the field of eating disorders. Each therapist has access to the hard copy file of the adolescents interviewed. Each clinician is required to sign a confidentiality agreement, as a condition of employment. Agency policies and procedures are in place, which reflect the Personal Health Information Privacy and Protection Act (2005) to safeguard the collection, storage, and disposal of the data that is being collected. Undergraduate and graduate students in social work and psychology programs entered data. The data entry personnel were required to sign a confidentiality agreement and an offer to hire stating that they will abide by agency policy and procedures relating to data collection and storage, as a conditon of employment and prior to entering data. All hard copy data is stored in a locked file cabinet. All data is stored on the agency data base, protected by password. The principal investigator has access to the hard copy and the research database for the purposes of checking the data for accuracy. The coinvestigator also has access to the hard copy of the data collected and the research database for the purposes of checking accuracy. e) Evaluation limitations There were two limitations to the study: i) small sample size, and ii) loss of contact with the majority of the participants of New Directions. The loss of contact resulted because they either had moved away, or had changed their phone numbers. Results Research question #1: 12

13 (1) What are the changes in eating disorder symptoms, psychological functioning and eating behaviour at follow-up assessment, after participating in the New Directions program? All participants were female. Approximately fifty percent were from divorced or single parent families. At follow-up assessment the average age for the adolescents was years (SD= 1.9 years). All were Caucasian and tended to be middle class. All participants spoke English. Participants of the follow-up study were adolescent clients who presented for the initial assessment at BANA with typical patterns of eating and compensatory behaviours to persons with eating disorders. At intake, the participants mean height and weight was 5 3 (SD=.42), and 125 lb. (SD=33.16). The mean body mass index (BMI) was (SD=4.25). At the post assessment the participants mean height and weight was 5 3 (SD=.42), and 130lb. (SD=39.68) and the mean BMI was (SD=4.8). At follow-up assessment participants mean height and weight was 150 lb. (SD=50.94) and 5 3 (SD=.42). The mean BDI was (SD=5.88). BMI increased from intake to post and follow-up assessment. There were four main classifications of diagnoses: Anorexia Nervosa (AN), Bulimia Nervosa (BN), Binge Eating Disorder (BED), and Eating Disorder Not otherwise Specified (EDNOS). At the initial intake, 1 (16.67%) participant was diagnosed with AN, 3 (50%) with BN, 1 (16.67) with EDNOS, and 1(16.67%) with no eating disorder or body image concerns. At the post assessment, 1 (16.67%) participant was diagnosed with AN partial remission, 4 (66.67%) with BN, 1 (16.67) with EDNOS. At follow-up, 2 (33.33%) were diagnosed with BN-partial remission, 1 (16.67) with BN, and 2 (33.3%) with EDNOS. One participant was unclassified, because the participant did not complete the interview. At follow-up assessment approximately 17 % of the categories of diagnoses did not change; however there was some movement towards partial remission for approximately one third of the participants. Table 1: Body Mass Index of participants at pre, post and follow-up testing BMI at intake BMI at post assessment 6 BMI at follow-up 6 N Mean Std. Range Dev Suicide: History of suicide attempts, current and past thoughts or plans for suicide were reported for many (83.3%) of the participants at intake. Tables 2-4 illustrate a description of the suicidality for participants of New Directions at intake, post and followup assessment. Current and past thoughts of suicide and suicide attempts dropped from intake to post and follow-up assessment. 13

14 Table 2: Percent and frequency of suicidality at intake No Suicide History Current Suicidal Thoughts Current Suicidal Plan Past Suicide Thoughts Past Suicide Attempt 16.7 (1) % 33.3% (2) 0.0(0) 33.3% (2) 16.7%(1) Table 3: Percent and frequency of suicidality at post assessment No Suicide History Current Suicidal Thoughts Current Suicidal Plan Past Suicide Thoughts Past Suicide Attempt 66.7 % (4) 0.0% (0) 0.0% (0) 33.3% (2) 0.0% (0) Table 4: Percent and frequency of suicidality at follow-up assessment No Suicide History Current Suicidal Thoughts Current Suicidal Plan Past Suicide Thoughts Past Suicide Attempt 83.3% (5) 16.7% (1) 0.0% (0) 0.0% (0) 0.0% (0) Alcohol/Drug Use: At intake, all participants indicated no form of drug use, (100%), and few indicated alcohol use (16.7%). Those who reported using alcohol indicated being a social drinker. At post assessment, approximately 17% of the participants indicated drug use; cannabis use was reported. Approximately one third (33.3%) of the participants indicated alcohol use. Of those who reported using alcohol, approximately one third (33.3%) indicated being a social drinker. At follow-up assessment all participants indicated alcohol use (100%) and one third (33.3%) indicated drug use. All participants reported being social drinkers and those who reported drug use indicated cannabis. Self - Harm: At the initial intake, approximately 67% of the participants reported selfharm behaviours, and the remaining one third of the participants (33%) reported no selfharming behaviours. Most participants reported that the New Directions group was the first attempt at any form of psychiatric treatment. At the post assessment, approximately 83% of the participants reported no self-harm behaviours, and 17% reported some form of self-harm behaviours. At follow-up assessment 83% percent reported no self-harm behaviours, and 17% reported some form of self-harm behaviour. At intake and post and follow-up assessment, participants reported cutting as the self-harm behaviour. Abuse: At the initial intake, post and follow-up assessment all participants reported no form of physical, sexual or emotional abuse (100%). Research question #2: What are the factors related to recidivism and variables which may lead to a greater understanding of recovery, relapse, and re-occurrence? 14

15 Factors Related to recidivism, recovery, relapse, and re-occurrence: In the previous PEG evaluation paired-sample t tests were conducted to evaluate the effects of the group on eating disorder symptoms. Overall, the findings were not significant however pre and post mean scores indicated a general reduction in eating disorder behaviours. Table 5 shows the means scores at post and follow-up assessment of the New Directions program. Statistical analyses were not conducted due to the small sample size. As shown in Table 5, overall, follow-up means indicate a reduction in eating disorders since post assessment. Table 5: Change in eating disorder symptoms: post to follow-up mean scores for New Directions Variable/Behaviour Post Follow-up Mean SD n Mean SD n Dieting days in past 28 days Dieting days in past 3 months Binge episodes in past 28 days Binge episodes in past 3 months Binge days in past 28 days Binge days in past 3 months Vomit days in past 28 days Vomit days in past 3 months Fasting days in past 28 days Fasting days in past 3 months Laxative days in past 28 days Laxative days in past

16 Variable/Behaviour Post Follow-up 3 months Exercise days in past 28 days Exercise days in past 3 months Changes in Diagnosis: Table 6 reports the frequencies and percentage for categories of diagnosis post and follow-up program. Table 6: Change in client diagnoses from post to follow-up New Directions Presenting Problem N % of Total No change EDNOS AR to EDNOS-BN BED to BN-NP BN-P to BN-P partial recovery AN-R partial recovery to unclassified As reported in the previous PEG evaluation significant differences between the pre and post assessment scores were not found, however the sample size did limit data analyses and interpretation (Lucier & Menna, 2007). From post to follow-up assessment, there was no change in approximately 17% of the participants; partial recovery was found in approximately one third of the participants. At post-testing self-esteem mean scores indicated a small increase from pre to post New Directions program. Eating disorder symptoms and depression scores showed little change from pre to post intervention. For the follow-up assessment we are not able to report means and standard deviations for eating disorders, self-esteem, and depression scores because of missing data. Only one participant completed the depression inventory, and two participants completed the eating disorders and self-esteem questionnaires. Research question # 3: What is the plan for an expanded methodology to compare the outcomes of New Direction Psycho-educational program pre and post intervention? 1. Given our findings to date, we have reviewed the New Directions manual and have gathered questionnaires that better assess material delivered in the program. 16

17 2. We have reviewed our current assessment battery and have revised the battery to include the self-perception profile (SPPA; Harter 1988), eating attitudes (EAT;Garner & Garfinkel, 1979) and eating (EDI-2; Garner, 1991) disorder inventories, a depression inventory (SMFQ; Angold, Cocstello, Messer, & Pickles, 1995), stages for change questionnaire (SCQ; McConnaughy, Prochaska, & Velicer, 1983), and a satisfaction questionnaire, in addition to the structured interview. 3. We are in the process of developing a client satisfaction questionnaire. 4. Submit application to the Research Ethics Board, University of Windsor to evaluate the New Directions program pre and post intervention. 5. Conduct assessments on 16 adolescents using the revised battery, and invite them to participate in the research study; 6. Conduct two, 10 week psycho-education groups, 8 members per group; 7. Conduct post testing one to two weeks after New Directions program 8. Three months after the completion of the groups, invite the members to participate in follow-up assessment; 9. Conduct follow-up assessments; 10. Analyse results from data collected; 11. Review findings, manual and test battery; 12. Design a new evaluation study which will include a control group. Conclusion & Recommendations/Next Steps a) Discussion and interpretation of findings The expected outcomes of the program were sustained levels of increased self-esteem, sustained reduction in eating disorder symptoms, depression, and a pattern of normalized eating behaviour for at least three consecutive months. Interpretation of the findings is limited by a small sample size. Overall, the participants who completed New Directions reported a general reduction in eating disorder symptoms and self harm behaviours. In addition, the majority of the participants in the post - post follow-up assessment indicated that the group environment was often warm and supportive; always felt respected, and they felt hopeful about their chances for recovery. One half of the participants indicated that they received the kind of help they needed, and one half indicated they could deal more effectively with their problems as a result of the programme. All of the participants indicated that they either improved a little or a great deal, however, 50% indicated they would refer others to the programme with reservations, or if no other treatment was available. The majority of the participants indicated that too much time was spent completing questionnaires, and exercises in group; and the time in group listening to others was extremely helpful. b) Conclusion reflects the findings Each variable collected at pre and post treatment was collected at follow-up treatment, utilizing a standardized structured interview and administration of psychometric tests. The total number of adolescent clients that were contacted to participate in the follow-up study was thirty-six. Nine clients indicated interest in participating, however only six clients completed the follow-up assessment. 17

18 c) Recommendations It was suggested from the first step in the program evaluation, that the completion rate for New Directions was made more difficult due to the increased level of psychological distress experienced by some of the participants as evidenced by the self harm and suicidalality self reports (Shapiro, Berkman, et al., 2007). Readiness to change and posttraumatic stress could be a factor in the outcome analysis. The next steps in the multi stage program evaluation project would be to review the manual and assessment protocol, include a follow-up component to the assessment and measure outcomes of the New Directions group measuring outcomes using the new assessment protocol and eventually a control group The agency and the University of Windsor, Psychology Department have agreed to partner again to continue to pursue funding from Provincial Centre of Excellence for Child and Youth Mental Health at CHEO towards that end. An important recommendation would be to build in a schedule for follow-up assessments following the completion or withdrawal from the adolescent treatment program at the agency. Another recommendation would be to add a family therapy component to the adolescent program, given the nature of the recovery rate. A final recommendation would be to re-structuring the administration of the questionnaires. Because the participants in the New Directions indicated that the number of questionnaires were too many, and not at all helpful, they could be administered within the first group session, with time allotted for discussing their feelings surrounding the questionnaires. d) Lessons learned from evaluation activities The agency was well equipped to collect, enter and analyze the data. Not enough time was allotted to analyze the data, and write the report. Data entry needs to be conducted by experienced and trained staff. A research assistant is needed to coordinate and oversee the projects initiated. The study provided an opportunity for the agency to assess follow-up, review the New Directions manual, make changes to the assessment battery and initiate a new evaluation of the program. e) Impact of having done this evaluation on clients served, staff, and the organization as a whole. Because the project was conducting follow-up assessments, analyzing the impact on the clients provided new insights into the relapse, recovery and re-occurrence rates. The impact on staff was important because the project allowed the opportunity to develop and appreciate the unique set of skills suitable for conducting research. The staff developed an appreciation for research because the results of these consecutive program evaluation grants have produced results that suggest and inform program changes and enhancements. The organization has also increased its capacity for 18

19 conducting more rigorous research. The organization has increased its credibility within the Ontario Outreach Network for Eating Disorders. f) Next steps This proposal is the second step in a multi-step approach to program evaluation. Step 1 provided a snapshot of the New Directions program by identifying demographic information, examined clients level of functioning pre and post treatment group and generated an understanding of the drop out rate for New Directions. The second step attempted to examine the long-term effectiveness of the New Directions psychoeducational group, by conducting follow-up assessments after the intervention was completed. The third step in the program evaluation approach will be to expand on the methodology by revising the test battery, and design of the evaluation and eventually including a waitlist control group, which will be compared to the treatment group pre and post evaluation and a three month follow-up assessment. An additional step would be to research the availability for training in family therapy to be added later as a component to the adolescent treatment program. These program evaluation grants have provided the agency with the opportunity to study the data collected from the clients, and to analyse their results. For example, from the last PEG, the agency presented a poster presentation to the Canadian Psychological Association (CPA) in June 2008, on the relationship between family functioning and attachment style for adolescents diagnosed with an eating disorder. Presentations and a possible publication about the program and research findings will be developed. Study findings will also be available on the BANA website, University of Windsor REB website, and Provincial Centre of Excellence for Child and Youth Mental Health CHEO website. We will also deliver presentations at professional meetings and conventions, and publish findings in newsletters, and professional journals. Knowledge Exchange Plan a. Overview of knowledge exchange activities relating to this program/project To post the results of the project on BANA website To publish the results of the project in the BANA newsletter by December 30, 2008; To submit an article based on our CPA presentation for publication in a professional journal; Post the results of the project on the Provincial Centre of Excellence for Child and Youth Mental Health at CHEO; b. Knowledge exchange activities accomplished to date: Report to University of Windsor Research Ethics Board, June 30, Report to BANA board of directors, June 18, 2008; c. Further plans regarding knowledge exchange activities 19

20 To generate an article based on CPA findings suitable for publication in a professional journal. To identify the areas for more research on program evaluation. To make recommendations to the Ontario Network of Eating Disorder Service Providers regarding treatment outcomes and program evaluation. To publish the results on BANA website, and Provincial Centre of Excellence for Child and Youth Mental Health at CHEO website. References 1. Berkman N, Lohr K, Bulik C. Outcomes of eating disorders: A systematic review of the literature. International Journal of Eating Disorders 2007; 40: Brownley J, Berkman N, Sedway J, Lohr K, Bulik C. Binge eating disorder treatment: A systematic review of randomized controlled trials. International Journal of Eating Disorders 2007; 40: Bulik C, Berkman N, Brownley J, Sedway J, Lohr K. Anorexia Nervosa Treatment: A systematic review of randomized controlled trials. International Journal of Eating Disorders 2007; 40: Shapiro J, Berkman N, Brownley J, Sedway J, Lohr K, Bulik C. Bulimia Nervosa Treatment: A systematic review of randomized controlled trials. International Journal of Eating Disorders 2007; 40:

21 Accounting Summary of Expenses Eligible Budget Items Approved Total Cost Cost per Actual Cost per Item ($) ($) Item ($) Personnel Costs January to June Salaries 10, Consultation Costs Computer Costs (hardware and software; Max. $3,000) SPSS licence renewal; Hard Drive Training Tools/Questionnaires $500 $500 $500 Copying/copyright $500 $500 $500 Administrative Costs (details $1000 required; Max. $10%) Canada Post $300 $300 $300 Copying $350 $350 $350 Printing $350 $350 $350 Office Supplies Knowledge Exchange $1000 $1000 $1000 Activities Conference Fee $300 $300 $300 Travel Travel (for data collection only) Total Cost of All Expenses (Max. $15,000) $15000 $15000 $

22 22