Assessing the Impact of Group-based Cognitive Behaviour Therapy with Children and Adolescents Presenting with Anxiety or Depression

Transcription

1 PEG-171 Assessing the Impact of Group-based Cognitive Behaviour Therapy with Children and Adolescents Presenting with Anxiety or Depression Joan Nandlal, PhD & John Robinson, MA Bennett Research and Evaluation Services March 2007

2 Executive Summary Project Description The Community Mental Health Clinic (CMHC) has been implementing a groupbased Cognitive Behaviour Therapy Program for children and adolescents presenting with depression and/or anxiety since The Program involves 8 to 10 group-based sessions and is based on the Cool Kids Program, a wellestablished cognitive behavior therapy program. Evaluation Questions and Purpose of Evaluation The aim of this project was to determine whether or not a group-based Cognitive Behaviour Therapy Program intervention with children and adolescents presenting with depression or anxiety had a positive impact on them as evidenced by improved scores on measures of depression and anxiety. Methodology Pre-intervention and post-intervention scores on two measures the Children s Depression Inventory and the Multidimensional Anxiety Scale for Children were analyzed using repeated measures ANOVAs with sex and setting as between subjects factors and pre and post-program test scores as a within subjects factor within both ANOVAs. Separate male and female paired samples (pre and postprogram) t-tests on sub-scale scores of both measures were conducted to examine whether there were statistically significant changes in pre and post-program subscale scores. Clients were classified pre and post-program according to test author categories in order to gain some insight respecting the clinical significance of changes, and Reliable Change (RC) index scores were calculated to identify clients whose change in their total scores on one or both measures were large enough that the change likely represented real change. Main Findings Overall, program participants scored significantly lower on both the measure of depression and the measure of anxiety after participating in the Cognitive Behaviour Therapy Program. Girls scores decreased significantly on four of the five subscales on the measure of depression, including the negative self-esteem subscale. Pre-intervention scores on both measures for a large percentage of boys fell within the normal range, consequently, there was minimal change observed in their scores. Of the 23 boys and girls demonstrating reliable change, 20 were improved, 2 were in the opposite direction of hoped for change, and 1 showed improvement on the MASC measure but a higher CDI score. Of the 14 clients showing reliable improvement on the CDI measure, 5 remained above average or higher at post-program and 9 were slightly above average or lower. Of the 12 clients showing reliable improvement on the MASC measure, 5 remained above average or higher and 7 were slightly above average or lower. i

3 Implications of Findings Findings suggest that, in general, the Cognitive Behaviour Therapy Program had a positive impact, particularly for girls. To the extent that the Program contributes to promoting improved self-esteem amongst girls this is in and of itself an important contribution because self-esteem is a protective factor associated with better mental and physical health outcomes. The interpretation of study findings is limited due to the study design, the absence of post-intervention data for many clients, and the lack of additional data sources. It is likely that had other sources of data been systematically gathered and analyzed (e.g., self report ratings on other measures, parental ratings, clinician observations) that a better understanding of the ways in which the intervention impacted on clients would have been gleaned. Recommendations o o o o o o Review client selection procedures Develop a program logic model Use a mixed-methods, multiple data source approach Measure additional variables Obtain follow up data from Program participants Engage dedicated evaluation expertise early and throughout ii

4 Table of Contents Page Background Program Description... 1 Rationale for and Purpose of Evaluation....2 Evaluation Questions Literature Review....2 Methodology....6 Findings....7 Implications of Findings Recommendations.. 11 Knowledge Exchange Plan References..13 iii

5 1.0 Background In response to client needs and in keeping with best practices, the Community Mental Health Clinic (CMHC) began implementing a group-based Cognitive Behaviour Therapy Program (herein after referred to as the Program or the CBTP ) for children and adolescents presenting with depression and/or anxiety in The Program is based on the Cool Kids Program, a well-established cognitive behaviour therapy (CBT) program aimed at assisting children and their parents to develop better ways of coping with their worries and fears that contribute to anxiety and depression. The Cool Kids Program is used by many mental health clinics across Canada. Prior to implementation, CMHC staff received two days of training on the Cool Kids Program. Dr. Krystella Calvert, a pediatrician from Hamilton who had been trained at McMaster University in using the program provided the training to CMHC staff. Entry to the Program is on a referral basis and children are divided into three age groups (i.e., 7 to 10, 11 to 13, and 14 to 18 year olds). Since initiating the Program, the CMHC has been asking children and youth to complete self-report measures of anxiety and depression both before and after participating in it. More specifically, the Children s Depression Inventory (CDI; Kovacs, 1992) is completed to assess level of depressive symptomology, and the Multidimensional Anxiety Scale for Children (MASC; March, 1997) is used to determine the degree of anxiety. In 2006 the CMHC obtained funding from the Provincial Centre of Excellence for Child and Youth Mental Health at CHEO to undertake a formative review of the Program and procured the services of Bennett Research and Evaluation Services to undertake the work in fulfillment of the Clinic s obligation to CHEO. This report summarizes the findings of the formative review undertaken by Bennett Research and Evaluation Services that was based on an analysis of pre and post-intervention scores on the CDI and the MASC. 2.0 Program Description The Program aims to enhance the wellbeing of children and youth who are experiencing either mild depression and/or anxiety through promoting awareness and understanding of maladaptive cognitions and coping strategies and the development of alternate adaptive thoughts, beliefs, and behaviours. The Program consists of eight to ten weekly group-based sessions that are co-led by either two clinicians, or a clinician and a child and youth worker. Sessions are conducted at either a CMHC location or a high-school location. Sessions include a focus on: 1) recognition of symptoms of anxiety or depression, 2) clarifying anxious or depressive cognitions, 3) developing a plan for coping, that is, actively working toward changing cognitions and behaviours, and 4) enhancing capacity to assess the efficacy of the coping plan as it is implemented. 1

6 3.0 Rationale for and Purpose of Evaluation A formative review of the Program was desired for three reasons. First, CMHC staff wanted to know whether or not this intervention was having a positive impact for children and adolescents presenting with anxiety or depression. Second, the CMHC sought to contribute to the body of research relating to the efficacy of group-based CBT interventions for children and adolescents presenting with anxiety or depression. Third, the CMHC wanted to use the current review in a formative way, that is, to aid in planning for future comprehensive evaluations of the Program. As a result, the information gleaned from this process could also inform the CMHC s Child and Youth Team s practices with regard to data monitoring and collection for evaluation purposes more generally. 4.0 Evaluation Questions The aim of this project was to determine whether or not a group-based CBT intervention with children and adolescents presenting with depression or anxiety had a positive impact on them as evidenced by improved scores on measures of depression and anxiety. 5.0 Literature Review Cognitive behaviour therapy is a commonly used and studied treatment approach for addressing psychological problems in children and youth (Prins & Ollendick, 2003). The literature review was limited in scope to published research or literature reviews and critiques related to the efficacy of cognitive behaviour therapy for children and adolescents presenting with depression or anxiety. The literature review was organized with reference to four key questions each of which is discussed in turn. 1) What evidence supports the use of CBT for children and adolescents experiencing anxiety or depression? Ample evidence points to the benefits of CBT for children and adolescents experiencing anxiety (American Academy of Child and Adolescent Psychiatry, 2006; Cartwright-Hatton et al., 2004; Compton et al., 2004; Friedberg et al., 2003; Powers, Jones, & Jones, 2005) or depression (Compton et al., 2004; Friedberg et al., 2003; Powers, Jones, & Jones, 2005), and Haby et al. (2004) suggest that CBT is a cost-effective first line order treatment when compared to psychopharmacological intervention. Murray and Cartwright-Hatton (2006) note that British guidelines for the use of CBT suggest that group based CBT is a first response option for mild depression. Fifty to eighty percent of children in CBT treatments for anxiety disorders have been found to be 2

7 diagnosis free or markedly improved after treatment (Siqueland, Rynn & Diamond, 2005); however, there is some evidence to suggest that the salutary impact of CBT may be less for younger clients than for older adolescents (Cartwright-Hatton et al., 2005; Durlak, Fuhrman, & Lampman, 1991). 2) What evidence supports age mates (i.e., children or adolescents) being in the same CBT intervention when some of them are experiencing depression and others are experiencing anxiety? Evidence suggests that having children or adolescents with anxiety or depression participate together in the same group based CBT is appropriate for at least three reasons. First, various studies have concluded that anxiety and depression are one construct among young children but clarification of the connection between these two disorders has been thwarted by the use of measures that purport to be tapping one of these constructs yet include items found in measures of the other construct (Murphy, Marelich, & Hoffman, 2000). Second, empirical evidence suggests that untreated childhood anxiety disorders may beget depression (Kendall et al., 2004). Conversely, CBT for youth being treated for depression can involve focusing on improving emotion management including anxiety (Gallagher, 2005) suggesting that anxiety may be borne from depression. Third, both anxiety and depression are often measured as outcomes in studies assessing the impact of CBT on children or adolescents regardless of which disorder is viewed as the presenting problem (see, for examples, Baer & Garland, 2005; Kendall et al., 2004; O Kearney et al., 2006, and Sheffield, et al., 2006). 3) What characteristics or features of the CBT are thought to be important in contributing to its effectiveness? The skills of clinicians involved in delivering CBT, attendance at CBT sessions, and extent and nature of co-morbidity may all impact on the efficacy of CBT. In their study on the effectiveness of CBT for adolescents with depression, Kerfoot et al. (2004) found that youths who received CBT did not differ significantly from youths who received routine care on measures of depression and a generic measure of child and adolescent mental health. The authors suggested that while their sample size meant their study lacked statistical power, co-morbidity, attrition (less than 50 % of youths received four or more CBT sessions), and the skills of the clinicians may have contributed to the poor outcomes. Compton et al. (2004) note that research on the efficacy of CBT lacks systematic consideration of the impact of comorbidity on outcomes, which also suggests that this is potentially an important factor in the efficacy of CBT. In addition, cohesion with others in group-based CBT and therapeutic alliance may also be important factors that impact on the effectiveness of CBT (Kaufman et al., 2005). 3

8 4) What study designs and data are most commonly used to assess the efficacy of CBT with children or adolescents experiencing anxiety or depression? Study designs have tended to include designs in which clients are randomly assigned to a group that receives a particular intervention or to a second group that receives that intervention plus CBT (see, for example, Siqueland, Rynn, & Diamond, 2005). Comparing outcomes across the two groups of clients enables a researcher to determine the value added or additional benefits that can likely be attributed to the CBT. A wait-list control group (see, for example, Nauta et al., 2003) is often used and compared to one or more CBT interventions, which may or may not be compared to other interventions. As a case in point, Spence et al. (2006) compared clinic based CBT, a combined internet and clinic based CBT, and a wait list control group and found that both CBT interventions netted improvements in children s and adolescent s anxiety. By contrast, Barbe et al. (2004) compared the effectiveness of CBT to two other types of interventions in helping depressed youths. Where no comparison group exists, data are generally collected pre, post, and at a follow up time period such as six to nine months after completion of a CBT program (see, for example, Siqueland, Rynn & Diamond, 2005). This is done to determine whether outcomes are sustained over time or just in the short term after CBT. Hybrids of these various designs are also used. In regard to data sources, Compton et al. (2004) observe that evaluation studies often involve the collection of data from multiple sources (e.g., child, parent, clinician) and through varied methods (e.g., self-report measures, clinical interview). This is in keeping with the principle of triangulation in research. In the case of evaluating CBT interventions with children and adolescents, parental ratings of children s and adolescents distress have been used. As examples, Suveg et al. (2006) and Wood et al. (2006) asked parents to complete a version of the Multidimensional Anxiety Scale for Children that was adapted for parents to provide ratings of their children s anxiety. Similarly, Spence et al. (2006) had parents complete the Spence Children s Anxiety Scale Parent Version. Outcomes assessed in research on group-based CBT interventions with anxious children have tended to be: anxiety symptom measures, cognitive indices, coping measures, and diagnostic status (Prins & Ollendick, 2003). Parental anxiousness has also been measured. In addition, given that CBT is intended to help with emotion regulation to the extent that it improves one s emotion focused coping efforts, researchers such as Suveg et al. (2006) have looked for improvements in positive and negative affect as a result of CBT and have done so through the use of measures such as the Positive and Negative Affect Scale for Children. Changes in thoughts and beliefs, as well as coping strategies and the effectiveness of those strategies are assumed to be key aspects of CBT and both have been measured as part of assessing the impact of CBT amongst children with anxiety (Prins & Ollendick, 2003). 4

9 Clinically important indicators, not just outcomes that are significant in a statistical sense, are often reported such as changes that move clients below a threshold score (Compton et al., 2004). It is important to keep in mind that if a group of clients is comprised of mostly individuals whose level of distress (anxiety and/or depression) is low then the group will have a low average score. As a result, the group s average score may not shift much after the intervention and whilst this reduction might be important clinically it may not yield statistically significant results. In interpreting both statistical and clinically important outcomes, it should be kept in mind that important sex differences and age differences (as a proxy for cognitive development) may exist in relation to the manifestation of anxiety and depression. Sheffield et al. (2006) found in a sample of 2479 adolescents aged 13 to 15 years that those scoring high on the Children s Depression Inventory were more likely to be female. The most frequent finding in studies of the efficacy of CBT with children and youth presenting with depression or anxiety is a lack of differences based on age or sex; however, the authors note that few studies have had adequate statistical power thereby leaving open the possibility that important age and/or sex differences exist (Compton et al., 2004). Certain family characteristics and ways of interacting may contribute to children s anxiety (Cartwright-Hatton et al., 2005; Siqueland, Rynn, & Diamond, 2005). Research points to the benefits of family-based cognitive behaviour therapy, including group based interventions, for anxious children (Barrett, Dadds, & Rapee, 1996; Wood et al., 2006). Where age of youngsters has been used as a proxy for cognitive development, parental distress appears to have a greater impact on younger children s psychological wellbeing than it does on that of adolescents (Berman et al., 2000). Given the importance of family functioning, it is not surprising that at least one study has measured family functioning. Siqueland and colleagues (2005) used the Children s Report of Parenting Behaviour Inventory, which is a 30-item measure of children s perceptions of parents actions. The authors had children complete the measure in relation to each parent separately and used the scores in relation to one parent in the analysis. In keeping with the notion that the skills of clinicians bear significantly on the impact of CBT, Siqueland and colleagues (2005) also used the Cognitive Therapy Scale and the Therapeutic Behavior Rating Scale 3 rd version to assesses the competency of clinicians who delivered the CBT intervention. The senior responsible author (Siqueland) rated each clinician on these scales. It is important to note that a detailed record of an intervention need be maintained to ensure that a study may be replicated (Powers, Jones, & Jones, 2005) and to demonstrate fidelity to a particular service delivery model. In addition, a detailed record of an intervention can provide insights that might aid other organizations in service planning and delivery. 5

10 6.0 Methodology Pre and post-intervention scores on measures of depression and anxiety constituted the data for this study. More specifically, the Children s Depression Inventory (CDI; Kovacs, 1992) was used to measure depression and the Multidimensional Anxiety Scale for Children (MASC; March, 1997) was used to measure anxiety. 1 Both measures are self-report measures that were completed by children and youth who participated in the Program, and both measures have been used in research to assess the efficacy of CBT interventions (see, as examples of studies that have used the CDI, Friedberg et al., 2003; Kendall et al., 2004; Lyneham & Rapee, 2006; Nauta et al., 2003; Sheffield et al., 2006; Spence et al., 2006; Suveg, et al., 2006; and see, as an example of a study that has used the MASC, Wood et al., 2006). The Children s Depression Inventory (CDI) has 27 items (e.g., I am sad, I feel like crying, I do everything wrong ) about the cognitive, mood and behavioural signs of depression and children choose one of three alternative responses to indicate which response best reflects them over the previous two weeks (Kendall et al., 2004). Scores range from 0 or once in a while reflecting no symptoms through 1 ( many times ) and 2 ( all the time ) reflecting mild and severe symptoms, respectively, and total scores range from 0 to 54 (Stark & Laurent, 2001). The scale has demonstrated sufficient to good psychometric properties (Kovacs, 1992; Nauta et al., 2003). The Multidimensional Anxiety Scale for Children is a 39-item, 4-point Likert-type, psychometrically sound measure (March et al., 1997). The Multidimensional Anxiety Scale for Children (MASC) asks questions related to: physical symptoms, harm avoidance, social anxiety, and separation anxiety (March 1997; March, Sullivan, & Parker, 1999). Data were analyzed using SPSS Version Pre-program data were available for 95 participants (59 female, 29 male and 7 sex unknown) and post-program data were available for 51 participants (36 female, 13 male and 2 sex unknown). Repeated measures ANOVAs with sex and location (clinic setting or high school setting) as between subjects factors and pre and post-program test scores as a within subjects factor were conducted on total CDI and MASC test scores. Separate male and female paired samples (pre and post-program) t-tests on CDI and MASC sub-scale scores were conducted to examine whether there were statistically significant changes in pre and post-program sub-scale scores. There were no useable data for 7-10 year olds as measures were not completed by those clients and there were insufficient post-program data for year olds to include age or age-group as a factor in any of the analyses. Table 1 displays sex 1 Copyright laws prohibit reproduction of these measures and thus copies of the measures are not provided in an appendix to this report. 6

11 and age information for clients completing pre and/or post program CDI and/or MASC tests. Study data were examined with respect to available norms for the CDI and MASC to offer some insights regarding program participants and the clinical significance of changes observed. As advocated by Jacobson and Traux (1991), Reliable Change (RC) scores were calculated to shed further light on the meaningfulness of changes. Table 1: Sex and Age of Participants Completing CDI and/or MASC Tests Pre-Program Data Post-Program Data Boys 10 yrs yrs yrs yrs yrs yrs yrs yrs yrs Total Girls 10 yrs yrs yrs yrs yrs yrs yrs yrs yrs Total Findings Repeated measures ANOVAs were conducted on CDI and MASC test data with sex and setting (clinic or high school) as between-subjects variables and pre and post-program test scores as the within-subjects variable. Results of the analysis on CDI data indicate a statistically significant decrease in scores from pre to post program, F(1,45)=11.40, p=.002 with means of (SD=10.08) and (SD=9.85), respectively. There was a significant main effect of sex, F(1,45)=5.91, p=.019, such that girls scored higher on the CDI tests (M=21.93, SD=9.70) than did boys (M=14.25, SD=8.54). The main effect of setting was non-significant, F(1,45)=.26, p=.612. None of the interactions were statistically significant 7

12 indicating a similar pattern of decrease in scores from pre to post program for both girls and boys at both settings. Results of the analysis on MASC data also indicate a statistically significant decrease in scores from pre to post-program, F(1,44)=6.86, p=.012 with means of (SD=21.50) and (SD=23.37), respectively. As with the CDI tests, girls scores on the MASC (M=57.41, SD=21.50) were higher than boys scores (M=44.46, SD=22.88) although this difference was not statistically significant, F(1,44)=2.53, p=.119. The main effect of setting was significant, F(1,44)=9.66, p=.003, with high school setting participants scoring higher (M=62.25, SD=14.27) on the MASC than clinic setting participants (M=47.95, SD=25.38). None of the interaction effects were statistically significant indicating that the pattern of decrease in scores from pre to post program was similar for boys and girls at both settings. Table 2 presents the results of t-tests comparing pre and post program test scores for the CDI and MASC sub-scales. As can be seen in Table 2, girls scores decreased significantly in four of five CDI subscales, including the negative self-esteem subscale, even at a fairly stringent P-value established to control for error associated with conducting multiple comparisons. On the MASC test, a significant decrease was observed on one of the four subscales (i.e., physical symptoms of anxiety). For boys, while most of the changes were in the anticipated direction, none of the subscale decreases were statistically significant at the.01 alpha level. The CDI total and MASC total test scores for clients who completed pre and post-program tests were recoded into standardized t-scores using the norms provided in the technical manuals (Kovacs, 1992; March, 1997) for the respective tests and frequencies for the interpretive guideline categories suggested by the test authors are presented in Table 3. Two observations are readily observable from the frequencies in Table 3. First, changes in frequencies between categories from pre to post-program are greatest for girls on the CDI measure where 32 of 36 (88%) clients where classified as above the normal average at pre-program and 22 of 36 (61%) clients were classified as above the normal average at post-program. Second, a relatively small percentage of boys were classified as above average or higher on the pre-program CDI and MASC tests and consequently there was little categorical change for boys. 8

13 * p <.01 Table 2: CDI and MASC Sub-Scale Pre and Post Program T-tests Pre Program Post Program Mean SD Mean SD DF t Sig. (2-tailed) Girls CDI negative mood * CDI interpersonal problems CDI ineffectiveness * CDI anhedonia * CDI negative self-esteem * MASC physical symptoms * MASC harm avoidance MASC social anxiety MASC separation Boys CDI negative mood CDI interpersonal problems CDI ineffectiveness CDI anhedonia CDI negative self-esteem MASC physical symptoms MASC harm avoidance MASC social anxiety MASC separation Table 3: Client Classification Frequencies for CDI and MASC Total T-Scores Boys Girls T-score CDI MASC CDI MASC Range Classification Pre Post Pre Post Pre Post Pre Post Above 70 very much above average to 70 much above average to 65 above average to 60 slightly above average to 55 Average to 44 slightly below average to 39 below average to 34 much below average Below 30 very much below average

14 Reliable Change (RC) index scores (Jacobsen & Traux, 1991) were calculated to identify clients whose change in MASC and/or CDI total scores were large enough that there is reason to be confident that the change represents real change. Thirty-one percent (4 of 13) of boys and 52 percent (19 of 36) of girls had scores at post program that one can be confident were real changes in the CDI and/or MASC measures (i.e., not likely due to measurement error). Table 4 summarizes the change experienced by these clients. The categories reported in Table 4 are equivalent to those reported in Table 3 and represent the same ranges as described in that table. Of the 23 participants demonstrating reliable change, 20 improved and 2 were in the opposite direction of hoped for change and 1 showed improvement on the MASC measure but a higher CDI score. Of the 14 clients showing reliable improvement on the CDI measure, 5 remained above average or higher at post-program and 9 were slightly above average or lower. Of the 12 clients showing reliable improvement on the MASC measure, 5 remained above average or higher and 7 were slightly above average or lower. Table 4: Client Classifications Pre and Post-Program for Clients Demonstrating Reliable Change CDI MASC Sex Pre Program Post Program Pre Program Post Program M* slightly above Very much above below very much below M slightly above n/c much above below M average below much above slightly below M much above average very much above much above F much above n/c very much above above F much above n/c very much above much above F very much above Very much above average below F very much above n/c average much below F* much above n/c much below above F much above average very much below n/c F much above much below slightly above very much below F above slightly below above average F very much above above above n/c F* above Very much above average n/c F very much above much above very much above slightly above F very much above n/c very much above above F very much above much above average n/c F very much above slightly above very much above n/c F very much above slightly above slightly below missing F above average much above n/c F very much above much above slightly above n/c F very much above n/c very much above very much above F very much above average above n/c * Indicates a client recorded a higher CDI and/or MASC score at post-program. n/c indicates no reliable change from pre-program. 10

15 8.0 Implications of Findings Findings suggest that, in general, the CBTP had a positive impact, particularly for girls. To the extent that the CBTP contributes to promoting improved self esteem amongst girls this is in and of itself an important outcome because high selfesteem is a protective factor associated with better mental and physical health outcomes (Mann et al., 2004). The finding that girls, on average, benefited more from the CBTP than did boys can be explained, at least in part, by the fact that the male clients resembled a normal population more than a clinical population. In fact, 10 of 29 (34%) boys did not have above average or higher pre-program CDI or MASC using the categorical guidelines suggested by the test authors. In addition, statistical power for analyses relating to boys scores was low given the small number of boys for whom post-program data were available. The interpretation of study findings is limited due to the study design, the absence of post-intervention data for many clients, and the lack of additional data sources. It is likely that had other sources of data been systematically gathered and analyzed (e.g., self report ratings on other measures, parental ratings, and clinician observations) that a better understanding of the ways in which the CBTP impacts on clients would have been gleaned. 9.0 Recommendations The following recommendations are suggestions for strengthening the CBTP s capacity for future program evaluation studies, including assessing the Program s fidelity to the Cool Kids model and impact on clients. 1. Review selection procedures. Examine the CBTP eligibility criteria and selection procedures for inclusion in the groups to avoid having a high percentage of children who test as normal in the groups. Consider using the CDI and MASC as screening tools. 2. Develop a program logic model. Consider developing a program logic model in which intermediate and long term goals for the program are articulated and linked to program activities, resources (inputs), and eligibility/selection criteria, and indicators of program success. A logic model will assist with determining if more appropriate and/or additional measures should be used in assessing program impact. 3. Use a mixed-methods, multiple data source approach. Consider using a study design that compares the CBT to a wait list control group and/or alternate interventions to assess the efficacy of the Program with clients in the future. Alternatively, use a pre and post-design but collect data from more 11

16 sources, at a minimum of three time intervals, and through different formats (e.g., self-report measures, chart review, observational research, parent reports, clinician observations). In designing an evaluation plan, demands on clients, their families, and staff need to be balanced with getting valid and reliable data. Charts provide a wealth of data and chart review as a data collection strategy poses no burden to clients; however, this approach can be time consuming for the researcher (or clinical staff in the absence of a researcher) and the data can be of limited reliability if clinicians are not given time to ensure consistency in the type, frequency, and format of information entered into charts. Also, consider interviewing a sub-group of families to obtain a rich understanding of the benefits of the Program, and develop insights that complement findings relating to quantitative data. 4. Measure additional variables. In statistical analyses, if the number of participants permits, measure and control for additional variables (e.g., type of anxiety/depression, co-morbidity, attendance at CBT sessions, extent of parental involvement in the CBT intervention, children s coping, parental interaction style, extent of training of professionals implementing the intervention) that might reasonably influence or be impacted by the Program and thus would effect scores on the measures of depression and anxiety. 5. Obtain follow up data from Program participants. More specifically, ask them to complete the Children s Depression Inventory and the Multidimensional Anxiety Scale for Children at 6 to 9 month follow up after completion of the CBTP. 6. Engage dedicated evaluation expertise early and throughout. This will facilitate a more focused effort at evaluation design and data collection than is likely when clinical staff assume these duties in addition to their direct service responsibilities Knowledge Exchange Plan Findings will be shared: o with members of CMHC s Child and Adolescent Services Team; o with agency staff as a whole through one or more of the following mechanisms: the CMHC s newsletter, a lunch and learn session, or at an All Staff Meeting; o with the Children s Mental Health Community Advisory Committee, which consists of various community partners including: the Upper Grand District School Board, Wellington Separate School Board, Family and Children s Services, Community Alcohol and Drug Services, Wellington Dufferin Health Unit, University of Guelph, Better Beginnings Better Futures, and the Guelph Police Service; 12

17 o o o o o at a Central West Regional meeting of Children s Mental Health Ontario; at a relevant conference as a poster presentation or an oral presentation; with families when presenting options and the benefits of groupbased CBT; the Ministry of Children and Youth Services to aid in service delivery planning; with prospective external funders, where appropriate, to support requests for funds for research support staff and other resources required to undertake a comprehensive evaluation of the Program References Note: The following is a list of references cited in this report. For information on measures mentioned other than the Children s Depression Inventory and the Multi-dimensional Anxiety Scale for Children, the reader should do one or more of the following: consult the cited studies, Google search the measure on the internet (to determine if a measure is publicly available and obtain author contact information), contact the author of the measure, or check test and measures guidebooks that are available within reference sections of university libraries. American Academy of Child and Adolescent Psychiatry. (2006). Practice parameter for the assessment and treatment of children and adolescents with anxiety disorders. [Available on line]. Baer, S., & Garland, E. J. (2005). Pilot study of community-based cognitive behavioral group therapy for adolescents with social phobia. Journal of the American Academy of Child & Adolescent Psychiatry, 44(3), Barbe, R. P., Bridge, J., Birhamer, B., Kolko, D., & Brent, D. A. (2004). Suicidality and its relationship to treatment outcome in depressed adolescents. Suicide and Life-Threatening Behavior, 34(1), Barrett, P., Dadds, M.R., & Rapee, R.M. (1996). Family treatment of childhood anxiety: A controlled trial. Journal of Consulting and Clinical Psychology, 64(2), Berman, S.L., Weems, C.F., Silverman, W.K., & Kurtines, W.M. (2000). Predictors of outcome in exposure-based cognitive and behavioral treatments for phobic and anxiety disorders in children. Behavior Therapy, 31,

18 Cartwright-Hatton, S., McNally, D., & White, C. (2005). A new cognitive behavioural parenting intervention for families of young anxious children: A pilot study. Behavioural and Cognitive Psychotherapy, 33(2), Cartwright-Hatton, S., Roberts, C., Chitsabesan, P., Fothergill, C., & Harrington, R. (2004). Systematic review of the efficacy of cognitive behaviour therapies for childhood and adolescent anxiety disorders. British Journal of Clinical Psychology, 43(4), Compton, S. N., March, J. S., Brent, D., Albano, A. M., Weersing, V. R., & Curry, J. (2004). Cognitive-behavioral psychotherapy for anxiety and depressive disorders in children and adolescents: An evidence-based medicine review. Journal of the American Academy of Child & Adolescent Psychiatry, 43(8), Durlak, J.A., Fuhrman, T., & Lampman, C. (1991). Effectiveness of cognitivebehavior therapy for maladapting children: A meta-analysis. Psychological Bulletin, 110, Friedberg, R. D., McClure, J. M., Wilding, L., Goldman, M. L., Long, M. P., & Anderson, L., & DePolo, M.R. (2003). A cognitive-behavioral skills training group for children experiencing anxious and depressive symptoms: A clinical report with accompanying descriptive data. Journal of Contemporary Psychotherapy, 33(3), Gallagher, R. (2005). Evidence-based psychotherapies for depressed adolescents: A review and clinical guidelines. Primary Psychiatry, 12(9), Haby, M. M., Tonge, B., Littlefield, L., Carter, R., & Vos, T. (2004). Costeffectiveness of cognitive behavioural therapy and selective serotonin reuptake inhibitors for major depression in children and adolescents. Australian and New Zealand Journal of Psychiatry, 38(8), Jacobson, N.S., & Truax, P. (1991). Clinical significance: A statistical approach to defining meaningful change in psychotherapy research. Journal of Consulting and Clinical Psychology, 59(1), Kaufman, N. K., Rohde, P., Seeley, J. R., Clarke, G. N., & Stice, E. (2005). Potential mediators of cognitive-behavioral therapy for adolescents with comorbid major depression and conduct disorder. Journal of Consulting and Clinical Psychology, 73(1), Kendall, P. C., Safford, S., Flannery-Schroeder, E., & Webb, A. (2004). Child anxiety treatment: Outcomes in adolescence and impact on substance use and depression at 7.4-year follow-up. Journal of Consulting and Clinical Psychology, 72(2),

19 Kerfoot, M., Harrington, R., Harrington, V., Rogers, J., & Verduyn, C. (2004). A step too far? randomized trial of cognitive-behaviour therapy delivered by social workers to depressed adolescents. European Child & Adolescent Psychiatry, 13(2), Kovacs, M. (1992). Children s Depression Inventory Manual. Multi-Health Systems: Toronto, ON. Lyneham, H. J., & Rapee, R. M. (2006). Evaluation of therapist-supported parentimplemented CBT for anxiety disorders in rural children. Behaviour research and therapy, 44(9), Mann, M., Hosman, C.M.H., Schaalma, H.P., & de Vries, N.K. (2004). Selfesteem in a broad-spectrum approach for mental health promotion. Health Education Research, 19(4), March, J. (1997). Multidimensional Anxiety Scale for Children. Multi-Health Systems: Toronto, ON. March, J.S., Parker, J., Sullivan, K., Stallings, P., & Conners, K. (1997). The Multidimensional Anxiety Scale for Children (MASC): Factor structure, reliability, and validity. Journal of the American Academy of Child and Adolescent Psychiatry, 36(4), March, J.S., Sullivan, K., & Parker, J. (1999). Test-retest reliability of the Multidimensional Anxiety Scale for Children. Journal of Anxiety Disorders, 13(4), Murphy, D.A., Marelich, W.D., & Hoffman, D. (2000). Assessment of anxiety and depression in young children: Support for two separate constructs. Journal of Clinical Child Psychology, 29(3), Murray, J., & Cartwright-Hatton, S. (2006). NICE guidelines on treatment of depression in childhood and adolescence: Implications from a CBT perspective. Behavioural and Cognitive Psychotherapy, 34(2), Nauta, M. H., Scholing, A., Emmelkamp, P. M. G., & Minderaa, R. B. (2003). Cognitive-behavioral therapy for children with anxiety disorders in a clinical setting: No additional effect of a cognitive parent training. Journal of the American Academy of Child & Adolescent Psychiatry, 42(11), O'Kearney, R., Gibson, M., Christensen, H., & Griffiths, K. M. (2006). Effects of a cognitive-behavioural internet program on depression, vulnerability to depression and stigma in adolescent males: A school-based controlled trial. Cognitive Behaviour Therapy, 35(1),

20 Powers, S. W., Jones, J. S., & Jones, B. A. (2005). Behavioral and cognitivebehavioral interventions with pediatric populations. Clinical Child Psychology and Psychiatry, 10(1), Prins, P. J. M., & Ollendick, T. H. (2003). Cognitive change and enhanced coping: Missing mediational links in cognitive behavior therapy with anxietydisordered children. Clinical Child and Family Psychology Review, 6(2), Sheffield, J. K., Spence, S. H., Rapee, R. M., Kowalenko, N., Wignall, A., & Davis, A., & McLoone, J. (2006). Evaluation of universal, indicated, and combined cognitive-behavioral approaches to the prevention of depression among adolescents. Journal of Consulting and Clinical Psychology, 74(1), Siqueland, L., Rynn, M., & Diamond, G. S. (2005). Cognitive behavioral and attachment based family therapy for anxious adolescents: Phase I and II studies. Journal of Anxiety Disorders, 19(4), Spence, S. H., Holmes, J. M., March, S., & Lipp, O. V. (2006). The feasibility and outcome of clinic plus internet delivery of cognitive-behavior therapy for childhood anxiety. Journal of Consulting and Clinical Psychology, 74(3), Stark, K. D., & Laurent, J. (2001). Joint factor analysis of the Children s Depression Inventory and the Revised Children s Manifest Anxiety Scale. Journal of Clinical Child Psychology, 30(4), Suveg, C., Kendall, P. C., Comer, J. S., & Robin, J. (2006). Emotion-focused cognitive-behavioral therapy for anxious youth: A multiple-baseline evaluation. Journal of Contemporary Psychotherapy, 36(2), Wood, J. J., Piacentini, J. C., Southam-Gerow, M., Chu, B. C., & Sigman, M. (2006). Family cognitive behavioral therapy for child anxiety disorders. Journal of the American Academy of Child & Adolescent Psychiatry, 45(3),