Laura D. Seligman Department of Psychology, University of Toledo

Transcription

1 Journal of Clinical Child and Adolescent Psychology 2004, Vol. 33, No. 3, Copyright 2004 by Lawrence Erlbaum Associates, Inc. The Utility of Measures of Child and Adolescent Anxiety: A Meta-Analytic Review of the Revised Children s Manifest Anxiety Scale, the State Trait Anxiety Inventory for Children, and the Child Behavior Checklist Laura D. Seligman Department of Psychology, University of Toledo Thomas H. Ollendick Child Study Center, Department of Psychology, Virginia Polytechnic Institute and State University Audra K. Langley Childhood OCD, Anxiety, and Tourette s Program, University of California at Los Angeles Heidi Bechtoldt Baldacci Department of Psychology, University of Toledo We evaluated the ability of the Revised Children s Manifest Anxiety Scale (RCMAS), the State Trait Anxiety Inventory for Children (STAIC), and the Child Behavior Checklist (CBCL) to (a) discriminate between youth with an anxiety disorder and youth without a disorder, (b) discriminate between youth with an anxiety disorder and youth with either externalizing disorders or affective disorders, and (c) measure treatment change. In addition, variables, including age and sex, were explored as possible moderators of instrument utility. A meta-analysis of 43 articles was conducted. A large effect size was found when the instruments were used to compare youth with an anxiety disorder to youth without a disorder. When comparing anxious youth to psychiatric control groups, the picture was mixed; the instruments were found to be useful when discriminating between youth with an anxiety disorder and youth with an externalizing disorder, but not between youth with an anxiety disorder and children and adolescents with an affective disorder. The RCMAS, STAIC, and CBCL were found to be moderately sensitive to treatment gains. The increasing complexity of the questions addressed in research on child psychopathology in general, and anxiety disorders in particular, and the evolving demands in clinical research and practice, both in terms of demonstrating treatment efficacy and effectiveness and doing more in less time, has resulted in the demand for effective assessment instruments. Useful assessment instruments must fulfill a variety of functions, including providing data for differential diagnosis and assessing change over time (March & Albano, 1996). Recently, several new self-report instruments have been developed for assessing symptoms of childhood anxiety, with the hopes that they will meet these demands. With these new developments has come the belief that existing instruments are inadequate for the task at hand. In fact, in a recent narrative review of Requests for reprints should be sent to Laura D. Seligman, Department of Psychology, The University of Toledo, Mail Stop #948, Toledo, OH Laura.seligman@utoledo.edu scales for assessing anxiety disorders in children and adolescents, Myers and Winters (2002) concluded that the discriminant validity of a host of older traditional measures was found wanting, especially in regard to differentiating children with attention deficit hyperactivity disorder (ADHD) and major depressive disorder from children and adolescents with anxiety disorders. Myers and Winters further suggested that some of the newer measures, such as the Multidimensional Anxiety Scale for Children (MASC; March, 1997) held greater promise for discriminant validity than did older ones such as the Revised Children s Manifest Anxiety Scale (RCMAS; Reynolds & Richmond, 1985) and the State Trait Anxiety Inventory for Children (STAIC; Spielberger, 1973). They concluded their review with statements such as Although still widely used, the RCMAS is being supplanted by newer scales (p. 647) and the STAIC is being eclipsed by newer scales (p. 649). The obvious conclusion is that researchers and clinicians alike should discontinue use of traditional instruments such 557

2 SELIGMAN, OLLENDICK, LANGLEY, BALDACCI as the RCMAS and STAIC and use the newer and more psychometrically sound measures. However, narrative reviews are limited in that they are open to interpretive biases. Unfortunately, there are no explicit criteria available to reviewers to guide them in the selection of studies to use in arriving at their conclusions, and often it is unclear what data to use in formulating findings. Additionally, studies have shown that it is difficult to accurately integrate the complex types of data often included in qualitative reviews (Fiske & Taylor, 1991). Moreover, several studies have examined the ability of the RCMAS and STAIC to discriminate between youth with anxiety disorders and youth with other disorders. Although results have not been uniformly positive, some support for the discriminant validity of the instruments has been found. For example, Perrin and Last (1992) showed that the RCMAS total score, along with each of its three subscale scores (worry oversensitivity, fear concentration, and physiological anxiety) differentiated between boys with anxiety disorders and boys with no disorder. However, the scale scores did not differentiate between boys with an anxiety disorder and boys with ADHD. Moreover, the total score from a version of the STAIC modified to include state and trait scales for both cognitive and state anxiety also differentiated between the youth with an anxiety disorder and youth with no disorder, but not between the two psychiatric groups. The subscale scores from the modified STAIC, with the exception of the cognitive-state-anxiety subscale, were also able to differentiate between youth with an anxiety disorder and youth with no disorder, but not between the youth with anxiety disorder and youth with ADHD. On the other hand, Lonigan, Carey, and Finch (1994) found the worry oversensitivy subscale of the RCMAS was able to distinguish between hospitalized youth with pure anxiety disorders and those with pure affective disorders, a distinction that has often proven to be problematic and elusive (Seligman & Ollendick, 1998). The RCMAS total score and the concentration and physiological anxiety subscales, however, did not differentiate between the two groups. Although comparative studies that include data on the RCMAS and STAIC provide tests of statistical significance, they do not make clear the clinical significance or magnitude of differences even when statistical significance is reached. Therefore, even when group differences are found, it is unclear whether these are meaningful differences. Therefore, taken in isolation, these studies, like current narrative reviews, are limited in their ability to synthesize the area in a systematic, unbiased way that proves useful to researchers and clinicians who use these instruments as part of a diagnostic assessment or to monitor changes in response to treatment. The goal of this investigation was to provide a quantitative review of the literature on the RCMAS and STAIC to address questions regarding the clinical utility of the most widely used self-report instruments for anxiety in youth for discriminating between diagnostic groups. Because these measures are also frequently used as indictors of treatment outcome, a second goal was to examine their utility for this purpose. In addition, given the oft-reported finding of low reliability between parent and child reports (cf. Grills & Ollendick, 2002), as well as ongoing questions about the advantages and disadvantages of different informants, we provide data on the internalizing scale of the Child Behavior Checklist (CBCL; Achenbach, 1991), one of the most widely used parent reports. We build on studies such as those conducted by Lonigan et al. (1994) and Perrin and Last (1992), which directly investigated the discriminant validity of child anxiety assessment instruments but also included studies that did not aim to answer this question but nonetheless provided sufficient data to do so. In addition, given preliminary evidence that social desirability may decrease with age and impact self-reports of anxiety more for anxious boys than girls (Dadds, Perrin, & Yule, 1998; Pina, Silverman, Saavedrea, & Weems, 2001), where possible we explore sex and age as moderators of effect size. In addition, study characteristics such as the informant for diagnostic information and the method used to recruit the sample are explored as possible moderators. Method Literature Search Studies included in these analyses were gleaned from several sources: (a) searches of PsychLit, PsychFirst, Medline, ERIC, and Dissertation Abstracts databases; (b) manual searches of the eight journals deemed most likely to publish anxiety studies involving children and adolescents; 1 (c) letters sent to 22 leading childhood anxiety researchers, requesting copies of both published and unpublished studies; and (d) inspection of reference lists of all selected studies. This literature search included studies conducted over a two-decade period, dating from January 1980 through December The rationale for commencing the search in 1980 was that the Diagnostic and Statistical Manual for Mental Disorders (3rd ed. [DSM III], American Psychiatric Association, 1980) was the first edition to specifically recognize childhood anxiety diagnoses. 1 Journal of Anxiety Disorders, Journal of Clinical Child Psychology (now known as Journal of Clinical Child and Adolescent Psychology), JournaloftheAmericanAcademyofChild& AdolescentPsychiatry, Journal of Abnormal Child Psychology, Journal of Child Psychology and Psychiatry, Behavior Therapy, Behaviour Research and Therapy, Behavior Therapy and Experimental Psychiatry. 558

3 ASSESSMENT OF CHILD AND ADOLESCENT ANXIETY Inclusion and Exclusion Criteria Effect sizes were coded for the RCMAS and its subscales, the two scales of the STAIC (i.e., the state and trait scale), and the internalizing scale from the CBCL (parent report), provided studies met the following criteria: 1. The article was published in English; measures could have been given in other languages as we included studies that were conducted in the United States (k = 31), Australia (k = 4), Israel (k = 3), the Netherlands (k = 3), Great Britain (k = 1), and Canada (k = 1). However, no articles specifically indicated that the instruments were translated into other languages. 2. Participants were diagnosed youth. Diagnoses could have been derived from a variety of methods, including structured or semistructured interviews, unstructured clinical interviews, chart reviews, consensus diagnosis, or a combination of these methods. Studies that used a cutoff score on some self- or other-report of anxiety were not included. In addition, studies needed to include data on at least one of the measures of interest, and those measures could not have been used in arriving at the diagnosis. This ensured the independence of the data. 3. Data for a comparison group also needed to be reported. Comorbid conditions in both the anxiety group and the comparison group, if present, were coded. Comorbid status did not exclude groups from the comparisons. Coding Procedures Trained undergraduate and graduate research assistants coded all studies. Effect size calculations were coded twice, and any discrepancies were resolved by consensus. Demographic information, including age, sex, and diagnosis, was coded separately for anxiety and comparison groups in each of the effect size calculations. Although characteristics such as ethnicity and socioeconomic status were coded under demographic characteristics, few studies reported this information separately by diagnosis. Similarly, we attempted to collect data on the internal consistency of the instruments of interest in the study samples; however, very few studies reported reliability data by diagnostic group. In addition to the data for each effect size calculation, the method of diagnosis (e.g., structured or semistructured interview, chart review), informant for diagnostic information, help-seeking status of the sample, and diagnostic system used was coded for each study. Calculation of Effect Sizes When means and standard deviations were available, effect sizes were computed using Hedges and Olkin s (1985) formula for Hedges g. When means and standard deviations were not reported, alternative effect size estimation procedures were used according to Wolf s (1986) guidelines; however, only one of the effect sizes coded required use of estimation methods. Effect sizes were converted to Hedges d (Hedges & Olkin, 1985) to correct for bias associated with sample size. For effect sizes comparing diagnostic groups, positive effect sizes indicated higher scores for the youth with an anxiety disorder; for effect sizes measuring treatment changes, positive effect sizes were indicative of improvement posttreatment. Results A total of 26 studies produced effect sizes that compared children and adolescents with anxiety disorders (with or without comorbid conditions) to either (a) youth with no diagnosis (k = 23), (b) youth with one or more externalizing disorders (k = 6), or (c) youth diagnosed with a major depressive disorder (k = 4). 2 An additional 17 studies contained data for anxiety-disordered youth pre- and posttreatment. Together, these 43 studies included a total of 1,509 children and adolescents in the anxiety groups of interest and 1,170 children and adolescents in the comparison groups of interest. All but one of the studies was published by the time of this review. Twelve percent of the studies used DSM III criteria for diagnosis; 56% followed Diagnostic and Statistical Manual for Mental Disorders (3rd ed., rev. [DSM III R], American Psychological Association, 1987) criteria, 25% utilized Diagnostic and Statistical Manual for Mental Disorders (4th ed. [DSM IV], American Psychological Association, 1994) criteria, 5% of studies used a combination of diagnostic systems, and 2% (one study) did not report which diagnostic system was used. In the overwhelming majority of studies, a structured or semistructured diagnostic interview was used to determine diagnoses either alone (74% of the studies) or in conjunction with other information (21% of the studies). The two studies (5%) that did not use a structured or semistructured interview relied on a consensus diagnosis or a combination of methods. In 33% of the studies, diagnosis was based on child report only; 7% of the studies used parent report only, 42% used data from both parent and child, 2% based diagnoses on parent and teacher reports, and 9% collected data from the child, a parent, and a teacher. Three additional studies (7%) did not provide data on the informant. The children and adolescents with an anxiety disorder included in 49% of the studies were seeking treatment whereas the re- 2 Note that the total exceeds 26 because some studies contributed effect sizes to multiple comparisons. 559

4 SELIGMAN, OLLENDICK, LANGLEY, BALDACCI Table 1. Effect Sizes Comparing Youth With Anxiety Disorders to Youth Without a Disorder, by Instrument Instrument k n Hedges D SD 95% CI All 23 1, RCMAS Total RCMAS Physiological Arousal RCMAS Worry/Oversensitivity RCMAS Concentration STAIC Trait STAIC State CBCL Internalizing Note: CI = confidence interval; RCMAS = Revised Children s Manifest Anxiety Scale; STAIC = State Trait Anxiety Inventory for Children; CBCL = Child Behavior Checklist. maining studies reported data for youth that were either recruited for the study (35%) or included both help-seeking and recruited children (16%). Differentiating Youth With Anxiety Disorders From Youth Without Disorders A total of 23 studies yielded effect sizes that compared children and adolescents with anxiety disorders (i.e., youth with one or more anxiety disorders only, youth with one or more anxiety disorders and an affective disorder, youth with one or more anxiety disorders and an externalizing disorder, or a combination of these) to a nonpsychiatric control group. To minimize nonindependence of observations, when studies produced multiple effect sizes, either because they yielded effect sizes for multiple instruments or because they included multiple diagnostic groups for which effect sizes could be calculated, the effect sizes were averaged so that each study contributed only once to the mean effect size. The weighted mean effect size from the 23 studies was (95% confidence interval [CI] = 1.10 to 1.37). According to Hemphill s (2003) analysis of the distribution of effect sizes found in the psychological assessment literature, small effects sizes are those below.43, medium effects fall between.43 and.70, and a large effect is above.70. Thus, an effect size of 1.23 would be considered a large effect. To examine the possibility that this effect size could be due to a bias toward publishing only studies finding significant results, Rosenthal s fail-safe N (Rosenthal, 1979) was calculated. The fail-safe N represents the number of studies averaging null results that would be needed to reduce the effect size to a just significant (p =.05) effect. The Rosenthal s fail-safe N of 3,015 suggests that the observed effect size is likely to be a highly reliable estimate even accounting for publication bias. 3 This effect size was recalculated excluding one outlier. The difference in the overall effect size was negligible. All effect sizes were positive, meaning that the group with anxiety disorders reported greater symptomatology than the group without a disorder. However, a heterogeneity analysis suggested that moderating variables were likely to have contributed to the variability present in the effect sizes estimates, Q T (22) 54.81, p <.01. To investigate age and sex as possible moderators, two correlations coefficients were computed. The correlation between the observed effect size in each study and the mean age of the sample was computed; additionally, the ratio of boys to girls in each sample was calculated and correlated with the observed effect size. Missing data resulted when age or sex data were not available for all effect size estimates in a study. Neither the correlation between age, r(16) =.02, p >.05, or sex, r(12) =.38, p >.05, was statistically significant. Moreover, when age and sex were entered simultaneously in a regression equation predicting effect sizes, the two predictors together were not significant. Given the small sample size, however, the magnitude of the relation between sex and observed effect size may be potentially meaningful. The negative relation between effect size and sex (i.e., the proportion of boys included in the sample) suggests that larger effect sizes were found for samples that included a higher proportion of girls. The possibility that the heterogeneity in effect sizes may have resulted from disparities in the ability of the individual instruments to discriminate youth with an anxiety disorder from youth with no disorder was also investigated. Mean effect sizes for each of the instruments examined were calculated separately. Again, nonindependence of data was avoided by averaging effect sizes for each study so that a study contributed only once to the mean effect size for each instrument; however, if a study included data on more than one instrument, the study contributed one effect size estimate to the mean effect size for each of the various instruments. When the effect sizes from these studies were separated out to examine specific instruments, the 23 studies yielded 38 effect sizes for each of the different instruments and instrument subscales of interest. Results are shown in Table 1. As can be seen in Table 1, 560

5 ASSESSMENT OF CHILD AND ADOLESCENT ANXIETY many effect sizes are based on a small number of studies and therefore should be interpreted with caution. A heterogeneity analysis suggested significant variability in the effect sizes across instruments, Q T (37) = 143.4, p <.05; however, a one-way analysis of variance did not reveal significant differences among instruments, F(6, 31) =.36, p >.05. Moreover, effect sizes for all of these instruments and instrument subscales fall within the large range according to Hemphill s (2003) guidelines. Moderator analyses were then conducted to determine whether the informant providing information for diagnosis was related to effect size and more specifically whether there was an Informant Instrument interaction. No significant effects were found. Additional analyses were conducted to determine whether the method of diagnosing youth (e.g., structured interview, consensus diagnosis) or recruitment method (i.e., help-seeking or recruited) was related to effect size. Again, no significant effects were found. Due to the small number of studies that contributed to the effect size for most instruments and missing data on age and sex, exploratory moderator analyses were conducted only for the RCMAS total score. Again, although the correlations between effect size and age and effect size and sex were not statistically significant, r(10) =.36, p >.05 and r(8) =.14, p >.05, respectively, the magnitude of the correlations suggests these may be meaningful relations despite the inadequate power to find a statistically significant relation. Differentiating Youth With Anxiety Disorders from Psychiatric Controls In clinical practice, often the most difficult questions center on discriminating youth with anxiety disorders from children and adolescents with other psychiatric disorders that often have overlapping symptoms with the anxiety disorders. Therefore, we compared children and adolescents with anxiety disorders to youth diagnosed with either one or more externalizing disorders or an affective disorder (in the absence of a co-occurring anxiety disorder). Again, effect sizes within studies or within instruments for each study were averaged so that a study only contributed once to each effect size. Six studies yielded effect sizes comparing youth with an anxiety disorder to youth with externalizing disorders. An overall weighted mean effect size of.54 was found, suggesting a medium effect according to Hemphill s (2003) guidelines (95% CI =.26 to.83). Rosenthal s (1979) fail-safe N indicates that 82 nonsignificant findings would need to be added to the analysis to change the effect size to zero, thus suggesting that even accounting for publication bias and missing studies the observed effect size can be considered a reliable estimate of the true effect. Although a heterogeneity analysis indicated possible moderators, Q T (5) = 38, p <.05, the small number of studies in addition to missing data for sex and age did not allow for investigation of sample or study characteristics as possible moderators. Additionally, specific instruments were not investigated as potential moderators given the small number of effect sizes available for each instrument or instrument subscale. A total of only four studies provided data that allowed for comparison of youth with anxiety disorders to nonanxious youth with an affective disorder. The weighted mean effect size was.25, suggesting a small effect (Hemphill, 2003). Moreover, the 95% CI for the observed effect size ranged from.17 to.66, suggesting that the true effect may not be significantly different than zero. The small number of effect sizes did not allow for analysis of individual instruments or other moderating variables; however, all effects were derived from the RCMAS total or subscale scores. Therefore the overall effect size is essentially the effect size for the RCMAS and its subscales, as data for the other instruments were not available. Measuring Treatment Effects Last, we examined the ability of the RCMAS, STAIC, and CBCL to measure treatment effects. A total of 17 studies reported pre- and posttreatment scores for youth with an anxiety disorder in active treatment conditions. When effect sizes for each study were averaged across instrument so that each study contributed only once, a weighted mean effect size of.76 (95% CI =.63 to.89) was found, with the positive effect size indicating that treated individuals improved. Although this would be considered a large effect according to Hemphill s (2003) guidelines, it should be noted that these were within-subjects analyses, which often result in larger effect sizes than between-groups comparisons. Still, Rosenthal s (1979) fail-safe N of 890 suggests that this is a reliable estimate of the true effect. A heterogeneity analysis approached significance, suggesting possible moderating variables, Q T (16) = 25, p =.07. Although age and sex were not significantly related to effect size when considered separately, r(9) =.05, p >.05; r(10) =.04, p >.05, respectively, or together in a regression model, F(2, 4) =.06, p >.05, an analysis of variance examining differences between instruments did reveal a significant effect, F(6, 40) = 4.51, p <.01. Effect sizes for the individual instruments and instrument subscales can be seen in Table 2. Student Newman Keuls post hoc analyses revealed that the effect size for the internalizing scale of the CBCL was significantly higher than those found for the subscales of the RCMAS but not significantly different than the effect sizes for the STAIC scales or the RCMAS total score. Analyses examining the rela- 561

6 SELIGMAN, OLLENDICK, LANGLEY, BALDACCI Table 2. Effect Sizes Comparing Pretreatment and Posttreatment Scores of Youth With an Anxiety Disorder, by Instrument Instrument k n Hedges D SD 95% CI All RCMAS Total RCMAS Physiological Arousal RCMAS Worry/Oversensitivity RCMAS Concentration STAIC Trait STAIC State CBCL Internalizing Note: CI = confidence interval; RCMAS = Revised Children s Manifest Anxiety Scale; STAIC = State Trait Anxiety Inventory for Children; CBCL = Child Behavior Checklist. tion of effect size to method of diagnosis, recruitment method, informant, and the Informant Instrument interaction revealed no other significant effects. Discussion Results suggest that the RCMAS, STAIC, and CBCL do well in discriminating between children and adolescents with anxiety disorders and youth without a disorder and moderately well in measuring treatment response and discriminating youth with anxiety disorders from those with externalizing disorders. Moreover, the analyses suggest little difference among the instruments investigated, even between parent-reported internalizing problems, including anxiety (on the CBCL) and children and adolescents self-reports of anxiety. This finding suggests that decisions regarding which of these instruments to use can be based on factors other than the ability of the instruments to differentiate between diagnostic groups and detect treatment changes (e.g., ease of use, availability of reporter). Effect sizes of the magnitude found in this study suggest that research comparing youth without a diagnosis to youth with an anxiety disorder on the CBCL internalizing scale, the RCMAS, or the STAIC should employ samples of approximately 20 to 30 children or adolescents (10 to 15 per group) to obtain adequate statistical power (i.e.,.80 to.90). Studies comparing youth with an anxiety disorder to youth with an externalizing disorder would need to use much larger samples sizes around 70 children or adolescents per group to obtain adequate statistical power. Similarly, researchers using these instruments to examine pre post changes in youth with anxiety disorders would be well advised to employ sample sizes of at least 20 to 25 participants. For the clinician using these instruments the question is much more difficult in that it involves drawing conclusions based on individual data and not on group data. Of course, each of these instruments provides normative data and guidelines for identifying extreme scores. The question here is whether differences on these instruments can be useful in assisting clinicians in making decisions related to diagnostic status and treatment change for the individual child. On the one hand, effect sizes for detecting treatment gains and distinguishing between youth with anxiety disorders and youth without a disorder or youth with an externalizing disorder are in the range Cohen (1988) suggested could be detected in the course of normal experience (p. 26). On the other hand, our analysis of the overlap of scores for the populations in question suggests that clinicians would require a good deal of experience using these instruments with youth evidencing a wide range of symptomatology before differences in individual scores related to diagnosis and treatment gains would likely be readily detectable. The degree of overlap also suggests that establishing statistically based cutoff scores for diagnosis would likely result in a high rate of errors. As such, we would recommend that for clinical and research purposes these instruments can be useful as one piece of data in confirming diagnosis or monitoring treatment outcome; however, cutoff scores cannot be confidently used as a time-saving device to supplant careful diagnostic interviewing and comprehensive assessment. Results also suggest that the instruments investigated may be a better indicator of anxiety disorder for girls than for boys. This may be due in part to the effect of social desirability on boys reports and outward expressions of anxiety. For example, Dadds et al. (1998) found that high scores on the lie scale of the RCMAS were associated with teacher child discrepancies in judgments ofanxietyforboysbutnotforgirls.however,despitethe widespread use of the RCMAS and the availability of the lie scale, very little research has been done on its validity or potential for aiding in the interpretation of self-reported anxiety. We also found some evidence to suggestthatthercmasmaybemoreusefulwithyounger children than older children and adolescents. However, given the small number of studies employed in this analysis and previous research to the contrary (cf. Joiner, Schmidt, & Schmidt, 1996; Pina et al., 2001), we must await replication of this finding before concluding that the RCMAS has decreased utility with older children and adolescents. Despite Lonigan et al. s (1994) finding to the contrary, our results suggest that youth with anxiety disorders and affective disorders do not respond differen- 562

7 ASSESSMENT OF CHILD AND ADOLESCENT ANXIETY tially on the RCMAS and its subscales, the only instrument for which sufficient data were available. There are several reasons, aside from the small sample size, why this may have occurred. First is the high rate of comorbidity between anxiety and depression in youth (Nottelmann & Jensen, 1995; Seligman & Ollendick, 1998). Although comparisons included in the reported effect sizes were between youth with an anxiety disorder and youth with an affective disorder but no comorbid anxiety disorder, it may be that these depressed youth still evidenced subclinical levels of anxiety. This may be a circular argument, however, given that much of the literature documenting the overlap of anxiety and affective disorders in youth relies partially on instruments such as those investigated here, which do not appear to be effective in discriminating between these two psychiatric disorders. This leads to a second important point although the self-report instruments included in this review were those specific to anxiety, many instruments for anxiety query about symptoms that might be more readily classified as affective symptoms or negative affectivity. In fact, when Seligman and Ollendick (1998) investigated the overlap of self-report instruments of anxiety with the Children s Depression Inventory (CDI; Kovacs, 1985), they found that approximately 21% of RCMAS items overlapped with the CDI. The picture was mixed when examining the STAIC, with about 10% of the STAIC State scale overlapping with the CDI and 25% of the STAIC Trait scale overlapping with the CDI. So the limited utility of the RCMAS in discriminating diagnostic groups within the internalizing disorders may, in fact, represent real difficulties in the construct validity of these instruments rather than with the construct validity of the anxiety and affective disorders themselves or issues with comorbidity. It remains to be seen whether newer instruments such as the MASC, the Screen for Child Anxiety Related Emotional Disorders (Birmaher et al., 1997), and the Spence Children s Anxiety Scale (Spence, 1998), which were not represented in our study, but which purport to map more directly onto DSM disorders, demonstrate greater discriminant validity. To date, however, it must be noted that the discriminant validity of these instruments may not be significantly better. For example, although the MASC has received some support in this regard, Manassis, Tannock, Mendlowitz, Laslo, and Masellis (1997) reported that children with anxiety disorders, normal controls, and children with ADHD did not differ on the total score or the four factor scores. Limitations Quantitative reviews provide some advantages over qualitative reviews in that they provide a systematic way of combining data, and inclusionary and exclusionary criteria are explicitly stated and therefore open to evaluation; however, several limitations of such reviews remain. Foremost is that these reviews are limited by the type of data reported in the studies included. Because meta-analyses and effect size calculations have recently become more popular in the social sciences, recently published articles are more likely to include the data required to do these calculations than older studies. Even with recent studies, however, data are often not reported for the subsamples of interest (e.g., diagnostic groups, boys vs. girls, children vs. adolescents), and thus the ability to answer questions related to these variables is limited. For example, although we attempted to explore the influence of several demographic characteristics on effect sizes, we were able to do this only in a very limited way. Moreover, any systematic covariation of moderating variables across studies may affect results; however, in a study of this type, with a limited literature to draw from, which has included a wide variety of samples, it is difficult to completely account for this possibility. Additionally, the differential reliability of the instruments in the different samples we included could in part account for the range of effect sizes we observed; however, given that studies typically did not report reliability estimates for the specific samples we examined, we were unable to test this hypothesis. Conclusions Although Myers and Winters (2002) qualitative review casts doubt on the utility of some of the older instruments used to assess anxiety, we exert caution based on our quantitative review of these same instruments. In fact, we found considerable support for the RCMAS, STAIC, and CBCL in discriminating between youth with anxiety disorders and youth without a disorder, as well as between youth with anxiety disorders and youth with externalizing disorders. Only in discriminating between youth with anxiety disorders and youth with affective disorders were these instruments found wanting. We would suggest that the jury is still out on the discriminant validity of the recently developed instruments, although again we stress that initial findings are indeed promising, especially for the Screen for Child Anxiety Related Emotional Disorders and the MASC. We would also suggest that it is premature to throw out the old instruments at this time. From a psychometric standpoint, they are the only ones we have that have stood the test of time. References References marked with an asterisk indicate studies included in the meta-analysis. Achenbach, T. M. (1991). Manual for the Child Behavior Checklist/4 18 and 1991 profile. Burlington: University of Vermont, Department of Psychiatry. 563

8 SELIGMAN, OLLENDICK, LANGLEY, BALDACCI American Psychiatric Association. (1980). The diagnostic and statistical manual of mental disorders (3rd ed.). Washington, DC: Author. American Psychiatric Association. (1987). The diagnostic and statistical manual of mental disorders (3rd ed., rev.). Washington, DC: Author. American Psychiatric Association. (1994). The diagnostic and statistical manual of mental disorders (4th ed.). Washington, DC: Author. *Barrett, P. M., Dadds, M. R., & Rapee, R. M. (1996). Family treatment of childhood anxiety: A controlled trial. Journal of Consulting and Clinical Psychology, 64, *Beidel, D. C. (1991). Social phobia and overanxious disorder in school age children. Journal of the American Academy of Child & Adolescent Psychiatry, 30, *Beidel, D. C., Turner, S. M., & Morris, T. L. (2000). Behavioral treatment of childhood social phobia. Journal of Consulting and Clinical Psychology, 68, *Bernstein, G. A. (1991). Comorbidity and severity of anxiety and depressive disorders in a clinical sample. Journal of the American Academy of Child & Adolescent Psychiatry, 30, *Bernstein, G. A., Borchardt, C. M., Perwien, A. R., Crosby, R. D., Kushner, M. G., Thuras, P. D., et al. (2000). Imipramine plus cognitive behavioral therapy in the treatment of school refusal. Journal of the American Academy of Child & Adolescent Psychiatry, 39, *Bernstein, G. A., & Garfinkel, B. D. (1986). School phobia: The overlap of affective and anxiety disorders. Journal of the American Academy of Child Psychiatry, 25, Birmaher, B., Khetarpal, S., Brent, D., Cully, M., Balach, L., Kaufman, J., et al. (1997). The Screen for Child Anxiety Related Emotional Disorders (SCARED): Scale construction and psychometric characteristics. Journal of the American Academy of Child & Adolescent Psychiatry, 36, *Chansky, T. E. (1997). Social expectancies and self-perceptions in anxiety-disordered children. Journal of Anxiety Disorders, 11, Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates, Inc. Dadds, M. R., Perrin, S., & Yule, W. (1998). Social desirability and self-reported anxiety in children: An analysis of the RCMAS Lie Scale. Journal of Abnormal Child Psychology, 26, *Dadds, M. R., Spence, S. H., Holland, D. E., Barret, P. M., & Laurens, K. R. (1997). Prevention and early intervention for anxiety disorders: A controlled trial. Journal of Consulting and Clinical Psychology, 65, *Deblinger, E., McLeer, S. V., & Henry, D. (1990). Cognitive behavioral treatment for sexually abused children suffering post-traumatic stress: Preliminary findings. Journal of the Academy of Child & Adolescent Psychiatry, 29, Fiske, S. T., & Taylor, S. E. (1991). Social cognition (2nd ed.). New York: McGraw-Hill. *Flannery-Schroeder, E. C., & Kendall, P. C. (2000). Group and individual cognitive behavioral treatments for youth with anxiety disorders: A randomized clinical trial. Cognitive Therapy and Research, 24, Grills, A. E., & Ollendick, T. H. (2002). Issues in parent child agreement: The case of structured diagnostic interviews. Clinical Child and Family Psychology Review, 5, *Halperin, J. M., Newcorn, J. H., Matier, K., Sharma, V., McKay, K. E., & Schwartz, S. (1993). Discriminant validity of attention-deficit hyperactivity disorder. Journal of the American Academy of Child & Adolescent Psychiatry, Hedges, L. V., & Olkin, I. (1985). Statistical methods for meta-analysis. New York: Academic. Hemphill, J. F. (2003). Interpreting the magnitudes of correlation coefficients. American Psychologist, 58, Joiner, T. E., Schmidt, K. L., & Schmidt, N. B. (1996). Low-end specificity of childhood measures of emotional distress: Differential effects for depression and anxiety. Journal of Personality Assessment, 67, *Kashani, J. H., Vaidya, A. F., Soltys, S. M., Dandoy, A. C., Katz, L. M., & Reid, J. C. (1990). Correlates of anxiety in psychiatrically hospitalized children and their parents. American Journal of Psychiatry, 147, *Kendall, P. C. (1994). Treating anxiety disorders in children: Results of a randomized clinical trial. Journal of Consulting and Clinical Psychology, 62, *Kendall, P. C., Flannery-Schroeder, E., Panichelli-Mindel, S. M., Southam-Gerow, M., Henin, A., & Warman, M. (1997). Therapy for youths with anxiety disorders: A second randomized clinical trial. Journal of Consulting and Clinical Psychology, 65, *King, N. J., Tonge, B. J., Heyne, D., Pritchard, M., Rollings, S., Young, D., et al. (1998). Cognitive behavioral treatment of school-refusing children: A controlled evaluation. Journal of the American Academy of Child & Adolescent Psychiatry, 37, *Klein, R. G., Koplewicz, H. S., & Kanner, A. (1992). Imipramine treatment of children with separation anxiety disorder. Journal of the American Academy of Child & Adolescent Psychiatry, 31, Kovacs, M. (1985). Children s Depression Inventory. Psychopharmacology Bulletin, 21, *Last, C. G. (1991). Somatic complaints in anxiety disordered children. Journal of Anxiety Disorders, 5, *Last, C. G., Hansen, C., & Franco, N. (1998). Cognitive behavioral treatment of school phobia. Journal of the American Academy of Child & Adolescent Psychiatry, 37, *Lonigan, C. J., Carey, M. P., & Finch, A. J. (1994). Anxiety and depression in children and adolescents: Negative affectivity and the utility of self-reports. Journal of Consulting and Clinical Psychology, 62, *Lubow, R. E., Toren, P., Laor, N., & Kaplan, O. (2000). The effects of target and distractor familiarity on visual search in anxious children: Latent inhibition and novel pop-out. Journal of Anxiety Disorders, 14, Manassis, K., Tannock, R., Mendlowitz, S., Laslo, D., & Masellis, M. (1997). Distinguishing anxiety disorders psychometrically. Journal of the American Academy of Child & Adolescent Psychiatry, 36, March, J. S. (1997). Multidimensional Anxiety Scale for Children: Technical manual. North Tonawanda, NY: Multi-Health Systems. March, J. S., & Albano, A. M. (1996). Assessment of anxiety in children and adolescents. In L. J. Dickstein, M. B. Riba, & J. M. Oldham (Eds.), American Psychiatric Press review of psychiatry (Vol. XV, pp ). Washington, DC: American Psychiatric Press. *McLeer, S. V., Deblinger, E., Atkins, M. S., Foa, E. B., & Ralphe, D. L. (1988). Post-traumatic stress disorder in sexually abused children. Journal of the American Academy of Child & Adolescent Psychiatry, 27, *Mendlowitz, S. L., Manassis, K., Bradley, S., Scapillato, D., Meiezitis, S., & Shaw, B. F. (1999). Cognitive behavioral group treatments in childhood anxiety disorders: The role of parental involvement. Journal of American Academy of Child & Adolescent Psychiatry, 38, *Messer, S. C., & Beidel, D. C. (1994). Psychosocial correlates of childhood anxiety disorders. Journal of the American Academy of Child & Adolescent Psychiatry, 33, *Moradi, A. R., Taghavi, R., Neshat-Doost, H. T., Yule, W., & Dalgleish, T. (2000). Memory bias for emotional information in children and adolescents with posttraumatic stress disorder: A preliminary study. Journal of Anxiety Disorders, 14,

9 ASSESSMENT OF CHILD AND ADOLESCENT ANXIETY *Muris, P., Meesters, C., Merckelbach, H., Sermon, A., & Zwakhalen, S. (1998). Worry in normal children. Journal of the American Academy of Child & Adolescent Psychiatry, 37, *Muris, P., Merckelbach, H., Gadet, B., Moulaert, V., & Tierney, S. (1999). Sensitivity for treatment effects of the Screen for Child Anxiety Related Emotional Disorders. Journal of Psychopathology and Behavioral Assessment, 21, Myers, K., & Winters, N. C. (2002). Ten-year review of rating scales: II. Scales for internalizing disorders. Journal of the American Academy Child & Adolescent Psychiatry, 41, Nottelmann, E. D., & Jensen, P. S. (1995). Comorbidity of disorders in children and adolescents developmental perspectives. In T. H. Ollendick & R. J. Prinz (Eds.), Advances in clinical child psychology (Vol. 17, pp ). New York: Plenum. *Ollendick, T. H., Seligman, L. D., & Butcher, A. T. (1999). Does anxiety mitigate the behavioral expression of severe conduct disorder? Journal of Anxiety Disorders, 13, *Perrin, S., & Last, C. G. (1992). Do childhood anxiety measures measure anxiety? Journal of Abnormal Child Psychology, 20, Pina, A. A., Silverman, W. K., Saavedrea, L. M., & Weems, C. F. (2001). An analysis of the RCMAS lie scale in a clinic sample of anxious children. Journal of Anxiety Disorders, 15, *Pliszka, S. R., Hatch, J. P., Borcherding, S. H., & Rogeness, G. A. (1993). Classical conditioning in children with attention deficit hyperactivity disorder (ADHD) and anxiety disorders: A test of Quay s model. Journal of Abnormal Child Psychology, 21, *Pliszka, S. R., Maas, J. W., Javors, M. A., Rogeness, G. A., & Baker, J. (1994). Urinary catecholamines in attention-deficit hyperactivity disorder with and without comorbid anxiety. Journal of the American Academy of Child & Adolescent Psychiatry, 33, *Rapee, R. M. (2000). Group treatment of children with anxiety disorders: Outcome and predictors of treatment response. Australian Journal of Psychology, 52, Reynolds, C. R., & Richmond, B. O. (1985). Manual for the Revised Children s Manifest Anxiety Scale. Los Angeles: Western Psychological Services. Rosenthal, R. (1979). The file drawer problem and tolerance for null results. Psychological Bulletin, 86, *Sallee, F. R., Richman, H., Sethuraman, G., Dougherty, D., Sine, L., & Altman-Hamamdzic, S. (1998). Clonidine challenge in childhood anxiety disorder. Journal of the American Academy of Child & Adolescent Psychiatry, 37, *Sallee, F. R., Sethuraman, G., Sine, L., & Liu, H. (2000). Yohimbine challenge in children with anxiety disorders. American Journal of Psychiatry, 157, Seligman, L. D., & Ollendick, T. H. (1998). Comorbidity of anxiety and depression in children and adolescents: An integrative review. Clinical Child and Family Psychology Review, 1, *Silverman, W. K., Kurtines, W. M., Ginsburg, G. S., Weems, C. F., Lumpkin, P. W., & Carmichael, D. H. (1999). Treating anxiety disorders in children with group cognitive behavioral therapy: A randomized clinical trial. Journal of Consulting and Clinical Psychology, 67, *Silverman, W. K., Kurtines, W. M., Ginsburg, G. S., Weems, C. F., Rabian, B., & Serafini, L. T. (1999). Contingency management, self-control, and education support in the treatment of childhood phobic disorders: A randomized clinical trial. Journal of Consulting and Clinical Psychology, 67, *Southam-Gerow, M. A., & Kendall, P. C. (2000). A preliminary study of the emotion understanding of youths referred for treatment of anxiety disorders. Journal of Clinical Child Psychology, 29, Spence, S. H. (1998). A measure of anxiety symptoms among children. Behaviour Research and Therapy, 36, Spielberger, C. D. (1973). Manual for the State Trait Anxiety Inventory for Children. Palo Alto, CA: Consulting Psychologists Press. *Stark, K. D., Humphrey, L. L., Crook, K., & Lewis, K. (1990). Perceived family environments of depressed and anxious children: Child s and maternal figure s perspectives. Journal of Abnormal Child Psychology, 18, *Szollos, A. A. (1998). Early life experiences of anxious children: Risk factors in the aetiology of child anxiety disorders. Unpublished master s thesis, Macquarie University, Sydney, Australia. *Tannock, R., Ickowicz, A., & Schachar, R. (1995). Differential effects of Methylphenidate on working memory in ADHD children with and without comorbid anxiety. Journal of the American Academy of Child & Adolescent Psychiatry, 34, *Toren, P., Wolmer, L., Rosental, B., Eldar, S., Koren, S., Lask, M., et al. (2000). Case series: Brief parent child group therapy for childhood anxiety disorders using a manual-based cognitive behavioral technique. Journal of the American Academy of Child & Adolescent Psychiatry, 39, *Tracey, S. A., Chorpita, B. F., Douban, J., & Barlow, D. H. (1997). Empirical evaluation of DSM IV generalized anxiety disorder criteria in children and adolescents. Journal of Clinical Child Psychology, 26, *Turner, S. M. (1991). Vulnerability and risk for anxiety disorders. Journal of Anxiety Disorders, 5, *Vasey, M. W., Daleiden, E. L., Williams, L. L., & Brown, L. M. (1995). Biased attention in childhood anxiety disorders: A preliminary study. Journal of Abnormal Child Psychology, 23, Wolf, F. M. (1986). Meta-analysis: Quantitative methods for research synthesis. Beverly Hills, CA: Sage. Received September 18, 2002 Accepted February 3,