An empirical analysis of the BASC Frontal Lobe/Executive Control scale with a clinical sample

Archives of Clinical Neuropsychology 21 (2006) 495 501 Abstract An empirical analysis of the BASC Frontal Lobe/Executive Control scale with a clinical sample Jeremy R. Sullivan a,, Cynthia A. Riccio b a University of Texas at San Antonio, Department of Counseling, Educational Psychology, and Adult & Higher Education, 501 West Durango Blvd., San Antonio, TX 78207-4415, United States b Texas A&M University, Department of Education Psychology, College Station, TX 77843-4225, United States Accepted 30 May 2006 Data from a clinical sample of children and adolescents were used to examine the characteristics of the Frontal Lobe/Executive Control (FLEC) scale of the Behavior Assessment System for Children Parent Rating Scales, including preliminary evidence of the scale s clinical utility and relationship to other behavioral measures of executive function and characteristics of Attention Deficit Hyperactivity Disorder (ADHD). Results indicate that participants in the ADHD and other clinical groups received very similar scores on the FLEC scale, and both groups were rated significantly higher in behaviors associated with executive dysfunction than were participants in the no diagnosis group. Correlational results indicate that scores on the FLEC scale were significantly correlated with scores on the Behavior Rating Inventory of Executive Function Parent Form and Conners Parent Rating Scales Revised Short Form. Based on our results, the FLEC scale appears to be a useful behavioral rating tool in the assessment of executive function that may serve as a supplement to more traditional measures. 2006 National Academy of Neuropsychology. Published by Elsevier Ltd. All rights reserved. Keywords: ADHD; Executive function; Attention; Children; Adolescents Executive function represents a complex construct that includes multiple abilities, such as working memory, planning, emotional self-regulation, attention, organization, motivation, motor control and goal-directed behaviors (Barkley, 1997; Powell & Voeller, 2004). Impairment in these abilities is thought to be central to the manifestation of ADHD (Barkley, 1997; Gioia, Isquith, Kenworthy, & Barton, 2002; Mahone et al., 2002), and appears to be related to frontal lobe dysfunction (Boucugnani & Jones, 1989). Neuropsychologists, school psychologists and clinical psychologists often are called upon for assistance with diagnosing ADHD in children and adolescents, as well as for making recommendations for interventions and modifications to be implemented in educational and clinical settings. Thus, measures of executive function are potentially important tools for psychologists as they conduct evaluations and design interventions for individuals with ADHD. Further, recent research suggests that clinical neuropsychologists frequently include assessment of executive function abilities when conducting their evaluations (Rabin, Barr, & Burton, 2005). A preliminary version of this article was presented at the annual meeting of the National Association of School Psychologists, March 2006, Anaheim, CA. Corresponding author. Tel.: +1 210 458 2408; fax: +1 210 458 2605. E-mail address: jeremy.sullivan@utsa.edu (J.R. Sullivan). 0887-6177/$ see front matter 2006 National Academy of Neuropsychology. Published by Elsevier Ltd. All rights reserved. doi:10.1016/j.acn.2006.05.008

496 J.R. Sullivan, C.A. Riccio / Archives of Clinical Neuropsychology 21 (2006) 495 501 As noted by Gioia et al. (2002), behavior rating scales, such as the Behavior Rating Inventory of Executive Function (BRIEF; Gioia, Isquith, Guy, & Kenworthy, 2000) are increasingly being used as ecological measures of deficits in executive function. Anderson, Anderson, Northam, Jacobs, and Mikiewicz (2002) found that scores on the BRIEF Parent Form were not strongly correlated with selected measures of cognitive functioning in children with neurological disorders. The authors concluded that behavioral rating scales may provide unique information regarding the behavioral dimension of executive dysfunction not provided by other assessment methods. Similarly, Vriezen and Pigott (2002) found that in their sample of children with traumatic brain injury, scores on the BRIEF Parent Form were not strongly correlated with performance-based measures of executive function. The authors speculated that when compared to performance-based measures, the BRIEF may be more sensitive to symptoms of executive dysfunction that are observed in real-world activities and situations. Based on the results of these studies, it seems reasonable to conclude that observational or indirect measures of executive function (i.e., behavior rating scales) should be considered when conducting neuropsychological evaluations due to the additional information that they provide. Indeed, incorporating behavioral observations and normative behavioral ratings into comprehensive evaluations of ADHD has been advocated by numerous experts in the field (e.g., Barkley, 1998; DuPaul & Stoner, 2003). One such scale, the Behavior Assessment System for Children Parent Rating Scales (BASC PRS; Reynolds & Kamphaus, 1992) is frequently used as a broad band behavior rating scale, as it includes scales assessing clinical problems (e.g., Hyperactivity, Attention Problems) as well as adaptive skills (e.g., Adaptability, Social Skills). In addition to the scores routinely obtained from the BASC, supplemental scales were derived that are available in the recent revision of the BASC (Reynolds & Kamphaus, 2004). The Frontal Lobe/Executive Control (FLEC) scale of the BASC PRS was recently described (Reynolds & Kamphaus, 2002) as a supplemental index that may be useful in assessing problems in frontal lobe and executive functioning. Such an index may provide information that would be helpful in diagnostic decision-making, especially in relation to ADHD and other disorders that involve deficits in executive function. In response to the potential value of the FLEC scale, as well as the lack of information on this scale, the objective of this paper is to use data from a clinical sample of children and adolescents to examine the clinical utility of the FLEC scale in assessing behaviors associated with executive function. 1. Method 1.1. Participants Participants in this study included 92 children and adolescents who were recruited for a memory, attention and planning study at a large university in the southwest. Participants for the larger study were recruited with announcements distributed to local physicians, schools, bulletin boards, a counseling center and the newspaper. The announcement indicated that the investigation focused on memory, attention and planning/problem-solving, with no direct mention of ADHD. Criteria for inclusion in the larger study were: (a) Full Scale IQ greater than or equal to 80 on the Wechsler Intelligence Scale for Children-Third Edition (WISC-III; Wechsler, 1991), (b) ability to speak and read English, (c) no history of severe head injury and (d) no previous diagnosis of schizophrenia. For purposes of the present study, participants were divided into three groups: those who met criteria for ADHD (n = 41), those who met criteria for some other clinical disorder (n = 25) and those who received no diagnosis (n = 26). Of the participants in the other clinical group, the diagnoses included learning disabilities, adjustment disorders, mood disorders, substance use disorders and conduct and oppositional defiant disorders. Within the ADHD group, 14 participants were classified as ADHD Predominantly Inattentive Type and 27 were classified as ADHD Combined Type. Interestingly, none were classified as ADHD Predominantly Hyperactive/Impulsive Type. Our sample of children and adolescents ranged in age from 9 to 15 years (M = 11.32, S.D. = 1.99); 62 (67%) were male and 30 (33%) were female. With regard to ethnicity, for the total sample 74 (80%) of the participants were White, 10 (11%) were African American, 7 (8%) were Hispanic and 1 (1%) was Asian. Detailed demographic information for the three groups is provided in Table 1.

Table 1 Demographic information for the sample by group J.R. Sullivan, C.A. Riccio / Archives of Clinical Neuropsychology 21 (2006) 495 501 497 No diagnosis (n = 26) ADHD (n = 41) Other clinical (n = 25) Gender Male 11 35 16 Female 15 6 9 Ethnicity White 19 34 21 African American 3 4 3 Hispanic 3 3 1 Asian 1 0 0 History of grade retention 1 10 6 History of special education 1 10 5 History of medication 2 30 7 Mean age in years 11.15 (2.19) 11.44 (1.99) 11.28 (1.86) Mean parent education level 15.80 (2.35) 14.83 (2.25) 14.60 (2.43) Mean Full Scale IQ 109.38 (14.22) 100.27 (9.97) 98.16 (11.71) Note. ADHD, Attention Deficit Hyperactivity Disorder. 1.2. Procedure All participants received a comprehensive psychological evaluation that included measures of cognitive ability, achievement, language, memory, executive function, attention, behavior and emotional functioning. All measures were administered consistent with standardized procedures, and were administered in random order. Participation was voluntary with consent obtained from parents and assent obtained from participants. With physician permission, participants who regularly took medication did not take their medication on the day(s) they were evaluated; evaluations were often conducted on weekends and holidays to accommodate this condition. Following completion of the evaluations, participants received a comprehensive report of the results, along with recommendations when appropriate. Based on the results of these evaluations, which were conducted in a clinic setting by licensed psychologists and doctoral students supervised by licensed psychologists, diagnoses were made independently by two raters according to DSM-IV criteria. However, raters were blind to the results of executive function measures during the diagnostic decision-making process, due to the experimental nature of many of these measures. Thus, scores on the BASC FLEC scale were not considered in reaching diagnostic decisions; rather, scores on this index were derived from the BASC PRS after the conclusion of data collection, using the procedures and normative data provided by Reynolds and Kamphaus (2002). The additional measures used in the correlational analysis included two popular parent rating scales often utilized in the assessment of executive function and ADHD characteristics: the Behavior Rating Inventory of Executive Function Parent Form (BRIEF; Gioia et al., 2000) and the Conners Parent Rating Scales Revised Short Form (CPRS; Conners, 1997). 1.3. Instruments 1.3.1. BASC FLEC scale The Frontal Lobe/Executive Control scale of the Behavior Assessment System for Children includes 18 items from the BASC Parent Rating Scales. This item set was chosen based on a survey of experts in clinical neuropsychology, in which respondents were asked to rate each PRS item with regard to the extent to which items were associated with frontal lobe and executive function (Barringer & Reynolds, 1995; Reynolds & Kamphaus, 2002). Seventeen of the 18 FLEC items are common across the child and adolescent forms of the BASC, and coefficient alphas of.84 for both forms suggest high reliability (Reynolds & Kamphaus, 2002). Examples of items on the FLEC scale include: Cannot wait to take turn, Hits other children, Forgets things, Needs too much supervision, Is easily distracted and Is easily upset. After parents completed the BASC PRS, responses to these items were combined in order to obtain a raw score and then the normative tables provided by Reynolds and Kamphaus (2002) were used to obtain T-scores.

498 J.R. Sullivan, C.A. Riccio / Archives of Clinical Neuropsychology 21 (2006) 495 501 The Executive Functioning Content scale on the new BASC-2 is based on the FLEC scale (Reynolds & Kamphaus, 2004). 1.3.2. BRIEF Parent Form The Behavior Rating Inventory of Executive Function Parent Form is an 86-item behavioral rating scale designed to measure executive function in children and adolescents aged 5 18. From the manual (Gioia et al., 2000), the eight clinical scales on the BRIEF Parent Form include: inhibit (impulse control and ability to inhibit behaviors), shift (flexibility and ability to transition appropriately), emotional control (ability to control emotional responses), initiate (ability to initiate tasks or activities appropriately), working memory (task persistence and ability to retain information while completing a task), plan/organize (ability to set goals and develop a plan of action based on anticipation of future circumstances), organization of materials (ability to keep materials and workspaces organized) and monitor (ability to assess performance and monitor behaviors). These scales combine to form the broad indexes of Behavioral Regulation (sum of raw scores for inhibit, shift and emotional control) and Metacognition (sum of raw scores for initiate, working memory, plan/organize, organization of materials and monitor), in addition to an overall score called Global Executive Composite (sum of raw scores for all eight clinical scales). The BRIEF manual provides evidence for the reliability, validity and diagnostic utility of the scales (Gioia et al., 2000), and the BRIEF has received favorable reviews (e.g., Pizzitola, 2002). 1.3.3. CPRS Short Form The Conners Parent Rating Scales Revised Short Form is a 27-item rating scale designed to assess characteristics of ADHD and oppositional behaviors in children and adolescents aged 3 17. From the manual (Conners, 1997), the four scales on the CPRS Short Form include: oppositional (tendency to break rules, have difficulty with authority and become easily annoyed or angered), cognitive problems/inattention (difficulties related to inattention, disorganization, poor task completion and academic struggles), hyperactivity (restlessness and impulsivity) and ADHD Index (used to identify children who are at-risk for a diagnosis of ADHD). Evidence for the Short Form s internal consistency, test retest reliability and factorial validity is provided in the manual (Conners, 1997). 2. Results 2.1. Group comparisons Mean T-scores and standard deviations for all three groups on the BASC FLEC scale are provided in Table 2. Higher obtained T-scores on the FLEC scale are interpreted as a higher degree of pathology or dysfunction. The ADHD and other clinical groups received very similar scores, both almost 20 points higher than the no diagnosis group. A univariate ANOVA conducted with these mean T-scores as the dependent variable and group (i.e., no diagnosis, ADHD, other clinical) as the independent variable revealed a significant main effect for group, F (2, 88) = 31.00, p <.001, partial eta squared effect size =.413. Because the overall F test was significant and the homogeneity of variance assumption was violated with this analysis, the Games Howell post hoc procedure was used, as this procedure is appropriate when the variances and sample sizes within ANOVA designs are unequal (Games & Howell, 1976; Jaccard, Becker, & Wood, 1984). Results of the post hoc analysis indicate that the mean T-scores for both the ADHD and other clinical groups were significantly higher than the mean T-score for the no diagnosis group (p <.001 for both differences), and the mean T-scores for the ADHD and other clinical groups were not significantly different from one another. Table 2 Mean T-scores on the BASC FLEC by group No diagnosis ADHD Other clinical Mean T-score 47.50 (7.70) 67.18 (8.87) 66.52 (15.00) Note. ADHD, Attention Deficit Hyperactivity Disorder; BASC, Behavior Assessment System for Children; FLEC, Frontal Lobe/Executive Control. Post hoc analysis indicates that scores for the ADHD and other clinical groups are significantly higher than the score for the no diagnosis group, p <.001.

J.R. Sullivan, C.A. Riccio / Archives of Clinical Neuropsychology 21 (2006) 495 501 499 Table 3 Correlations of BASC FLEC scores with BRIEF Parent Form and CPRS Short Form scores FLEC BRIEF Parent Form Inhibit.69 Shift.70 Emotional control.71 Initiate.66 Working memory.69 Plan/organize.73 Organization of materials.45 Monitor.71 Behavioral Regulation Index.80 Metacognition Index.76 Global Executive Composite.83 CPRS Short Form Oppositional.77 Cognitive problems/inattention.68 Hyperactivity.68 ADHD Index.63 Note. BASC, Behavior Assessment System for Children; FLEC, Frontal Lobe/Executive Control; BRIEF, Behavior Rating Inventory of Executive Function; CPRS, Conners Parent Rating Scale. All correlations are significant at the p <.001 level. 2.2. Correlational analysis To determine the extent to which scores on the BASC FLEC scale were related to scores on other behavioral measures of executive function and ADHD characteristics, FLEC scores were correlated with scores on the BRIEF Parent Form and CPRS Short Form. All three instruments are rating scales completed by a parent or guardian, raw scores on all scales are converted to T-scores and higher T-scores on all scales are interpreted as higher degrees of executive or behavioral dysfunction. The Pearson r correlations for FLEC scores and scores on the additional measures are provided in Table 3. Scores on the FLEC were significantly correlated with all of the scales on the BRIEF Parent Form and CPRS Short Form. 3. Discussion The purpose of this study was to examine the extent to which ratings on the BASC FLEC scale were related to ratings on other measures of ADHD characteristics and behaviors associated with executive dysfunction, and also to investigate differences in diagnostic group performance on the FLEC scale. With regard to group differences, children and adolescents in the ADHD and other clinical groups received significantly higher mean T-scores on the BASC FLEC scale than did participants in the no diagnosis group. At the same time, scores for the ADHD and other clinical groups were quite similar, suggesting that the executive functioning deficits measured by the FLEC scale may not be specific to children and adolescents with ADHD. This is consistent with research suggesting that executive dysfunction may be involved in psychiatric disorders other than just ADHD, such as bipolar disorder (Shear, DelBello, Rosenberg, & Strakowski, 2002) and autistic spectrum disorders (Gioia et al., 2002). As Anderson et al. (2002) observed, executive dysfunction may be implicated in multiple disorders as a result of the wide range of behavioral, emotional, social and cognitive symptoms that appear to be related to executive dysfunction. Our results seem to add support to this perspective, and suggest that the FLEC scale is useful not only in evaluating children and youth with ADHD, but in identifying behaviors associated with executive dysfunction across disorders. It is interesting to note the differences in the amount of variance in scores among the ADHD and other clinical groups, with the scores for the other clinical groups being more spread out (see standard deviations in Table 2). The high variance of scores for this group is likely explained by the diagnostic heterogeneity of the group; this heterogeneity within the other clinical group prevents any definitive conclusions about how children and adolescents with specific diagnoses other than ADHD may perform on the FLEC scale.

500 J.R. Sullivan, C.A. Riccio / Archives of Clinical Neuropsychology 21 (2006) 495 501 Results of the correlational analysis indicate that scores on the BASC FLEC scale were significantly correlated with scores on all of the scales on the BRIEF and CPRS. Highest correlations were between the FLEC and the global scales of the BRIEF (i.e., Behavioral Regulation Index, Metacognition Index and Global Executive Composite), although FLEC correlations with the CPRS scales also were high, ranging from.63 to.77. The fact that scores on the FLEC scale were significantly correlated with all of the other scales suggests that to some extent, the FLEC scale seems to measure the same dimensions that are assessed by the BRIEF and CPRS, including both the cognitive (e.g., Metacognition Index, Cognitive Problems/Inattention Scale) and behavioral (e.g., Behavioral Regulation Index, Oppositional Scale, Hyperactivity Scale) dimensions of executive dysfunction. Unfortunately, none of the participants in our study were diagnosed as ADHD Predominantly Hyperactive/Impulsive Type. Thus, comparisons among the three ADHD subtypes on performance on the BASC FLEC were not possible. Further, given the criteria for inclusion in the study, the extent to which our results are generalizable is limited to populations similar to our sample (e.g., children and adolescents of average or better cognitive ability, with the ability to speak and read English and with no history of severe head injury or schizophrenia). Given the relatively high number of children and adolescents with ADHD and the importance of considering executive function deficits in diagnosing and developing appropriate interventions for these individuals, it is advantageous to explore novel means of assessing executive function and associated behaviors. Based on our preliminary data, it appears that the FLEC scale represents an additional measure of behaviors commonly associated with executive dysfunction that is easily derived from scores on the BASC PRS, appears to discriminate between children and adolescents with and without ADHD and other psychiatric disorders, and seems to correlate highly with other parent-rated measures of executive function and ADHD characteristics. Although, we do not advocate the use of Behavioral Rating scales in place of more traditional, performance-based measures of executive function, we feel that these measures are potentially useful in comprehensive neuropsychological evaluations because they provide information about the child s behavior from the perspective of parents and teachers, which can serve as a valuable supplement to more traditional measures. Acknowledgement This research was funded in part by a grant from the National Academy of Neuropsychology awarded to the second author. References Anderson, V. A., Anderson, P., Northam, E., Jacobs, R., & Mikiewicz, O. (2002). Relationships between cognitive and behavioral measures of executive function in children with brain disease. Child Neuropsychology, 8, 231 240. Barkley, R. A. (1997). Behavioral inhibition, sustained attention, and executive functions: Constructing a unifying theory of ADHD. Psychological Bulletin, 121, 65 94. Barkley, R. A. (1998). Attention-deficit hyperactivity disorder: A handbook for diagnosis and treatment (2nd ed.). New York: Guilford. Barringer, M. S., & Reynolds, C. R. (1995). Behavior ratings of frontal lobe dysfunction. Paper presented at the annual meeting of the National Academy of Neuropsychology. Boucugnani, L. L., & Jones, R. W. (1989). Behaviors analogous to frontal lobe dysfunction in children with attention deficit hyperactivity disorder. Archives of Clinical Neuropsychology, 4, 161 173. Conners, C. K. (1997). Conners Rating Scales revised. North Tonawanda, NY: Multi-Health Systems. DuPaul, G. J., & Stoner, G. (2003). ADHD in the schools: Assessment and intervention strategies (2nd ed.). New York: Guilford. Games, P. A., & Howell, J. F. (1976). Pairwise multiple comparison procedures with unequal Ns and/or variances: A Monte Carlo study. Journal of Educational Statistics, 1, 113 125. Gioia, G. A., Isquith, P. K., Guy, S. C., & Kenworthy, L. (2000). Behavior Rating Inventory of Executive Function. Odessa, FL: Psychological Assessment Resources. Gioia, G. A., Isquith, P. K., Kenworthy, L., & Barton, R. M. (2002). Profiles of everyday executive function in acquired and developmental disorders. Child Neuropsychology, 8, 121 137. Jaccard, J., Becker, M. A., & Wood, G. (1984). Pairwise multiple comparison procedures: A review. Psychological Bulletin, 96, 589 596. Mahone, E. M., Cirino, P. T., Cutting, L. E., Cerrone, P. M., Hagelthorn, K. M., Hiemenz, J. R., et al. (2002). Validity of the Behavior Rating Inventory of Executive Function in children with ADHD and/or Tourette syndrome. Archives of Clinical Neuropsychology, 17, 643 662. Pizzitola, K. M. (2002). Test review: Behavior Rating Inventory of Executive Function. Journal of Psychoeducational Assessment, 20, 400 405. Powell, K. B., & Voeller, K. K. S. (2004). Prefrontal executive function syndromes in children. Journal of Child Neurology, 19, 785 797. Rabin, L. A., Barr, W. B., & Burton, L. A. (2005). Assessment practices of clinical neuropsychologists in the United States and Canada: A survey of INS, NAN, and APA Division 40 members. Archives of Clinical Neuropsychology, 20, 33 65.

J.R. Sullivan, C.A. Riccio / Archives of Clinical Neuropsychology 21 (2006) 495 501 501 Reynolds, C. R., & Kamphaus, R. W. (1992). Behavior Assessment System for Children. Circle Pines, MN: American Guidance Service. Reynolds, C. R., & Kamphaus, R. W. (2002). The clinician s guide to the Behavior Assessment System for Children. New York: Guilford. Reynolds, C. R., & Kamphaus, R. W. (2004). Behavior Assessment System for Children (2nd ed.). Circle Pines, MN: American Guidance Service. Shear, P. K., DelBello, M. P., Rosenberg, H. L., & Strakowski, S. M. (2002). Parental reports of executive dysfunction in adolescents with bipolar disorder. Child Neuropsychology, 8, 285 295. Vriezen, E. R., & Pigott, S. E. (2002). The relationship between parental report on the BRIEF and performance-based measures of executive function in children with moderate to severe traumatic brain injury. Child Neuropsychology, 8, 296 303. Wechsler, D. (1991). Wechsler intelligence scale for children (3rd ed.). San Antonio, TX: Psychological Corporation.