Innovations in Assessing ADHD: Development, Psychometric Properties, and Factor Structure of the ADHD Symptoms Rating Scale (ADHD-SRS)

Transcription

1 Journal of Psychopathology and Behavioral Assessment, Vol. 20, No. 4, 1998 Innovations in Assessing ADHD: Development, Psychometric Properties, and Factor Structure of the ADHD Symptoms Rating Scale (ADHD-SRS) Melissa Lea Holland,1 Gretchen A. Gimpel,1 and Kenneth W. Merrell2 Accepted: October 3, 1998 This research involved the development of a behavior rating scale designed to measure ADHD and the investigation of the scale's psychometric properties and factor structure. This scale, the ADHD Symptoms Rating Scale (ADHD-SRS), was developed for the assessment of ADHD in the school-age (K-12) population. Participants were 1006 children and adolescents (in grades K-12) who were rated by their parents and/or teachers. The results indicate that the ADHD-SRS possesses strong internal consistency reliability and test-retest reliability and moderate cross-informant reliability. The data also suggest that the ADHD-SRS has strong content validity. Convergent validity of this instrument was also high, as demonstrated by correlations with three previously validated behavior rating scales. Significant age and gender differences in ADHD symptoms were found with both the parent and teacher respondent populations. Finally, the factor analysis of the ADHD-SRS suggested a two factor oblique rotation as the best fit for both the parent and the teacher data. After a visual inspection of the items which loaded on each factor, Factor 1 was named Hyperactive-Impulsive and Factor 2 was named Inattention. These two factors, along with the items which loaded on each factor, appear to be remarkably similar to the two categories listed in the DSM-IV for ADHD. Directions for future research, as well as clinical implications and limitations of the research are discussed. KEY WORDS: attention deficit-hyperactivity disorder (ADHD); assessment; children; adolescents. 1Utah State University, Logan, Utah The University of Iowa, Iowa City, Iowa /98/ $ 15.00/ Plenum Publishing Corporation

2 308 Holland, Gimpel, and Merrell INTRODUCTION Attention deficit-hyperactivity disorder (ADHD) is one of the most frequent problems for which children are referred to mental health clinics in the United States, constituting up to half of the referrals to outpatient clinics (Cohen, Becker, & Campbell, 1990; Frick & Lahey, 1991). It is estimated that approximately 3-5% of the childhood population has ADHD (Barkley, 1990; Burnley, 1993; Fowler, 1991; McBurnett, Lahey, & Pfiffner, 1993), though some studies have reported an even higher incidence (Ross & Ross, 1982; Silver, 1992; Whitman, 1991). The Diagnostic and Statistical Manual of Mental Disorders Fourth Edition (DSM-IV) defines ADHD as "a persistent pattern of inattention and/or hyperactivity-impulsivity that is more frequent and severe than is typically observed in individuals at a comparable level of development" (APA, 1994, p. 78). These two broad areas of inattention and hyperactivity-impulsivity each consist of nine different symptoms within the DSM-IV These symptoms must be causing impairment in at least two settings (i.e., home and school), and there must be clear evidence that the symptoms interfere with appropriate academic, social or occupational functioning. The symptoms cannot be better accounted for by another mental disorder (APA, 1994). Differentiation of inattentive symptoms and hyperactive-impulsive symptoms and the addition of four behavioral descriptors in the DSM-IV is a change from the diagnostic criteria in the DSM-III-R which did not differentiate between these different types of symptoms and had fewer behavioral descriptors. This change is substantiated in the literature (Bauermeister et al., 1995; Lahey et al., 1994; Sabatino & Vance, 1994), which indicates that the 14 symptom ADHD syndrome in the DSM-III-R was not inclusive enough to explain adequately the full range of complex behaviors that can occur in ADHD. In addition to the primary symptomatology of inattention, impulsivity and hyperactivity, children with ADHD often experience other difficulties. One such difficulty is poor academic performance, with almost all children referred for ADHD typically underachieving in relation to their known ability levels (Barkley, 1990; Durbin, 1993). In addition to school performance, significant problems with oppositional and defiant behaviors, aggressiveness, and antisocial behaviors are also very prevalent in these children, with 35-60% of all ADHD clinic-referred children also meeting the criteria for oppositional defiant disorder (Barkley, 1990; Frick & Lahey, 1991) and 41-75% also meeting the criteria for conduct disorder (Frick, Strauss, Lahey, & Christ, 1993). Studies on childhood ADHD have indicated that peer relationship problems for children may be related to their inattentive, disruptive socially immature and provocative behaviors (Barkley, 1990; Frick

3 The ADHD Symptoms Rating Scale 309 & Lahey, 1991). Research has also found that these children tend to elicit negative interactions with their parents and teachers as a result of their behavior (Guevremont & Barkley, 1992). Poor self-esteem and emotional disorders may emerge as a result of chronic failure and conflict in family and social functioning (Frick & Lahey, 1991). These primary and secondary symptoms of ADHD appear to affect both males and females. However, boys are approximately three times more likely to be diagnosed with ADHD than girls (Barkley, 1990; Brown, Madan-Swain, & Baldwin, 1991). In clinic referred populations, the maleto-female ratio rises to 9:1, respectively, indicating that boys with ADHD are far more likely to be referred to clinics for evaluation and treatment than girls (APA, 1994; Barkley, 1996). Few studies have investigated gender differences in childhood ADHD symptomatology. In a study conducted by Brown et al. (1991), it was found that girls with ADHD were retained in school more frequently than boys, were more underidentified than boys, and were less aggressive than their ADHD male counterparts. Few gender differences, however, were obtained on measures of concentration and attention, intellectual functioning, academic achievement, distractibility, parent and teacher ratings of internalizing and externalizing behavioral symptoms, and social competence (Brown et al., 1991). Silverthorn, Frick, Kuper, and Ott (1996) also found no differences across gender on measures of neurological and cognitive status. In a study of situational variability conducted by Breen and Altepeter (1990), no clear gender differences were found in children identified as ADHD. Barkley (1990) noted that, in general, girls may have fewer conduct problems and may be less aggressive than boys, but otherwise appear to be little different in their pattern of ADHD symptoms. More research should be conducted in order to define more clearly the differences that exist, if any, between male and female ADHD symptomatology (Faraone, Biederman, Keenan, & Tsuang, 1991). Much of the literature indicates that ADHD symptomatology changes as children with ADHD progress in age (Barkley, 1990, 1996; Sleator, 1986; Wender, 1987). The symptoms of ADHD often first appear in the preschool years (Barkley, 1996). Children with ADHD at this age level are described by parents as always on the go, restless, acting as if driven by a motor, and frequently getting into or climbing on things. These children often have injuries as a result of these overactive and impulsive behaviors (Barkley, 1990). Preschoolers with ADHD are also characterized as having a short attention span, being unable to pay attention to activities for any length of time (Wender, 1987), and being talkative and noisy (Barkley, 1996). The hyperactive and impulsive behaviors of the preschool years persist as the child with ADHD enters elementary school. Elementary-age children with ADHD often are restless in their seats, fidgeting and squirming during

4 310 Holland, Gimpel, and Merrell school or home work time (Wender, 1987). These children also begin to have difficulties with sustained attention, including forgetfulness and distractibility (Barkley, 1996). With the increased amount of homework and school supplies to keep track of, elementary age children with ADHD often are disorganized and do not follow through on many tasks and activities. It is often during this time that the child with ADHD also begins to experience social rejection from both peers and adults (Barkley, 1990). In adolescence there is often a decline in ADHD symptomatology as reported on behavior rating scales. However, simply because the severity levels of symptoms may decline with age does not necessarily mean that children with ADHD are outgrowing their disorder (Barkley, 1996). Instead, it is estimated that approximately 50-80% of all clinic-referred children for ADHD will continue to have ADHD symptomatology into their teenage years (Barkley, Fischer, Edelbrock, & Smallish, 1990), and 30-50% of children with ADHD continue to exhibit ADHD symptoms in adulthood (Silver, 1992; Weiss & Hechtman, 1986). As children mature, the symptoms of ADHD usually become less conspicuous. For example, signs of excessive gross motor activity (excessive climbing or running) may be confined instead to feelings of inner restlessness or fidgetiness (APA, 1994). It is thought that adolescents and adults with ADHD may develop adaptive skills to cope with ADHD symptomatology but that they may still face the same core symptoms of inattention, hyperactivity, and impulsivity. One of the first steps in helping children who exhibit ADHD symptomatology is to conduct an assessment to detect if the child does have ADHD (Durbin, 1993). Unfortunately, there is no simple test that can detect whether the child has this disorder (Fowler, 1991). A diagnosis of ADHD is typically made by a clinician after comparing the results of various assessment measures against the definitional criteria for the disorder. The most frequently used assessment methods for the identification of ADHD in children are attentional and cognitive tasks, interviews, observational methods, and rating scales (Barkley, 1990; Guevremont & Barkley, 1992). Rating scales offer numerous advantages over the other assessment methods (Barkley, 1990; Sleator, 1986). For example, rating scales permit data collection of infrequent behaviors that are likely to be missed by observations (Barkley, 1990), and they can be used to gather information from those who have been responsible for the care and management of the child across different situations and settings (Barkley, 1990; Blondis, Snow, Stein, & Roizen, 1991). They are also relatively easy to administer and inexpensive (Ross & Ross, 1982). Rating scales often have normative data available for establishing the statistical significance of the child's behaviors (Guevremont & Barkley, 1992; Guevremont, DuPaul, & Barkley, 1993), and they permit the quantification of qualitative aspects of behavior which are often

5 The ADHD Symptoms Rating Scale 311 difficult or impossible to obtain through interviews, cognitive tasks, or direct observation (Barkley, 1990; Sleator, 1986). Although rating scales offer numerous advantages, many of the currently existing scales are not adequate for assessing ADHD (Reid, Maag, & Vasa, 1993). Some of them were developed before the publication date of the DSM-IV, and therefore are not based on the current criteria most commonly used for the diagnosis of ADHD in children. Many of the rating scales have unreported or inadequate reliability and validity (Reid et al., 1993). Some of the rating scales focus on other disorders along with ADHD, which may not generate an in-depth and complete assessment of ADHD. Although some new scales have been developed based on the DSM-IV criteria, these scales vary in how long or thorough they are, and each scale is bound to have its own "instrument variance." Instrument variance entails the phenomenon of different rating scales often measuring related, but slightly different constructs, and, as a result, a severe behavior problem score on one rating scale may be equivalent to only a moderate behavior problem score on another scale. Also, if the normative populations for scoring comparisons are not representative of the population as a whole or not randomly selected, similar results on two different rating instruments may not mean the same thing. One way to control for instrument variance is by using the aggregation principle, in which more than one type of rating scale is used for assessing problem behaviors. The use of this principle helps to reduce response bias and variance problems in the assessment through obtaining rating evaluations from different sources and by using more than one type of rating scale in the assessment (Merrell, 1994). For this reason it is good for researchers and practitioners to have several instruments to choose among. Therefore, there is still a need for additional assessment instruments to be developed. The ADHD Symptoms Rating Scale (ADHD-SRS) is a new research instrument designed to assess the symptoms of attention deficit-hyperactivity disorder in the K-12 school age population. The ADHD-SRS offers numerous advantages over other currently existing ADHD behavior rating scales. The purpose of this article is to detail the development of the ADHD-SRS and to present research evidence regarding its psychometric properties and factor structure. METHOD Instrument Development The initial development of the items for the ADHD-SRS utilized a rational-theoretical approach to test construction (Lanyon & Goldstein, 1982).

6 312 Holland, Gimpel, and Merrell A thorough review of the literature on ADHD in children and adolescents was conducted to facilitate the selection of potential items to represent the two DSM-IV domains of ADHD: inattention and hyperactivity-impulsivity. Eighty-one symptom descriptors were developed through this search. These descriptors were reviewed and corresponding items were developed. Some descriptors were eliminated due to redundancy, improper fit with the DSM- IV ADHD categories, and vagueness or complexity of the descriptor. Sixtyone potential items remained after this process. Content validation was conducted on the 61 potential items. Thirty-five expert judges in ADHD (e.g., child clinical psychologists, university professors, school psychologists, pediatricians) from 10 U.S. states were asked to rate the 61 items on a 3-point scale (2 = definitely appropriate, 1 = borderline appropriate, and 0 = inappropriate for inclusion) for (a) representation of construct, (b) appearance of gender or culture bias, and (c) appropriateness for parent and teacher judgment. Items that received less than a borderline appropriate rating were removed from the item pool, and several items were revised on the basis of qualitative feedback. This process resulted in 56 final items, with two to five items remaining for each of the 18 specific ADHD symptoms listed in the DSM-IV In addition to validity, it is also important that a measure be usable (Worthen, Borg, & White, 1993). The overall usability and item quality of the ADHD-SRS items were rated on a 5-point scale (i.e., 1 = "poor" to 5 = "excellent") by both a parent (n = 36) and a teacher (n = 21) panel. The overall usability of the scale was rated as being between adequate and excellent (mean = 4.42), and the overall quality of the items as being between adequate and excellent (mean = 4.19). Through the ratings of the content validation panel and the parent and teacher panels, a frequency of behavior rating format ("behavior does not occur" to "behavior occurs one to several times an hour") was found to be the desired rating scale format over a traditional rating scale format ("behavior never occurs" to "behavior often or to a great degree occurs). This final version of the scale was used in all following research. Subjects The preliminary normative sample for this research included 1006 children and adolescents in grades K-12 who were rated by their parents and/or teachers. A grade by gender breakdown of the subjects is presented in Table I. The sections which follow more closely examine the characteristics of this sample.

7 The ADHD Symptoms Rating Scale 313 Table I. Grade by Gender Breakdown for the Subjects (N = 1006) Grade K Total a Missing = 2. Male Gender Female Total a Geography. The subjects were obtained from six public school districts in the United States. These participating school districts were from five U.S. states with the following breakdown of subjects: California, 21.6%; Idaho, 7.8%; Kentucky, 14.0%; Ohio, 27.7%; and Utah, 28.9%. The six participating school districts were a mix of urban, suburban, small town, and rural communities. Race/Ethnicity. The racial/ethnic makeup of the ADHD-SRS preliminary normative sample was as follows: 88.4% Caucasian, 3.2% African American, 2.9% Hispanic, 2.8% Asian or Pacific Islander, and 0.4% described as "other." Approximately 2.4% of the sample did not respond to this question. The percentage of ethnic/racial minorities in the norm group was only 9.3%, compared with 30.8% of the general U.S. population, indicating that the non-white population was underrepresented in this sample (U.S. Bureau of the Census, 1990). Special Education/Previous ADHD Diagnosis. Of the 1006 children and adolescents in the preliminary normative sample, 90% were not receiving special education services and 9.2% did receive services (0.8% were unknown). This percentage approximates the estimated 12% of students nationally who receive special education services (U.S. Department of Education, 1995). In terms of specific special education service categories, 2.3% were classified as Learning Disabled, 2.1% were classified as Speech Language Disordered or Communication Disordered, 0.6% were classified as Mentally Retarded or Intellectually Disabled, 0.6% were classified as

8 314 Holland, Gimpel, and Merrell Emotionally or Behaviorally Disturbed, and 3.5% were identified as having other disability conditions. With regard to a previous diagnosis of ADHD, 83.2% of the preliminary normative group had never been diagnosed with ADHD, while 6.4% had been previously diagnosed with ADHD (for 10.4% of the sample this information was unknown). This approximates the prevalence rate of ADHD in the general population. Data Collection Procedures Data were collected with the assistance of coordinators working in public schools (school psychologists and teachers). Following district approval, coordinators distributed to teachers three ADHD-SRS protocols with an instructional sheet. Teachers were asked to complete the rating scales on the first three students on their class role and then return the completed protocols to the coordinator, who returned them to the investigators. Participating teachers were also asked to send home a packet with each child in their class containing rating scales for the parents to complete. A business-reply envelope was enclosed so that parents mailed their scales directly back to the authors. Children's names were not put on the scales. Identification numbers were assigned to the scales and the teachers were asked to match the numbers on the scales they completed to the scales the parents received so that for those children on whom both a parent and a teacher completed a scale, the results could be matched. Instructions at the top of the ADHD- SRS directed participants to complete an informational section provided in the packet about their child or student (i.e., age, sex, grade, if the child is receiving special education services, if the child has been diagnosed with ADHD, etc.) and to complete the ADHD-SRS carefully, without skipping any items, and deciding how often the child being rated has demonstrated the behaviors in the past 3 months. Convergent Validity Measures During the collection of data from the preliminary normative group, several other ADHD behavioral rating scales were also administered to subgroups of the normative population to obtain information on the convergent construct validity of the instrument. These measures are described as follows: ADDES. The Attention Deficit Disorders Evaluation Scale (ADDES) is a behavior rating instrument based on the DSM-IV criteria designed to assess ADHD symptoms in the child and adolescent populations. The AD- DES has two subscales, Inattentive and Hyperactive-Impulsive. Two ver-

9 The ADHD Symptoms Rating Scale 315 sions of this rating scale exist: a home version (46 items) and a school version (60 items). The school version was normed on a total of 5795 American students ages 4-19, and the home version was normed on a total of 2415 children and youth ages Adequate psychometric properties are reported for the scale in the ADDES manuals (McCarney, 1995). CTRS-39. The Conners' Teacher Rating Scale (CTRS-39) is a 39-item behavior rating instrument. The CTRS-39 rating format involves responding with one of the following four responses to the items: not at all, just a little, pretty much, or very much. This rating scale has six subscales, including Hyperactivity, Conduct Problem, Emotional-Overindulgent, Anxious-Passive, Asocial, and Daydream-/Attention Problem. In addition, the scale contains a Hyperactivity Index, a collection of 10 items from the other CTRS-39 subscales that were found to be especially sensitive to pharmacological treatment effects with ADHD children. The CTRS-39 was normed on over 9,500 Canadian children. Separate norms are available for both age and gender. Adequate psychometric properties are reported for the scale in the CTRS-39 manual (Conners, 1990). AD/HD Rating Scale-IV The AD/HD Rating Scale-IV is an 18-item behavior rating scale based on the DSM-IV criteria for ADHD. This scale, developed by DuPaul, Anastopoulos, Power, Murphy, and Barkley (1996) includes both home and school versions. The AD/HD Rating Scale-IV contains two subscales, the Inattention Scale and the Hyperactivity-Impulsivity Scale, which are summed to calculate the Total Score of the items. Normative data are available for children and adolescents between 5 and 18 years old (kindergarten through 12th grade) and were obtained from over 2000 teachers and 4500 parents in a national sample. The normative group reportedly closely matched the 1990 U.S. Census data for distribution across regions and ethnic groups. Adequate test-retest reliability (>.75 for 4-week interval) and internal consistency (>.80) has been reported by the authors. Scores of both the home and school version correlate significantly with the Conners' Parent and Teacher Rating Scales, and confirmatory factor analyses support the two-factor model that conforms to the DSM-IV breakdown of symptoms (DuPaul et al., 1996). RESULTS Factor Structure Exploratory factor analyses of the ADHD-SRS were conducted separately for both the parent and teacher ratings. These analyses are discussed below.

10 316 Holland, Gimpel, and Merrell Parent Ratings. A total of 650 parent ratings were collected. Protocols missing item responses were excluded from the analyses, resulting in a total of 559 parent ratings used in the parent factor analyses. There appeared to be no particular pattern or trend regarding which responses were missing on these protocols. This sample size exceeds the 4:1 or 5:1 (subjects to variables) ratio recommended for exploratory factor analysis (Floyd & Widaman, 1995). One common guideline regarding the number of factors is to extract factors with eigenvalues greater than 1.00 (Tabachnik & Fidell, 1989). Using this guideline, a maximum of five factors would be retained. Numerous exploratory principal component factor analyses were conducted, using both orthogonal and oblique rotations and retaining between one and five factors. A principal-component two factor-specified direct oblique rotation resulted in the fewest double loadings and appeared to be the most clinically interpretable. This two-factor solution resulted in only five double loadings. The first factor, consisting of 34 items, accounted for 29.9% of the variance (eigenvalue = 29.29). This factor was labeled Hyperactive-Impulsive, as it consisted primarily of items related to hyperactivity and impulsivity (e.g., "restless or overactive," "makes excessive noise," "blurts out," "has difficulty waiting turn in line"). The second factor, consisting of 27 items, accounted for 27.9% of the variance (eigenvalue = 3.12). This factor was labeled Inattention, as it consisted primarily of items related to being inattentive (e.g., "is disorganized with school work or homework assignments," "is forgetful (forgets things)," "has difficulty remaining on task," "does not organize activities"). The factor structure of this rotation is presented in Table II. The correlation between the two factors was.69. Teacher Ratings. A total of 432 teacher ratings were collected. Again, protocols missing item responses were excluded from the analyses, resulting in a total of 392 teacher ratings used in the teacher sample factor analyses. There appeared to be no particular pattern or trend regarding which responses were missing on thee protocols. This sample size also exceeds the 4:1 or 5:1 (subjects to variables) minimum ratio. Principal-component factor analyses with both orthogonal and oblique rotations were conducted. Again, initially only factor with eigenvalues greater than 1.00 were extracted (Tabachnik & Fidell, 1989). As in the analyses of parent ratings, item-level data for the 392 teacher ratings were subjected to both orthogonal and oblique rotation methods, with a range of one to five factors specified. A principal component two factor specified direct oblique rotation resulted in the fewest double loadings and appeared to be the most clinically interpretable. This two-factor solution resulted in only four double loadings. The first factor, consisting of 35 items, accounted for 38.3% of the variance (eigenvalue = 35.91).

11 The ADHD Symptoms Rating Scale 317 Table II. Two-Factor Oblique Rotation Factor Structure for Parent Ratings Item Factor 1 Factor 2 1) Has a short attention span 2) Talks too much 3) Loses things that he/she needs 4) Needs to have questions and directions repeated 5) Has difficulty delaying gratification 6) Fidgets and squirms 7) Gets "out of control" when playing 8) Makes excessive noise 9) Bothers others when they are trying to work or play 10) Unable to tolerate delays 11) Becomes overexcited 12) Blurts out 13) Rushes through chores or tasks 14) Does not hear all of what has been said 15) Has difficulty sitting appropriately on furniture 16) Does not prepare for school assignments 17) Rocks in seat 18) Has difficulty waiting in turn in line 19) Restless or overactive 20) Has difficulty following rules of games or activities 21) Shifts from one activity to another 22) Does not follow the necessary steps in order to complete things 23) Makes odd or annoying noises 24) Produces messy or sloppy school work 25) Has difficulty sustaining play activities 26) Does not organize activities 27) Leaves seat without permission 28) Does not finish projects that he/she has started 29) Has difficulty remaining on task 30) Make careless mistakes 31) Runs in the halls/runs in the house 32) Does not follow directions 33) Interferes with other's activities 34) Is easily distracted 35) Asks irrelevant questions 36) Does not seem to listen to what others are saying 37) Dislikes doing things that require sustained mental effort 38) Is forgetful (forgets things) 39) Interrupts others when they are talking 40) Calls out answers before the question is finished 41) Has difficulty taking turns 42) Has difficulty remaining seated 43) Is inattentive 44) Talks at inappropriate times 45) Acts as if "driven by a motor" 46) Gives up easily 47) Has difficulty concentrating 48) Always "on the go" 49) Cannot find things that he/she needs 50) Moves about unnecessarily

12 318 Holland, Gimpel, and Merrell Item Table II. Continued Factor 1 Factor 2 51) Has difficulty playing or working quietly 52) Moves about while seated 53) Fails to complete school work or homework 54) Shifts position in seat 55) Is disorganized with school work or homework 56) Climbs on things Percentage of variance Correlation between 2 factors =.69 Note. Factor loadings of less than.30 are left blank This factor was labeled Hyperactive-Impulsive, as it consisted primarily of items relating to hyperactivity and impulsivity (e.g., "acts as if driven by a motor," "becomes overexcited," "blurts out," "has difficulty waiting turn in line"). The second factor, consisting of 25 items, accounted for 31.8% of the variance (eigenvalue = 3.70). This factor was labeled Inattention, as it consisted primarily of items related to being inattentive (e.g., "fails to complete school work or homework," "has a short attention span," "has difficulty remaining on task," "is inattentive"). The factor structure of this rotation is presented in Table III. The correlation between the two factors was.69, and the overall factor structure was virtually identical to the structure obtained with parent ratings. Reliability Reliability is defined as the stability or consistency of an instrument (Borg & Gall, 1989). Internal consistency and temporal stability are the two primary types of reliability which are extensively discussed in the literature (Cronbach, 1990). These different forms of test reliability for the ADHD-SRS, along with cross-informant information, are presented in this section. Internal Consistency. Internal consistency reliability was assessed by computing Cronbach's coefficient alpha for the ADHD-SRS total score (i.e., the sum of all items) for the 650 parent ratings and 432 teacher ratings. The obtained coefficient for the parent data was.98, and the coefficient alpha for the teacher data was.99. These coefficients are exceptionally strong. Test-Retest. The temporal stability (test-retest reliability) of the ADHD-SRS was calculated using ratings from teachers of 78 elementary

13 The ADHD Symptoms Rating Scale 319 Table III. Two-Factor Oblique Rotation Factor Structure for Teacher Ratings Item Factor 1 Factor 2 1) Has a short attention span 2) Talks too much 3) Loses things that he/she needs 4) Needs to have questions and directions repeated 5) Has difficulty delaying gratification 6) Fidgets and squirms 7) Gets "out of control" when playing 8) Makes excessive noise 9) Bothers others when they are trying to work or play 10) Unable to tolerate delays 11) Becomes overexcited 12) Blurts out 13) Rushes through chores or tasks 14) Does not hear all of what has been said 15) Has difficulty sitting appropriately on furniture 16) Does not prepare for school assignments 17) Rocks in seat 18) Has difficulty waiting in turn in line 19) Restless or overactive 20) Has difficulty following rules of games or activities 21) Shifts from one activity to another 22) Does not follow the necessary steps in order to complete things 23) Makes odd or annoying noises 24) Produces messy or sloppy school work 25) Has difficulty sustaining play activities 26) Does not organize activities 27) Leaves seat without permission 28) Does not finish projects that he/she has started 29) Has difficulty remaining on task 30) Make careless mistakes 31) Runs in the halls/runs in the house 32) Does not follow directions 33) Interferes with other's activities 34) Is easily distracted 35) Asks irrelevant questions 36) Does not seem to listen to what others are saying 37) Dislikes doing things that require sustained mental effort 38) Is forgetful (forgets things) 39) Interrupts others when they are talking 40) Calls out answers before the question is finished 41) Has difficulty taking turns 42) Has difficulty remaining seated 43) Is inattentive 44) Talks at inappropriate times 45) Acts as if "driven by a motor" 46) Gives up easily 47) Has difficulty concentrating 48) Always "on the go" 49) Cannot find things that he/she needs 50) Moves about unnecessarily

14 320 Holland, Gimpel, and Merrell Item Table HI. Continued 51) Has difficulty playing or working quietly 52) Moves about while seated 53) Fails to complete school work or homework 54) Shifts position in seat 55) Is disorganized with school work or homework 56) Climbs on things Percentage of variance Correlation between 2 factors =.69 Note. Factor loadings of less than.30 are left blank. Factor Factor and middle school students in the ADHD-SRS preliminary normative sample. Teachers completed ratings of the students at a 2-week time interval. Pearson product-moment correlations between the scores of the two administrations of the ADHD-SRS were calculated. The resulting coefficients for the ADHD-SRS were as follows:.96 for the Hyperactive-Impulsive subscale,.95 for the Inattention subscale, and.97 for the total score. These coefficients are exceptionally high in magnitude, and are equal to or higher than stability coefficients reported for other ADHD rating scales. For example, at 1-month test-retest intervals, the Attention Deficit Disorders Evaluation Scale (ADDES; McCarney, 1995a, b) coefficients ranged between.88 and.91 for the home version and.90 and.94 for the school version, and for the Conners' Teacher Rating Scale (Conners, 1990) coefficients ranged between.72 and.91 for a 30-day test-retest interval. These results show that the ADHD-SRS provides stable ratings over short time intervals and that ADHD behavioral characteristics are generally stable across short time intervals. Cross-Informant. A sample size of 76 children and adolescents was utilized to calculate the cross-informant correspondence for the ADHD-SRS. A Pearson product-moment correlation was calculated between the parent and teacher ratings, resulting in a correlation of.26 (p =.02). This correlation approximates the average correlation of.28 obtained in most studies for cross-informant ratings (Achenbach, McConaughy, & Howell, 1987). Validity In the jointly produced Standards for Educational and Psychological Testing (APA, 1985), it is stated that "validity is the most important consideration in test evaluation" (p. 9). Validity refers to the extent of how

15 The ADHD Symptoms Rating Scale 321 well an instrument measures what it is purported to measure (Anastasi, 1988). Content validity for the ADHD-SRS was previously discussed under Instrument Development. This section explores the face and convergent validity of the ADHD-SRS. Face. Face validity is the degree to which an instrument appears to measure what it purports to measure (Borg & Gall, 1989). The two-factor structure obtained for the ADHD-SRS is remarkably similar to the DSM-IV categories for ADHD. An objective visual inspection of the items which loaded on each factor and their relationship to the DSM-IV categories was conducted. Because the DSM-IV categories for ADHD were originally used as a guideline for the ADHD-SRS item inclusion, each item on the ADHD-SRS had previously been categorized into one of the two DSM-IV ADHD categories. In examining face validity, if the items which loaded on each factor appeared to be directly related to the corresponding DSM-IV category (i.e., "is inattentive" is related to the DSM-IV category of inattention, but "fidgets and squirms" is related to the category of hyperactivity-impulsivity), then that item was counted as being directly related to the corresponding DSM- IV category. Percentages of these corresponding items were calculated for each factor for both the parent and the teacher data. The majority of the items which loaded on each factor were directly related to the corresponding DSM-IV category (i.e., the items which loaded on the Hyperactive-Impulsive Factor represent primarily the DSM-IV ADHD category of hyperactivity-impulsivity, not the category of inattention). For the parent ratings, 93% of the items that loaded on Factor 1 (Hyperactive-Impulsive) were directly related to the DSM-IV category of hyperactivity-impulsivity. For Factor 2 (Inattention), 98% of the items were directly related to the DSM-IV category of inattention. For the teacher ratings, 91% of the items that loaded on Factor 1 (Hyperactive-Impulsive) were directly related to the corresponding DSM-IV category of hyperactivity-impulsivity, and on Factor 2 (Inattention), 96% of the items were directly related to the DSM-IV category of inattention. It is important to note that almost all of the ADHD-SRS items which did not correspond to the appropriate DSM-IV ADHD category for each factor were double loadings (i.e., they loaded on both Factor 1 and Factor 2). The majority of these double-loaded items loaded higher on the factor with which they appeared to belong [i.e., "Is Inattentive" on the parent ratings rotation loaded at.54 on Factor 2 (Inattention), while it loaded at only.35 on factor 1 (Hyperactive-Impulsive)]. Convergent. The convergent validity of the ADHD-SRS was investigated by calculating Pearson product-moment correlation coefficients between the ADHD-SRS subscales (Hyperactive-Impulsive and Inattention) and total score and (a) the Attention Deficit Disorders Evaluation Scale (ADDES),

16 322 Holland, Gimpel, and Merrell home and school versions; (b) the Conners' Teacher Rating Scale (CTRS- 39); and (c) the ADH/HD Rating Scale-IV, home and school versions. ADDES. The ADDES's two subscales (Inattention and Hyperactive- Impulsive) and total score were correlated with the two subscales and total score of the ADHD-SRS. The sample for these analyses included ratings of 124 children and adolescents (96 subjects rated by teachers and 28 subjects rated by parents). Separate analyses were conducted for both parent and teacher ratings. The resulting correlations are displayed in Table IV For the teacher-rated subjects, all correlations between the ADDES and the ADHD-SRS were above.80. For the parent-rated subjects, all obtained correlations were above.84. CTRS-39. The sample for the correlational comparison between the CTRS-39 and the ADHD-SRS included teacher ratings of 63 children. Data were obtained for grades K-3 only. The CTRS-39's six subscales, Hyperactivity Index, and total score were correlated with the two subscales and total score of the ADHD-SRS. Resulting correlations are displayed in Table V The Conduct Problem, Emotional-Indulgent, Asocial, and Daydream- Attention Problem subscales on the CTRS-39 all correlated at or above.71 with the ADHD-SRS total score. The Hyperactivity subscale and Hyperactivity Index on the CTRS-39 were found to correlate highly with the ADHD-SRS's Hyperactive-Impulsive subscale, Inattention subscale, and total score. The Inattention subscale on the ADHD-SRS is thought to have correlated highly with the Hyperactivity subscale and the Hyperactivity Index on the CTRS-39 because the Hyperactivity subscale and the Hyperactivity Index also measure some inattentive behaviors (i.e., Table IV. Correlations Between the ADHD-SRS Subscales and Total Score and the Attention Deficits Disorder Evaluation Scale (ADDES), Home and School Versions Scale ADDES (school) Inattentive scale Hyperactive-Impulsive scale ADDES total score ADDES (home) Inattentive scale Hyperactive-Impulsive scale ADDES total score Hyperactive-Impulsive subscale Note. All correlations are significant at p <.001. ADHD-SRS Inattention subscale Total score

17 The ADHD Symptoms Rating Scale 323 Table V. Correlations Between the ADHD-SRS Subscales and Total Score and the Conners' Teacher Rating Scale (CTRS-39) CTRS-39 scale Hyperactive-Impulsive subscale ADHD=SRS Inattention subscale Total score Hyperactiviry Conduct Problems Emotional-Indulgent Anxious-Passive Asocial Daydream-Attention Hyperactivity Index Total score Problem Note. AH correlations are significant at p < a Inattentive-Easily Distracted," "Fails to Finish Things S/he Starts-Short Attention Span"). The Anxious-Passive subscale, however, only had a weak correlation with the ADHD-SRS total scores (.30) and subscales (.35 for the Hyperactive-Impulsive subscale and.24 for the Inattention subscale). This finding is to be expected as the Anxious-Passive subscale clearly appears to be measuring a different construct (i.e., leadership skills and submissiveness) than the ADHD-SRS. These low correlations actually provide evidence of the discriminant construct validity of the ADHD-SRS. AD/HD Rating Scale-IV. The sample for the correlational comparison of the AD/HD Rating Scale-IV, home and school versions, and the ADHD- SRS included ratings of 129 children and adolescents (43 subjects rated by parents and 89 subjects rated by teachers). Separate analyses were conducted for both parent and teacher ratings. The AD/HD Rating Scale-IV's two subscales (Inattention and Hyperactivity-Impulsivity) and total score were correlated with the subscales and total score of the ADHD-SRS. Resulting correlations are displayed in Table VI. For the parent-rated subjects, the AD/HD Rating Scale-IV's subscales and total score were found to correlate at or above.86 with the ADHD- SRS total score. The ADHD-SRS Inattention subscale correlated at.89 with the AD/HD Rating Scale-IV Inattention Scale, while the ADHD-SRS Hyperactive-Impulsive subscale and the Hyperactivity-Impulsivity Scale on the AD/HD Rating Scale-IV correlated at.90. For the teacher-rated subjects, the AD/HD Rating Scale-IV's subscales and total score were found to correlate at or above.84 with the ADHD- SRS total score. The ADHD-SRS Inattention subscale correlated at.94 with the AD/HD Rating Scale-IV Inattention Scale, while the ADHD-SRS

18 324 Holland, Gimpel, and Merrell Table VI. Correlations Between the ADHD-SRS Subscales and Total Score and the AD/HD Rating Scale-IV, Home and School Versions Scale Hyperactive-Impulsive subscale ADHD-SRS Inattention subscale Total score AD/HD Rating Scale-IV (home) Inattention scale Hyperactivity-Impulsivity scale AD/HD Rating Scale-IV total score AD/HD Rating Scale-IV (school) Inattention scale Hyperactivity-Impulsivity scale AD/HD Rating Scale-IV total score Note. All correlations are significant at p <.001. Hyperactive-Impulsive subscale and the Hyperactivity-Impulsivity Scale on the AD/HD Rating Scale-IV correlated at.93. Gender Differences To determine if significant gender differences were present for parent and teacher ratings of children on the ADHD-SRS, t-tests and effect size estimates were calculated. As shown in Table VII, parents' ratings of subjects on the ADHD-SRS were significantly different for male and female subjects, with male subjects obtaining a higher mean rating on the ADHD- SRS than female subjects. Teachers' ratings were also significantly different for male and female subjects. Again, male subjects obtained a higher mean rating than did female subjects for teacher ratings on the ADHD-SRS. Effect size estimates were calculated between the male and female samples for both parent and teacher ratings to help determine the practical meaning of the score differences. This procedure was done by using the standard procedure of dividing the difference in group means by the pooled standard deviation for both groups (Cohen, 1988). Results are shown in Table VII. For the parent ratings, males were rated approximately one-third of a standard deviation higher on the ADHD-SRS than were females (ES =.28). For the teacher ratings, males were rated more than one-half of a standard deviation higher than were females (ES =.50). According to Cohen's (1988) paradigm for effect size power analysis, the parent ratings effect size difference is considered to be of a significant but small magnitude, whereas the teacher ratings effect size difference is considered to be of a medium magnitude.

19 The ADHD Symptoms Rating Scale 325 Rater Parent Teacher Table VII. t-tests and Effect Size Correlations Between Teacher- and Parent-Rated Male and Female Subjects M Males SD M Females SD t df P ES Relationship of Age on ADHD-SRS Ratings) To determine the effect of children's ages on the ADHD-SRS ratings, Pearson bivariate correlations were computed between the ages of the subjects and the total scores they received by parents and teachers. The resulting correlation coefficient for both the parent and the teacher ratings was -.20 (p <.0010). These results indicate that as the subjects get older, their obtained scores by parent and teacher raters on the ADHD-SRS tend to diminish. These correlations were statistically significant, but small. Means and standard deviations for the ADHD-SRS total scores for children and adolescents rated by parents and teachers at three separate grade levels (K-5, 6-8, 9-12) were also calculated. As shown in Table VIII, as the subjects get older, their obtained total scores on the ADHD-SRS become lower. Effect size estimates were calculated for this data and are presented in Table IX. Effect sizes were largest between the K-5 and the 9-12 grade levels. DISCUSSION Psychometric Characteristics Data presented in this article regarding the factor structure of the ADHD-SRS indicates that a two-factor solution is the most appropriate and clinically interpretable structure for both parent and teacher respondent populations. This two-factor solution utilizing an oblique rotation had the fewest double loadings and was the most interpretable. The factors were named Hyperactive-Impulsive and Inattention, following a visual inspection of the content of the items which loaded on each factor. Factor 1 consisted of 34 items in the parent data analyses and 35 items in the teacher data analyses primarily relating to hyperactivity and impulsivity (i.e., "makes excessive nose," "blurts out," "fidgets and squirms," "restless or overactive").

20 326 Holland, Gimpel, and Merrell Table VIII. Means and Standard Deviations for the ADHD-SRS Total Score by Grade Level Grade level K n Parent ratings M SD n Teacher ratings M SD Table IX. Effect Size Estimates Matrix for the Means and Standard Deviations for the ADHD-SRS Total Scores by Grade Level-Parent and Teacher Ratings Grade level K Effect size 9-12 Parent ratings K Teacher ratings K (small).35 (small).94 (large).58 (medium).54 (medium).13 (not significant) Factor 2 consisted of 27 items in the parent data analyses and 25 items in the teacher data analyses primarily to relating inattention (i.e., "has a short attention span," "has difficulty remaining on task," "is inattentive," "fails to complete homework or school work"). One aspect of the factor analyses that should be mentioned is the moderate correlation between the two factors for both parent and teacher samples. This finding indicates that hyperactivity, impulsivity, and inattentiveness are not distinct, separate behaviors, but instead are moderately interrelated. The two-factor structure obtained for the ADHD-SRS appears, upon objective visual inspection, to be remarkably similar to the two DSM-IV categories for ADHD: inattention and hyperactivity-impulsivity. These similarities of the ADHD-SRS factor structure with the DSM-IV categories provide further face validity for the ADHD-SRS as the current criteria used to diagnose ADHD in the childhood population are the categories located in the DSM-IV for ADHD. The evidence indicates that the ADHD-SRS has strong internal consistency and test-retest reliability and moderate cross-informant reliability. The obtained alpha coefficients for the ADHD-SRS total score for parent

21 The ADHD Symptoms Rating Scale 327 and teacher ratings were.98 and.99 respectively. Virtually any general guideline for interpreting internal consistency coefficients would indicate that these internal consistency reliability estimates reflect very strong internal consistency reliability for the ADHD-SRS. High internal consistency means that each item in the ADHD-SRS is tapping the same construct (i.e., ADHD) (Mitchell & Jolley, 1988). The temporal stability found for the ADHD-SRS was also exceptionally high. Finally, the cross-informant correlation between parent and teacher ratings approximates the average correlation obtained in most studies for cross-informant ratings (Achenbach et al., 1987). The data also suggest that the ADHD-SRS has strong content, convergent, and construct validity. The content validation procedures utilizing expert judges and the usability ratings of parents and teachers provides evidence for the validity and usability of the ADHD-SRS. The strong relationships found between the ADHD-SRS and three criterion rating scales provide substantial supporting evidence of the construct validity of the ADHD-SRS as a measure of ADHD symptomatology. In addition, discriminant construct validity of the ADHD-SRS was also obtained through the weak correlations found between the ADHD-SRS and the Anxious-Passive subscale on the CTRS-39, which clearly appears to be measuring a different construct (i.e., leadership skills and submissiveness) than the ADHD-SRS. In this research, there was a general tendency for boys to be rated higher on the ADHD-SRS than girls of the same age. This tendency was true for both parent and teacher responses and was evidenced at all grade levels. These differences are also reflected in the literature with the higher prevalence rate of ADHD found for boys than for girls (Barkley, 1990). In addition, other rating scales currently used to assess for ADHD among the school-age population have also found this phenomenon of males receiving higher subscale and total score ratings than for females of the same age (Conners, 1990; DuPaul et al., 1996). As evidenced by the data, as the subjects get older, their obtained scores by parent and teacher raters on the ADHD-SRS tend to diminish. In other words, subjects' obtained total scores on the ADHD-SRS became smaller at later grade levels (i.e., K-12). This finding should not be surprising. In later childhood and adolescence, there is often a decline in ADHD symptomatology as reported on behavior rating scales (Barkley, 1996). This phenomenon may be due to a change in an individual's symptomatology (i.e., from "hyperactivity" to "a feeling of inner restlessness"), or it may be that adolescents with ADHD are able to develop adaptive coping skills to help them better manage their symptomatology. In any case, this phenomenon of decreasing scores on ADHD behavior rating scales with age has been well documented (Barkley, 1996; Sleator, 1986).