Using the Conners Teacher Rating Scale Revised in School Children Referred for Assessment

Transcription

1 Original Research Using the Conners Teacher Rating Scale Revised in School Children Referred for Assessment Alice Charach, MD, MSc, FRCPC; 1 Shirley Chen, MD, MPH; 2 Sheilah Hogg-Johnson, PhD; 3 Russell J Schachar, MD, FRCPC 4 Objective: Predictive validity of the Conners Teacher Rating Scale Revised (CTRS-R) was evaluated against a semi-structured clinical teacher interview in school children referred for diagnostic assessment of attention-deficit hyperactivity disorder (ADHD). We hypothesized that extreme scale values would increase diagnostic certainty and that classification errors would be associated with comorbid conditions. Method: Children (n = 1038), aged 6 to 12 years, were screened using the CTRS-R and their teachers were interviewed. Three levels of T scores on the 3 Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM-IV) subscales of the CTRS-R were compared with DSM-IV symptom thresholds by interview. Where subscale scores and interviews showed highest agreement, presence of comorbid disruptive behavioural disorders, reading disability, language impairment, and low IQ were investigated for children classified correctly, compared with incorrectly. Results: T scores of 60 and above on all CTRS-R DSM-IV subscales offered high sensitivity, from 91% to 94%. Only on subscales M (hyperactive impulsive) and N (total) did T scores of less than 60 offer posttest probabilities of less than 10%, confirming that a child does not reach diagnostic threshold by interview. T scores of 80 and more offered high specificity, from 88% to 93%, but did not provide high posttest probabilities that children reach diagnostic criteria. Classification errors were associated with more language impairment among false positives than true positives on the M (18.9%, compared with 11.3%, P = 0.04) and N (19.0%, compared with 9.5%, P = 0.023) subscales, and more reading disabilities among false positives than among true positives on the N subscale (35.2%, compared with 21.6%, P = 0.009). Conclusions: The ability of the CTRS-R to predict whether clinically referred children reach DSM-IV criteria for ADHD at school is limited. Can J Psychiatry. 2009;54(4): Clinical Implications As expected, low scores on DSM-IV teacher rating scales of the CTRS-R effectively rule out presence of impairing symptoms reaching diagnostic threshold at school in clinically referred children. High scores on the teacher rating scales do not confirm presence of impairing symptoms reaching diagnostic threshold at school in clinically referred children. Errors are more likely among girls with language and reading impairments and young boys with oppositional behaviours. Limitations The criteria imposed by the research design may influence the generalizability of the results as children with an IQ of less than 80, recent abuse or separations, history of perinatal problems or adoption, and comorbid physical illnesses requiring treatment were excluded. The rating scales were not compared with overall clinical diagnosis of ADHD, but with DSM-IV symptom thresholds as reported in teacher interviews. The rating scales were compared with a high threshold DSM-IV diagnostic criteria on interview, rather than a lower threshold, which could support parent-identified symptom impairment. 232 La Revue canadienne de psychiatrie, vol 54, no 4, avril 2009

2 Using the Conners Teacher Rating Scale Revised in School Children Referred for Assessment Key Words: attention-deficit hyperactivity disorder, diagnosis, teacher rating scales, teacher interviews The CTRS has been used as a tool to identify children at risk for ADHD for many years. While primarily used in research, clinicians have also used the scale when formulating diagnoses to improve diagnostic accuracy. 1 As behavioural concerns are often initially identified in school, teacher reports are vital to the clinical diagnostic assessment and are strongly recommended in clinical practice guidelines. 2,3 These guidelines recommend using rating scales, including the CTRS-R, as a practical alternative to directly interviewing the teacher during the clinical assessment procedure. 2,3 An earlier version of the scale, the CTRS-28, when combined with the Conners Parent Rating Scale 48, can discriminate between children with DSM-IV 4 ADHD and clinically referred children with other conditions resembling ADHD. 5 While the Conners rating scales are recommended for clinical purposes, users are cautioned about the risk of falsepositive and false-negative errors, and practioners are encouraged to use the CTRS-R as one of several sources of information. 6 This study evaluates 3 subscales of the CTRS-R. Subscales L, M, and N specifically identify teacher reports of DSM-IV inattentive and hyperactive impulsive items and are likely to be most relevant to ADHD diagnostic decisions. 6 Subscale L represents the 9 items for the DSM-IV inattentive subtype. Subscale M represents the 9 items for the DSM-IV hyperactive impulsive subtype, and subscale N represents the DSM-IV ADHD combined subtype. 6 Confirmatory factor analysis in a community sample of 1661 students indicated excellent internal reliability for both 9-item 6 factors and suggested ratings of inattention and hyperactivity impulsivity frequently co-occurred. The CTRS-R was designed to identify children in the community at risk for ADHD, and therefore diagnostic accuracy was evaluated only for the Conners ADHD Index. As with other diagnostic tools used to identify ADHD in school populations, 7 9 the DSM-IV subscales have not been evaluated in clinical samples. Sackett and Haynes 10 note that research questions investigating the usefulness of Abbreviations used in this article ADHD CD CELF CTRS DSM ODD TTI WRAT attention-deficit hyperactivity disorder conduct disorder Clinical Evaluation of Language Fundamentals Conners Teacher Rating Scale Diagnostic and Statistical Manual of Mental Disorders oppositional defiant disorder Telephone Teacher Interview Wide Range Achievement Test diagnostic tests follow a logical progression of study design with initial evidence for a test s validity resulting from comparisons of test characteristics in patient, compared with nonpatient, samples. Later evaluations should focus on the test s performance in clinical samples to assist clinicians who use the test to distinguish between patients with and without the disorder. As the CTRS-R DSM-IV subscales offer quantitative information on a child s variation from the population norm, it may be possible to evaluate the test s validity in clinical samples by relating degree of school impairment with T scores. High CTRS-R T scores could confirm impairment due to ADHD symptoms, while low T scores could indicate little dysfunction. Such an analysis could provide more useful diagnostic information for clinicians and allow clinical time and resources to be redirected to other aspects of patient care. 11 The ideal method of obtaining diagnostic information from teachers is the semi-structured interview. However, guidelines do not recommend routine clinical interviews of teachers, as it is resource-intensive. The current study uses the TTI-IV for DSM-IV, a reliable and valid semi-structured telephone interview with the child s teacher to obtain symptom criteria at school. 12 It provides information about a wide range of disruptive behaviour disorder symptoms in the school setting. The presence of impairment owing to specific symptoms is judged by the clinician based on teacher descriptions of behaviour in school contexts. The current research evaluates the DSM-IV subscales of the CTRS-R in a large clinical sample of children aged 6 to 12 years referred for evaluation of attention, learning, and behavioural difficulties. There are 2 primary objectives to this evaluation. First, we will evaluate the diagnostic accuracy of the DSM-IV subscales using T score cut-offs representing 1, 2, and 3 standard deviations above the population norm. These values represent atypical T scores. 6 We hypothesize that test values associated with extreme scores on the scales will improve diagnostic certainty; that is, the posttest probability of meeting diagnostic criteria at school will increase at high T score values, and the posttest probability of meeting diagnostic criteria at school is negligible at low T scores. The second objective of the study is to investigate whether we can identify specific patterns of comorbid conditions associated with classification errors. As clinical guidelines emphasize the need to identify alternative explanations for ADHD symptoms, it is important to know if the CTRS-R DSM-IV scales consistently misidentify or show systematic bias in the presence of comorbid conditions when used in clinical populations. We hypothesize that classification errors will be associated with cognitive and language difficulties, as well as teacher-identified ODD and conduct The Canadian Journal of Psychiatry, Vol 54, No 4, April

3 Original Research problems. As the basis for evaluation of the CTRS-R DSM-IV subscales, we have chosen DSM-IV symptom criteria obtained in the clinical interview with the teacher. The teacher-identified symptoms are a more proximal target than the clinical diagnosis of ADHD, and therefore are more likely to be highly correlated with the CTRS-R than the overall clinical diagnosis, which takes into account information from all sources, including parents and teachers. Methods Design The current study is a cross-sectional study measuring the diagnostic accuracy of CTRS-R DSM-IV subscales against the TTI-IV for DSM-IV, a reliable and valid semi-structured clinical interview. 12 The clinician interviewing the teacher had no access to scores on the CTRS-R. Sample Children, aged 6 to 12 years, consecutively referred between May 1996 and February 2006 for assessment of attention, learning, and behavioural problems, and for whom both CTRS-R and TTI were collected, were eligible for this study. Children were referred to an outpatient specialty clinic in a large pediatric hospital in Toronto. The current measurement study was embedded in the larger context of a study of genetics and cognition; this sample of 1038 children has been previously described. 13 Inclusion criteria were: behavioural difficulties with inattention, hyperactivity and (or) impulsivity, living with at least one parent, parent and child willing to participate in research assessment, and the child s teacher being able to participate in assessment by telephone. Exclusion criteria were: attendance at a full-time residential or day treatment program, premature birth, history of serious head trauma, a chronic medical condition requiring ongoing medical treatment, the child was adopted, recent history of physical or sexual abuse, and parental disagreements regarding custody. In addition, children on psychotropic medications other than stimulants (antidepressants, atomoxetine, beta blockers, or atypical neuroleptics) were excluded allowing participants to be removed from their medications for observations of cognition and behaviour. Parents and children provided informed written consent and assent prior to study initiation. All procedures were approved by the hospital research ethics board. Procedure Both the CTRS-R 6 and the TTI-IV for DSM IV 12 were obtained prior to assessment. During the assessment, all children received a thorough diagnostic evaluation. Parents were interviewed using the Parent Interview for Child Symptoms 14 for DSM-IV modelled on the Kiddie-Schedule for Affective Disorders and Schizophrenia, 15 with increased probes regarding disruptive behaviour disorder diagnoses. The psycho-educational and language screen used the Wechsler Intelligence Scale for Children-III and -IV, 16 CELF-3, 17 and the WRAT All interviewers and assessors had no knowledge of children s scores on the CTRS-R or the TTI-IV until after diagnostic measures and interviews were completed. Though final clinical diagnoses included information from all sources, specific diagnostic information reported in our study reflects information provided by either the teacher informant or gathered through child psychometric evaluation. Children were diagnosed with ODD and CD if they met DSM-IV criteria by TTI-IV. Children with a reading disability were identified by a full-scale IQ greater than 90 and scores less than 86 on at least 2 of 3 scales on the WRAT-3 (Read, Word Attack, or Word ID), or a score of less than 78 on any 1 of the 3 scales. Children with a language impairment were identified by scores of less than 85 on both CELF-3 receptive and expressive language scales, or a score of less than 78 on either 1 of the 2 scales. Children with a full-scale IQ of less than 85 were classified as low IQ. Measures TTI-IV for DSM-IV 12 is a reliable and valid semi-structured clinical interview for obtaining teacher descriptions of child behaviour in classroom and schoolyard settings. The clinician judges presence and (or) absence of impairing behaviours (inattention, hyperactivity, impulsiveness, opposition, defiance, and aggression) resulting in symptom counts consistent with DSM-IV ADHD, ODD, and CD. Interrater reliability is high, and the interview shows good convergent and divergent validity with standardized teacher-reported measures of impairment and child classroom behaviours. CTRS-R 6 is a reliable and valid 59-item teacher self-report form designed to identify children with ADHD and associated behavioural difficulties. Each item can be scored from 0 to 3; where 0 represents an item is not present and 3 represents an always or definitely present symptom. There are 12 subscales, of which 3 (subscales L, M, and N) are designed to identify DSM-IV subtypes. Raw scores have been converted to T scores representing scale values relative to population norms for age and sex. A T score of 50 is the mean score for the population. 6 A T score of 60 is the subscale score equivalent to 1 standard deviation above the mean. Similarly, a T score of 70 is equivalent to 2 standard deviations above the mean, and a T score of 80 is 3 standard deviations above the population mean. Parents gave the CTRS-R questionnaire to teachers to fill out and return to the clinic. In this sample, a minority of children (18.7 %) were using psychostimulants at the time of the assessment. For those children who were on psychostimulants, the parents were instructed to withhold it for 2 school days so that the teacher 234 La Revue canadienne de psychiatrie, vol 54, no 4, avril 2009

4 Using the Conners Teacher Rating Scale Revised in School Children Referred for Assessment Table 1 Sample characteristics, n = 1038 Age, years, mean (SD) 8.8 (2.1) Sex, male, % 75.5 DSM-IV ADHD subtypes a by TTI, % Inattentive 43.0 Hyperactive impulsive 27.0 Combined 17.0 Cognitive disorders, b % Reading disability 27.0 Language impairment 14.1 IQ< Externalizing disorders, a % ODD 13.9 CD 4.0 a Meeting DSM-IV criteria by semi-structured clinical interview with teacher b Child assessment could observe and score child behaviour off medication. This same observation period formed the basis for teachers to answer questions in the clinical interview. The child was also off medication on the day of the assessment for observation by clinicians. The 2-day stimulant-free observation period was chosen as a feasible compromise that would allow sufficient time for the teacher to obtain good observations, yet be short enough so that the child s ongoing functioning at school was not compromised. Data Analysis Objective 1 Quantify Diagnostic Accuracy for Extreme Values of the CTRS-R. For each of the 3 DSM-IV subscales, the scores were divided into 4 levels as follows: T < 60, T = 60 to 69, T =70to 79, T 80. Sensitivity, specificity, and likelihood ratios for presence and absence of DSM-IV symptom criteria at school are derived for each level. The thresholds for meeting DSM-IV symptom criteria at school on the TTI were as follows: for subscale L, 6/9 inattentive symptoms; for subscale M, 6/9 hyperactive impulsive symptoms; and for subscale N, both 6/9 inattentive and 6/9 hyperactive impulsive symptoms. As an additional evaluation of the CTRS-R DSM-IV subscales, the above analysis was repeated comparing scores on any one of the subscales, L, N, or M, with the final clinical diagnosis using information from all sources. The posttest probabilities for meeting criteria were derived for each T score level by obtaining the pretest odds (pretest odds = prevalence in sample / [1 prevalence in sample]), multiplying the pretest odds by the likelihood ratio to obtain the posttest odds, and calculating the posttest probability from the posttest odds (posttest probability = posttest odds / [1 + posttest odds]). 19 The posttest probability quantifies the chances that a child with a specific test result has the outcome of interest. 11 For example, a posttest probability of greater than 90% confers a very high likelihood that the outcome is present, whereas a posttest probability of less than 10% confers a very low likelihood that the outcome is present. A posttest probability between 10% and 90% implies that additional evaluation may be required prior to making diagnostic or treatment decisions. Objective 2 Identify Comorbid Conditions Associated With Diagnostic Errors. For each of the 3 subscales, a threshold was identified within the range of T scores of about 60 through 70, the range identified by the technical manual as clinically relevant, 6 representing the T score where agreement between scale value and criterion was highest as measured by kappa statistics. The proportions of children classified correctly with ADHD symptom criteria were compared with those who were classified incorrectly on rates of teacher-reported ODD or CD diagnosis, and cognitive or learning problems (reading disability, language impairment, IQ of less than 85). For each scale, a priori contrasts between false positives and true positives and between false negatives and true negatives were evaluated using chi square (Fisher exact test, 2-sided). These analyses were repeated using parent reports of ODD and CD symptoms with similar results. We report here the analyses based on teacher information. All statistical analyses were conducted using SPSS Version 14.0 software (SSPS Inc, Chicago, IL). Results The characteristics of the sample are described in Table 1. As is typical of many clinical samples of children with disruptive behaviour problems, the majority (75.5%) were boys. Using teacher reports by TTI, just over one-half of participants (53.7%) met symptom criteria for one of the DSM-IV ADHD subtypes, 13.9% met criteria for ODD, and 4% met criteria for CD. The measures evaluating diagnostic accuracy of the CTRS-R DSM-IV subscales are summarized in Table 2. Table 2A summarizes the diagnostic accuracy statistics of the CTRS-R with teacher-reported DSM-IV symptom criteria at the 3 levels corresponding to 1, 2, and 3 standard deviations above the population mean. As expected, children who were rated T of less than 60 on the CTRS-R DSM-IV subscales were least likely to meet DSM-IV symptom criteria by teacher report. The Canadian Journal of Psychiatry, Vol 54, No 4, April

5 Original Research Table 2A Sensitivity, specificity, and likelihood ratios for CTRS-R DSM-IV subscales in clinical sample (n = 1038) Subscales, T score Standard Sensitivity (95% CI) Specificity (95% CI) LR+ a (95% CI) LR b (95% CI) TTI symptoms L DSM-IV IA c 6IASx d ( ) 0.39 ( ) 1.52 ( ) 0.19 ( ) ( ) 0.66 ( ) 2.06 ( ) 0.49 ( ) ( ) 0.93 ( ) 1.65 ( ) 0.95 ( ) M DSM-IV HI e 6HISx f ( ) 0.53 ( ) 1.92 ( ) 0.18 ( ) ( ) 0.77 ( ) 3.08 ( ) 0.38 ( ) ( ) 0.92 ( ) 3.44 ( ) 0.79 ( ) N DSM-IV Total g 6IA&6HISx ( ) 0.32 ( ) 1.37 ( ) 0.20 ( ) ( ) 0.65 ( ) 2.10 ( ) 0.42 ( ) ( ) 0.88 ( ) 1.86 ( ) 0.89 ( ) L DSM-IV IA c Clinical ADHD ( ) 0.46 ( ) 1.57 ( ) 0.37 ( ) ( ) 0.65 ( ) 1.53 ( ) 0.72 ( ) ( ) 0.92 ( ) 1.43 ( ) 0.97 ( ) M DSM-IV HI e Clinical ADHD ( ) 0.60 ( ) 1.73 ( ) 0.52 ( ) ( ) 0.84 ( ) 2.75 ( ) 0.66 ( ) ( ) 0.95 ( ) 3.41 ( ) 0.87 ( ) N DSM-IV Total g Clinical ADHD ( ) 0.48 ( ) 1.58 ( ) 0.37 ( ) ( ) 0.75 ( ) 1.94 ( ) 0.69 ( ) ( ) 0.93 ( ) 2.18 ( ) 0.91 ( ) a Likelihood ratio for postive test result (scale score above threshold) b Likelihood ratio for negative test result (scale score below threshold) c Inattentive subscale d Inattentive symptoms e Hyperactive impulsive subscale f Hyperactive impulsive symptoms g Total symptoms subscale HI = hyperactive impulsive; IA = inattentive; Sx = symptoms Indeed, as shown in Table 2B, the posttest probability of a child scoring below a T of 60 for CTRS-R subscale M and subscale N meeting DSM-IV symptom criteria by teacher clinical interview is less than 10%, and therefore highly unlikely. If the child s T score on DSM subscale L is less than 60, the posttest probability of having 6/9 DSM-IV inattentive symptoms by teacher clinical interview is 12.6% (95% CI 10.7% to 14.8%). If the child s T score on DSM subscale M is less than 60, the posttest probability of having 6/9 DSM-IV hyperactive impulsive symptoms by teacher interview is 6.3% (95% CI 5.0% to 8.0%). And, if the child s T score on DSM subscale N is less than 60, the posttest probability of having 6/9 DSM-IV inattentive symptoms and having 6/9 hyperactive impulsive symptoms by teacher interview is 3.8% (95% CI 2.8% to 5.1%). Also as hypothesized, the highest specificity, over 90% for subscales L and M and 88% for subscale N, occurs where T is 80 or more, suggesting that CTRS-R DSM-IV subscale scores above 80 are likely to result in positive DSM-IV criteria by teacher clinical report. However, on further inspection, the likelihood ratios and calculated posttest probabilities (Table 2B) do not confirm hypothesized high levels of 236 La Revue canadienne de psychiatrie, vol 54, no 4, avril 2009

6 Using the Conners Teacher Rating Scale Revised in School Children Referred for Assessment Table 2B Calculation of posttest probabilities for CTRS-R DSM-IV subscales in clinical sample (n = 1038) Subscales, T score Pretest probabiliy (%) a If test positive, b % (95% CI) If test negative, c % (95% CI) TTI symptoms L DSM-IV IA ( ) 12.6 ( ) ( ) 27.1 ( ) ( ) 42.4 ( ) M DSM-IV HI ( ) 6.3 ( ) ( ) 14.2 ( ) ( ) 22.8 ( ) N DSM-IV Total ( ) 3.8 ( ) ( ) 7.7 ( ) ( ) 15.0 ( ) Clinical ADHD L DSM-IV IA ( ) 44.1 ( ) ( ) 60.6 ( ) ( ) 67.4 ( ) M DSM-IV HI ( ) 52.5 ( ) ( ) 58.3 ( ) ( ) 65.0 ( ) N DSM-IV Total ( ) 44.1 ( ) ( ) 59.5 ( ) ( ) 66.0 ( ) a Proportion of sample meeting symptom criteria on TTI or clinical ADHD diagnosis b Probability of meeting symptom criteria if CTRS-R score is above threshold c Probability of meeting symptom criteria if CTRS-R score is below threshold HI = hyperactive impulsive; IA = inattentive diagnostic certainty. For subscale L, the posttest probability of having 6/9 DSM-IV inattentive symptoms by teacher report is 56.1% (95% CI 53.0% to 59.1%). For subscale M, the posttest probability of having 6/9 DSM-IV hyperactive impulsive symptoms by teacher report is 56.2% (95% CI 53.2% to 59.2%). And, for subscale N the posttest probability of having 6/9 DSM-IV inattentive and 6/9 DSM-IV hyperactive impulsive symptoms by teacher report is 27.0% (95% CI 24.4% to 29.8%). These results show that while specificity may be high, diagnostic accuracy is not. The CTRS-R DSM-IV subscales demonstrate limited ability to distinguish which children in a clinical sample will be reported by their teachers to have enough impairing symptoms at school to meet DSM-IV criteria. Evaluation of the CTRS-R DSM-IV subscales against the overall clinical diagnosis of ADHD shows a similar pattern of limited diagnostic accuracy despite increased specificity with increased T scores. This should not be surprising as overall clinical ADHD is based on impairment reported as present both at home and at school. Two-thirds (68%) of the sample received a clinical diagnosis of ADHD. High T scores on the DSM-IV subscales of the CTRS-R increased the probability of an ADHD diagnosis above 75% for all thresholds The Canadian Journal of Psychiatry, Vol 54, No 4, April

7 Original Research Table 3 Characteristics of children classified by CTRS-R DSM-IV subscales Clinical characteristics True + False + P True False P A. Subscale L DSM-IV IA a n = 321 n = 210 n = 376 n = 123 Age, years, mean (SD) 8.74 (1.95) 8.75 (2.06) (2.30) 8.37 (1.80) Sex, male, % < <0.001 Cognitive disorders, b % Reading disability Language impairment IQ < Externalizing disorders, c % ODD CD B. Subscale M DSM-IV HI d n = 215 n = 209 n = 547 n =67 Age, years, mean (SD) 8.29 (1.82) 8.88 (2.20) (2.15) 7.84 (1.61) <0.001 Sex, male, % < <0.001 Cognitive disorders, b % Reading disability Language impairment IQ < Externalizing disorders, c % ODD < CD C. Subscale N DSM-IV Total e n = 118 n = 277 n = 589 n =54 Age, years, mean (SD) 8.44 (1.86) 8.79 (2.09) (2.16) 7.84 (1.55) <0.001 Sex, male, % < Cognitive disorders, b % Reading disability Language impairment IQ < Externalizing disorders, c % ODD CD a Inattentive subscale b Child assessment c Meeting DSM-IV criteria by semi-structured clinical interview with teacher d Hyperactive impulsive subscale e Total symptoms subscale HI = hyperactive impulsive; IA = inattentive evaluated. However, not even the highest level of specificity, 0.93, seen for a T score of 80 or greater on the L subscale, offered diagnostic certainty above 90% (Table 2B). Comorbid conditions associated with diagnostic errors are reported in Table 3. Table 3(A) reports the characteristics of children classified using CTRS-R subscale L: DSM-IV inattentive. The threshold used for classification where agreement between scale value and criterion was highest was a T score of A relatively high proportion, 40.5%, of the children identified inaccurately as having inattentive symptoms were girls, compared with the proportion of true positives that were girls, 23.4%, P < Conversely, a relatively high proportion of children identified as false negatives were boys, 96.8%, compared with the proportion of true negatives that were boys, 76.4%, P < The group of 238 La Revue canadienne de psychiatrie, vol 54, no 4, avril 2009

8 Using the Conners Teacher Rating Scale Revised in School Children Referred for Assessment children whose symptoms were missed by the CTRS-R subscale were younger, 8.37, SD 1.80 years, than those identified accurately 8.96, SD 2.30 years (P = 0.009). Learning difficulties and disruptive behaviour symptoms were not significantly more common among false positives than among true positives. Table 3(B) reports the characteristics of children classified using CTRS-R subscale M: DSM-IV hyperactive impulsive. The threshold used for classification where agreement between scale value and criterion was highest was a T score of A relatively higher proportion of the children identified inaccurately as having hyperactive impulsive symptoms were girls (36.4%, compared with 19.5%, P < 0.001). The children identified inaccurately as having symptoms were also older (8.88, SD 2.20 years) than those identified accurately (8.29, SD 1.82, P = 0.003). Children identified inaccurately as not having hyperactive impulsive symptoms were more likely to be boys (95.5 %, compared with 75.7%, P < 0.001) and younger, (7.84, SD 1.61 years, compared with 9.04, SD 2.15, P < 0.001) than those identified accurately. Language impairment was more common among false positives, 18.9%, than among true positives, 11.3% (P = 0.04). In contrast, ODD symptoms were more common among children identified accurately to have hyperactive impulsive symptoms, 28.8%, than among those identified inaccurately, 12.4% (P < 0.001). ODD symptoms were also more common among those identified as false negatives than among true negatives, 19.4% and 7.9% respectively, P = Similarly, CD symptoms were more common among true positives, 10.2 %, than among false positives, 3.3% (P = 0.006). Table 3(C) reports the characteristics of children classified using CTRS-R subscale N: DSM-IV total symptoms. The threshold used for classification was a T score of As for subscales L and M, a relatively high proportion of the children identified inaccurately as having both inattentive and hyperactive impulsive symptoms were girls (42.6%, compared with 19.5%, P < 0.001). Conversely, children likely to be identified as false negatives were more likely to be boys (96.3%, compared with 81.2%, P = 0.003) and younger (7.84, SD 1.55 years, compared with 8.93, SD 2.16, P < 0.001) than those identified accurately. Children identified as false positives were more likely to have reading disabilities (35.2%, compared with 21.6%, P = 0.008) and language impairment (19.0%, compared with 9.5%, P = 0.023), than those identified accurately. Additionally, the children identified as false positive were less likely to meet criteria for comorbid ODD by teacher report (15.2%, compared with 28.8%, P = 0.002) than children identified accurately. Children identified as false negatives were more likely to meet criteria for ODD on teacher interview than those identified accurately (22.2%, compared with 9.5%, P = 0.009). Discussion Our study explored the likelihood that information obtained using the DSM-IV subscales of the CTRS-R in a clinical sample would accurately match a clinical interview with the child s teacher and approached the topic of diagnostic accuracy from a different perspective than earlier evaluations. Initial evaluations of the CTRS-R focused on its use to screen children with ADHD from those without disorder in community samples. In such settings, and using a case control study design, the CTRS-R demonstrated sensitivity of 97.1% and specificity of 81.6%. 6 In our study, we evaluated the use of CTRS-R subscales to distinguish between children meeting DSM-IV criteria at school and children with other clinical explanations for attention, learning, and behavioural difficulties. In this clinical sample, no single threshold on the scales simultaneously offered high values of sensitivity and specificity owing to frequent classification errors. Previous studies also evaluated a different subscale of the CTRS-R, subscale H (Conners ADHD Index 6 ), than the DSM-IV subscales evaluated here. The ADHD Index is an index of items that overlap with, but are not identical to, items in the DSM-IV subscales and was validated against clinical, but not DSM-IV, diagnosis. The evidence presented here offers additional guidance for how clinicians should interpret the CTRS-R DSM-IV subscales when they use the full CTRS-R measure. Initially, we hypothesized that a low score on a subscale would indicate that the child does not demonstrate enough impairing ADHD symptoms in school to meet DSM-IV criteria, and that a high score on a subscale indicates enough impairing symptoms in school to meet diagnostic criteria. For the CTRS-R DSM-IV subscales the former, but not necessarily the latter, is generally true. For each subscale, a T score of 60 or more results in a sensitivity greater than 90% and a T score of less than 60 offers a low post-test probability, confirming that a low T score on the subscale is unlikely to be associated with teacher report of DSM-IV symptom criteria in the school setting. On subscale M (hyperactive impulsive) and on subscale N (total), the posttest probability for a T score of less than 60 is less than 10%, suggesting these subscales are the most diagnostically useful given that the low posttest probability effectively rules out the possibility that the child will meet DSM-IV symptom criteria in the school setting. 11 The posttest probability for a child with a T score of less than 60 on subscale L (inattentive) is 12.6%, suggesting the same level of certainty is not present with this subscale, and additional clinical investigation may be required. The Canadian Journal of Psychiatry, Vol 54, No 4, April

9 Original Research Where the CTRS-R DSM-IV scales show mid-range to high T scores, clarification regarding clinical symptoms at school is still required before the clinician can assume the child meets diagnostic criteria at school. Even a T score of 80 or greater on the CTRS DSM-IV subscales, while conferring a specificity ranging from 88% to 93%, does not translate into certainty regarding the teacher reporting in a clinical interview that the child meets DSM-IV symptom criteria in school. The second goal of our study was to identify patterns of comorbid conditions associated with classification errors. The children who were inaccurately identified by the CTRS-R subscales as asymptomatic were nearly all boys, and were younger than those correctly identified as asymptomatic. These young boys met criteria for ODD in the teacher clinical interview significantly more frequently than children who were true negatives for the M (hyperactive impulsive) and N (total) subscales, with a trend in this direction for the L (inattentive) subscale. These observations are consistent with literature suggesting that ODD identified in preschool years is an important risk factor for diagnosis of ADHD in primary school. 20,21 There were more girls among the children identified as having ADHD by the CTRS-R, but not by the DSM-IV clinical interview. Interestingly, language impairment was more common among children rated as false positive than those rated as true positive on M (hyperactive impulsive) subscale whereas both reading disability and language impairment were more common among false positives on the N (total) subscale. Low IQ did not appear more frequently among children who were misclassified. Though recent guidelines urge clinicians to refer for psychoeductional evaluation when inattention (but not hyperactivity and impulsiveness) appear primarily in academic contexts, 3 the results here suggest that clinicians need to maintain vigilance about learning and language impairments in the context of overactivity and impulsiveness as well as inattention. The CTRS-R DSM-IV subscales are highly similar to other teacher rating scales based on the DSM-IV symptom criteria. While there are some minor differences in items and in methods of scoring, there is little reason to think that other scales work better as diagnostic tests for ADHD in clinical samples than the CTRS-R. Clinicians using teacher rating scales to collect information about children in school should be cautious about using them as a substitute for a clinical interview with the teacher. Limitations While our study takes advantage of a large sample of clinically referred school children systematically assessed for ADHD, the criteria imposed by the research design may influence the generalizability of the results. For example, this study excluded children with a full-scale IQ of less than 80, and children with a recent history of trauma or abuse. The diagnostic statistics for the CTRS-R DSM-IV subscales (sensitivity, specificity, and likelihood ratios) will be similar in other clinical samples; however, the pretest probabilities may vary somewhat from setting to setting, affecting the posttest probabilities. It is important to keep in mind that the primary target criterion to validate these scales was the DSM-IV criteria identified by teacher clinical interview rather than the overall clinical diagnosis. Our purpose was to discover how well the scales could substitute for a clinical interview with the teacher when combining information from all sources to decide on DSM-IV diagnosis of ADHD for a child. Clinical Implications Overall the results confirm that the CTRS-R DSM-IV subscales can be a useful adjunct in individual clinical assessments of primary school children with ADHD. Clinicians can depend on the screening tool to rule out ADHD symptoms meeting DSM-IV diagnostic criteria at school when the T score from the teacher rating scale is less than 60 for subscale M (hyperactive impulsive) or for subscale N (total) symptoms; in these situations, the rating scales can be used as a substitute for a clinical interview with the teacher. In other situations, clinicians will need additional information about the child s behaviour in school to clarify whether the child reaches DSM-IV criteria in school. Teacher rating scale DSM-IV classification errors appear more likely for girls who have cognitive and language impairments and for young boys who show oppositional behaviours. Funding and Support Dr Charach is supported by a New Investigator Fellowship from the Ontario Mental Health Foundation. Funding was received from the Canadian Institute for Health Research, grant MOP 64277; Dr Schachar, principal investigator. Dr Schachar is a consultant to Eli Lilly Canada Inc and to Purdue Pharma. The other authors have no financial relations to disclose. Acknowledgements We thank Dr Rosemary Tannock for her conceptual input and Ms Amy Gajaria for her editorial assistance. References 1. Power TJ, Andrews TJ, Eiraldi RB, et al. Evaluating attention deficit hyperactivity disorder using multiple informants: the incremental utility of combining teacher with parent reports. Psychol Assess. 1998;10(3): American Academy of Pediatrics. Clinical practice guideline: diagnosis and evaluation of the child with attention-deficit/hyperactivity disorder. Pediatrics. 2000;105(5): La Revue canadienne de psychiatrie, vol 54, no 4, avril 2009

10 Using the Conners Teacher Rating Scale Revised in School Children Referred for Assessment 3. American Academy of Child and Adolescent Psychiatry. Practice parameter for the assessment and treatment of children and adolescents with attention-deficit/ hyperactivity disorder. J Am Acad Child Adolesc Psychiatry. 2007;46(7): American Psychiatric Association. Diagnostic and statistical manual of mental disorders. 4th ed. Washington (DC): APA; Forbes GB. A comparison of the Conners Parent and Teacher Rating Scales, the ADD-H Comprehensive Teacher s Rating Scale, and the Child Behavior Checklist in the clinical diagnosis of ADHD. J Atten Disord. 2001;5(1): Conners C. Conners Rating Scales: Revised technical manual. Toronto (ON): Multi-Health Systems; DuPaul G, Power T, Anastopoulos A, et al. Teacher ratings of attention deficit hyperactivity disorder symptoms: factor structure and normative data. Psychol Assess. 1997;9(4): Weiler MD, Bellinger DK, Simmons EK, et al. Reliability and validity of a DSM-IV based ADHD screener. Child Neuropsychol. 2000;6(1): Wolraich ML, Lindgren S, Stromquist A, et al. Stimulant medication use by primary care physicians in the treatment of attention deficit hyperactivity disorder. Pediatrics. 1990;86(1): Sackett DL, Haynes RB. The architecture of diagnostic research. BMJ. 2002;324(7336): Sackett D, Straus S, Richardson W, et al. Evidence-based medicine: how to practice and teach EBM. 2nd ed. Toronto (ON): Churchill Livingstone, Elsevier Ltd; Hum M. Psychometric properties of a teacher semi-structured interview for childhood externalizing disorders. Toronto (ON): University of Toronto; Schachar R, Chen S, Crosbie J, et al. Comparison of the predictive validity of hyperkinetic disorder and attention deficit hyperactivity disorder. J Can Acad Child Adolesc Psychiatry. 2007;16(2): Ickowicz A, Schachar RJ, Sugarman R, et al. The parent interview for child symptoms: a situation-specific clinical research interview for attention-deficit hyperactivity and related disorders. Can J Psychiatry. 2006;51(5): Chambers WJ, Puig-Antich J, Hirsch M, et al. The assessment of affective disorders in children and adolescents by semistructured interview. Test-retest reliability of the schedule for affective disorders and schizophrenia for school-age children, present episode version. Arch Gen Psychiatry. 1985;42(7): Wechsler D. Wechsler Intelligence Scale for Children: manual (revised). New York (NY): Psychological Corporation; Semel E, Wiig E, Secord W. Clinical evaluation of language fundamentals. 3rd ed. San Antonio (TX): The Psychological Corporation; Wilkinson G. Wide Range Achievement Test. 3rd ed. Wilmington (Del): Jastek Association; Deeks JJ, Altman DG. Diagnostic tests 4: likelihood ratios. BMJ. 2004;329(7458): Lavigne JV, Cicchetti C, Gibbons RD, et al. Oppositional defiant disorder with onset in preschool years: longitudinal stability and pathways to other disorders. J Am Acad Child Adolesc Psychiatry. 2001;40(12): Speltz ML, McClellan J, DeKlyen M, et al. Preschool boys with oppositional defiant disorder: clinical presentation and diagnostic change. J Am Acad Child Adolesc Psychiatry. 1999;38(7): Manuscript received October 2007, revised, and accepted May Associate Scientist, Research Institute, Hospital for Sick Children, Toronto, Ontario; Assistant Professor, Department of Psychiatry, University of Toronto, Toronto, Ontario. 2 Statistician, Research Institute, Hospital for Sick Children, Toronto, Ontario. 3 Senior Scientist and Manager, Management and Analysis of Data, Institute for Work and Health, Toronto, Ontario; Assistant Professor, Department of Public Health Sciences and Department of Health Policy, Management and Evaluation, University of Toronto, Toronto, Ontario. 4 Senior Scientist, Research Institute, Hospital for Sick Children, Toronto, Ontario; Professor, Department of Psychiatry, University of Toronto, Toronto, Ontario. Address for correspondence: Dr A Charach, c/o Department of Psychiatry, The Hospital for Sick Children, 555 University Avenue, Toronto, ON M5G 1X8, alice.charach@sickkids.ca Résumé : Utiliser l échelle révisée de Conners chez les enfants d âge scolaire adressés pour une évaluation Objectif : La validité prédictive de l échelle révisée de Conners (CTRS-R) a été évaluée par rapport à une entrevue d enseignants clinique semi-structurée, concernant des enfants d âge scolaire adressés pour une évaluation diagnostique du trouble d hyperactivité avec déficit de l attention (THADA). Nous avons posé l hypothèse que les valeurs extrêmes de l échelle augmenteraient la certitude du diagnostic et que les erreurs de classification seraient associées à des affections comorbides. Méthode : Les enfants (n = 1 038), âgés de 6à12ans, ont été dépistés à l aide de la CTRS-R et leurs enseignants ont été interviewés. Trois niveaux de scores T aux trois sous-échelles de la CTRS-R de la 4 e édition du Manuel diagnostique et statistique des troubles mentaux (DSM-IV) ont été comparés avec les seuils de symptômes du DSM-IV par entrevue. Lorsque les scores aux sous-échelles et les entrevues montraient la concordance la plus élevée, la présence de troubles de comportement perturbateur, d un trouble de lecture, d un trouble du langage, et d un faible QI comorbides a été investiguée chez les enfants classés correctement, comparativement à ceux classés incorrectement. Résultats : Les scores T de 60 et plus à toutes les sous-échelles de la CTRS-R du DSM-IV présentaient une sensibilité élevée, de 91 % à 94 %. Ce n est qu aux sous-échelles M (hyperactif impulsif) et N (total) que les scores T inférieurs à 60 offraient des probabilités post-test de moins de 10 %, confirmant qu un enfant n atteint pas le seuil diagnostique par entrevue. Les scores T de 80 et plus offraient une spécificité élevée, de 88 % à 93 %, mais ne présentaient pas de probabilités post-test élevées que les enfants atteignent les critères diagnostiques. Les erreurs de classification étaient associées avec plus de troubles de langage parmi les faux-positifs que les vrais-positifs à la sous-échelle M (18,9 %, comparé à 11,3 %; P = 0,04) et à la sous-échelle N (19,0 %, comparé à 9,5 %; P = 0,023), et avec plus de troubles de lecture chez les faux-positifs que les vrais-positifs à la sous-échelle N (35,2 %, comparé à 21,6 %; P = 0,009). Conclusions : La capacité de la CTRS-R à prédire si des enfants adressés cliniquement atteignent les critères du THADA du DSM-IV à l école est limitée. The Canadian Journal of Psychiatry, Vol 54, No 4, April