Matching Strategies in Cognitive Research with Individuals with High-Functioning Autism: Current Practices, Instrument Biases, and Recommendations

Transcription

1 Journal of Autism and Developmental Disorders, Vol. 34, No. 1, February 2004 ( 2004) Matching Strategies in Cognitive Research with Individuals with High-Functioning Autism: Current Practices, Instrument Biases, and Recommendations Laurent Mottron 1,2 A meta-analysis was performed on the 133 cognitive and behavioral papers in autism using comparison groups in the period. High-functioning (average IQ: 84.7), adolescents (average, 14.4 years) are largely dominant. IQ is the most frequent matching variable in use (51.2%). The instruments that are most frequently used to determine IQ or general level are Wechsler scales (46.9%), British Picture Vocabulary Scale (BPVS; 22.3%), and Raven Progressive Matrices (RPM; 16.9%). In order to determine if these instruments were equivalent when applied to individuals with pervasive developmental disorders (PDDs), Wechsler IQ, EVIP (a French Canadian translation of the BPVS), and RPM were given to a group of 14 individuals with autism and 12 with Asperger syndrome. Comparison of Wechsler and RPM IQs values, expressed as percentiles, to percentile values of EVIP score revealed that EVIP (and to a lesser extent RPM) considerably overestimates the level of all PDD participants as compared to Wechsler Verbal IQ (VIQ), Performance IQ (PIQ), or Full-Scale IQ (FSIQ), whereas these instruments are reported to be strongly correlated in typically developing individuals. This study reveals that identification of objects from a verbal label the BPVS PPVT EVIP task is a peak of ability in high functioning individuals with PDDs. This peak of ability, even superior to that of block design, has a detrimental effect on matching based on this instrument. A recommendation to replace BPVS/PPVT/EVIP or RPM by Wechsler scale as a basis of IQ/level matching is provided. Accordingly, the former instruments are a potential source of type-1 (for cognitive deficits) or type-2 (for cognitive hyperfunctioning) errors. KEY WORDS: British Picture Vocabulary Scale; high-functioning autism; IQ; matching strategy; methodology; peaks of ability; Raven Progressive Matrices; Wechsler Intelligence Scale. INTRODUCTION The charting of cognitive deficits and strengths among persons with autism requires comparison of their performance with that of another group. In addition to differences in clinical status, the target and comparison groups can differ on many factors, such as age and level of functioning, which may influence the cognitive 1 Centre de Recherche Fernand Seguin and Département de Psychiatrie, Université de Montréal, Montréal, Canada 2 Correspondence should be addressed to Laurent Mottron, Clinique Spécialisée de l Autisme, Hôpital Rivière des Prairies, 7070 Blvd. Perras, Montréal, Canada, H1E1A4; mottronl@istar.ca 19 performances under study. Therefore, some kind of control should be used to limit the confounding effects that may result from this heterogeneity in these groups. Abbreviations: ADI, Autism Diagnostic Interview; ADOS, Autism Diagnostic Observation Schedule; AS, Asperger syndrome; BPVS, British Picture Vocabulary Scale; CA, chronological age; CELF, Clinical Evaluation of Language Fundamentals; EVIP, Échelle de Vocabulaire en Images Peabody; FSIQ, full-scale IQ; HFA, highfunctioning autism; MA, mental age; PDD, pervasive developmental disorder; PDDNOS, pervasive developmental disorder not otherwise specified; PIQ, performance IQ; PPVT, Peabody Picture Vocabulary Test; RPM, Raven Progressive Matrices; RT, reaction time; ToM, theory of mind; TROG, Test of Reception of Grammar; VIQ, verbal IQ; WAIS, Wechsler Adult Intelligence Scale; WISC, Wechsler Intelligence Scale for Children /04/ / Plenum Publishing Corporation

2 20 Mottron As autism is a condition that begins early in life, is observed across the range of functioning levels from severe mental retardation to superior intelligence, and may or may not be accompanied by identified neurological syndromes, a random group of individuals who satisfy the criteria for the diagnosis is likely to be heterogeneous at multiple and possibly interacting levels. This heterogeneity complicates the matching procedure and, accordingly, leads to the study of specific subgroups. The choices of these subgroups may be arbitrary or driven by considerations of convenience rather than by purely scientific decisions. The search for the unique characteristics of autism entails the study of specific factors that are linked either to the phenotype or to the causal mechanism of autism per se and are not shared with other neurodevelopmental disorders. The attempt to disentangle specific and nonspecific factors in cognitive research on autism led to the mental age (MA) matching paradigm that is attributed to Hermelin and O Connor (1970). Before their seminal contribution, individuals with autism were typically compared to those of typically developing individuals of the same chronological age (CA), a problematic strategy due to the high incidence of low IQs among persons with autism. In their innovative series of experiments, Hermelin and O Connor compared the performance of persons with autism who functioned in the range of mental retardation to that of younger typically developing children matched on mental age (MA) or of nonautistic persons with mental retardation matched on MA. Despite the contributions of the MAmatching paradigm in controlling for differences in developmental level, this strategy introduces other confounding elements. For example, inherent differences in CA between younger typically developing children and older persons with low-functioning IQ limit the implications of findings based on MA-matched studies (for a review, see Burack, Iarocci, Bowler, & Mottron [2002]) because CA differences between groups cannot be disentangled from the differences related to autism per se. Alternatively, when persons with mental retardation matched on IQ are compared to persons with autism, the characteristics associated with the etiologies of mental retardation in the comparison group may be the source of artifactual group differences (Yirmiya, Shaked, Solomonica-Levi, 1998). In order to minimize the theoretical and practical shortcomings of MA matching, persons with highfunctioning autism (HFA) became the focus of cognitive research during the late 1980s. Individuals with high-functioning autism may be compared to typically developing individuals without the confounding factors of CA discrepancies and syndrome-specific findings. The study of autism is further facilitated in this group due to superior compliance and attention span and, for adults, the ability to provide legally their own consent to participate in the research. Although individuals with HFA display the same variability in average intelligence as typically developing individuals, they present an uneven profile of strengths and deficits within subtests that comprise the more general score of intelligence. Peaks and valleys may differentially influence averaged level measures according to the instruments used. Accordingly, the practical aspects of matching on intelligence level may have important consequences on the similarities between groups matched on intelligence level. Thus, the purpose of this paper is twofold. The first is to present a systematic survey of the current strategies for choosing subgroups, matching variables, and standardized instruments for assessment of level of functioning in the recent literature on cognitive neuroscience research with individuals with pervasive developmental disorders (PDDs). The second purpose of this paper is to compare empirically the most common instruments used for level matching which were identified in the first study. STUDY 1: META-ANALYSIS OF MATCHING STRATEGIES AND SUBGROUP CHOICES USED IN BEHAVIORAL NEUROSCIENCE OF PERVASIVE DEVELOPMENTAL DISORDERS Method Behavioral and cognitive, peer-reviewed, empirical papers involving group studies quoted in the MED- LINE database in the 3-year period from January 1999 to December 2002 under the entries Autism and Asperger syndrome were selected. The inclusion criteria were that the papers were published in English, included a full abstract, involved typical or clinical comparison groups or normative data, and that performance was based on reaction time (RT) scores, error/success rates, or another behavioral measure as the dependent variable. Behavioral findings of brain-imaging studies were included in the survey, but clinical trials, rehabilitation, brain imaging, and electrophysiological studies without behavioral-dependent measures as well as studies with only individuals with pervasive developmental disorder not otherwise specified (PDD-NOS) were excluded. One hundred and thirty-three papers in cognitive neuroscience on autism and Asperger syndrome were

3 Matching Issues and IQ-Instrument Biases 21 reviewed, with a total of 169 comparison groups (as multiple matching groups were used for the same experiment or for several experiments in the same paper). Among the 169 comparisons, 106 (62.7%) compared typically developing with individuals with autism, 24 (14.2%) with Asperger syndrome, and 39 (23.1%) with combined groups of individuals. For each comparison group, the mean IQ, MA, and CA of the clinical population, type of comparison group used (clinical, typical), type of matching (pair-wise, groupwise, covariation), number of matching variables (e.g., FSIQ, VIQ, PIQ, MA, CA), and tool to assess the level of intelligence or performance used for matching (e.g., WAIS, RPM, PPVT, BPVS) was noted. In order to determine the proportion of participants with and without mental retardation among the persons with autism, the intelligence levels provided by the various instruments across the studies was converted to a common metric. Because the Wechsler instruments are the most frequent measures of general intelligence in use, Wechsler Full-Scale IQ (FSIQ; Wechsler, 1974, 1981) was taken as reference measure and provided when available. In cases where only Wechsler Verbal IQ (VIQ) (34 comparisons) or Performance IQ (PIQ) (14 comparisons) were mentioned, this value was taken as indication of participants intelligence. In the cases of the British Picture Vocabulary Scale (BPVS; Dunn, Dunn, Whetton, & Pintilie, 1982), which provides MAs and percentiles, and Raven Progressive Matrices (RPM; Raven, 1938, 1947, 1995, 1996), which provides IQs and percentiles, an approximation of IQ was obtained using Wechsler Percentile-IQ Correspondence Scale. In the cases of other tests that provide only MA (e.g., Test of Reception of Grammar [TROG], Mullen, Bayley, Vineland Adaptive Behavior Scale, Clinical Evaluation of Language Fundamentals [CELF], Brunet Lézine, and Reynell), an approximation of IQs was computed using the IQ = MA/CA formula. Finally, a cut-off between high- and low-functioning of IQ = 67 rather than the usual value of 70 was selected, as this is the lowest value for which a correspondence between IQ and percentile value can be obtained according to Wechsler manual. Results Characteristics of Participants Distribution of Comparisons According to IQ Level of Participants. The distribution of comparisons according to IQ levels for the autism, the Asperger, and the combined PDD subgroups are presented in Table I. Due to the lack of availability of information for two Table I. Distribution of Comparisons According to the Intelligence Level of the Participants a Number of groups (%) Intelligence level Autism PDD Asperger Mild mental retardation 26 (22.0) 0 (0.0) 0 (0.0) (FSIQ < 67) Borderline intelligence 45 (38.2) 7 (41.2) 6 (18.8) (FSIQ 67 to 90) Normal intelligence 47 (39.8) 10 (58.8) 26 (81.2) (FSIQ > 90) Total number of 118 (100.0) 17 (100.0) 32 (100.0) comparisons PDD, pervasive developmental disorder; FSIQ, full-scale IQ. a PDD refers to studies where data from autism and Asperger syndrome cannot be differentiated. Table II. Distribution of Comparisons According to Chronological Age of Participants Asperger Autism PDD syndrome Number of Number of Number of Age range groups (%) groups (%) groups (%) 0 to 5 years old 17 (14.5) 1 (5.9) 0 (0.0) 6 years and older 100 (85.5) 16 (94.1) 35 (100.0) Total 117 (100.0) 17 (100.0) 35 (100.0) PDD, pervasive developmental disorder. papers, proportions are provided for 167 comparisons. The average IQ across comparisons, with each comparison weighted equally, was The majority (76.7%) of comparisons relied on participants with autism without mental retardation. Distribution of Comparisons According to the Age of Participants. The ages of the participants included in these studies are presented in Table II. The mean age of participants across studies, with each study weighted equally, was 14.4 years. In 85.5% of the studies with persons with autism, 94.1% of those with persons with PDD, and 100% of those with persons with Asperger syndrome (AS), the average age of the participants was older than 6 years. The lack of studies with preschoolers with AS likely results from the practical problem of providing this diagnosis before 5 years of age. Matching Strategies Socio-Demographic Variables. The relative frequency of the different socio-demographic variables that are used are depicted in Table III. Chronological

4 22 Mottron Table III. Distribution of Socio-Demographic Matching Strategies Among Comparisons (Total: 169) No differences Total number Variable Matching (%) Covariation (%) between groups (%) of groups (%) CA 83 (49.1) 3 (1.8) 18 (10.7) 104 (61.5) Gender 29 (17.2) 3 (1.8) 32 (18.9) Educational level 9 (5.3) 1 (0.6) 10 (5.9) Laterality 10 (5.9) 10 (5.9) SES 4 (2.4) 1 (0.6) 5 (3.0) Other (profession, interest, 4 (2.4) 2 (1.2) 6 (3.6) sensory profile) CA, chronological age; SES, socioeconomic status. age (approximately two-thirds of the groups, 104 comparisons = 61.5%) and gender (approximately one-fifth, 32 comparisons = 18.9%) were the most frequently used variables. Type and Number of Matching Variables. A single matching variable was used in 51 comparisons (30.2%). The most commonly used were VIQ or Verbal level in 20 comparisons (11.8%), CA in 12 = (7.1%), PIQ in 5 (2.9%), and MA in 5 (2.9%). Two variables were used in 55 comparisons (32.5%). The most frequent combinations were CA + VIQ or Verbal level (17 comparisons; 10%), CA + FSIQ (9 comparisons; 5.3%), CA + PIQ (7 comparisons; 4.5%), and CA + Gender (4 comparisons; 2.3%). When three variables were used, the most frequent combinations were VIQ/level + PIQ + CA (7 comparisons; 4.5%), FSIQ + Gender + CA (5 comparisons; 2.9%), and VIQ/level + PIQ + FSIQ (3 comparisons; 1.7%). Across all the comparisons, the most common matching variables were VIQ/level in 74 comparisons (43.8%), PIQ in 53 (31.4%), and FSIQ in 36 (21.3%). Instruments Used to Assess General Intelligence. Among the 130 comparisons where intelligence was used as matching variable, the Wechsler Intelligence Scale for Children/Wechsler Adult Intelligence Scale (WISC/WAIS) was the test used most frequently (61/130 = 46.9%), followed by the BPVS, the British version of PPVT-R 3 (29/130 = 22.3%) for verbal level, and by RPM for performance IQ (22/130 = 16.9%). Scales of adaptive functioning (VABS) were used only marginally (3/169 = 1.8%). 3 BPVS and PPVT (Dunn & Dunn, 1981) are vocabulary scales that consist in pointing to the picture corresponding to a verbally presented name among three distracters. BPVS has been derived from the PPVT-R. PPVT-III is similar to PPVT-R with the exception of enlarged age validity and improved culture and gender fair pictures. On the BPVS, only 3.9% of PPVT-III items/words corresponding to specifically U.S. idioms have been replaced by their U.K. equivalent. Discussion The current prototypical study in cognitive neuroscience on autism is a cross-sectional study using individuals with HFA with a FSIQ around 85 and CA of 14 years. Participants with low developmental and chronological levels are practically discarded from cognitive research on autism. As a spontaneous and unquestioned research strategy, this finding raises important issues of external validity. Can the knowledge based on this intelligence level and age group, now clearly the main source of knowledge regarding cognition in autism, be generalized to individuals in the entire range of age and IQ levels? Using individuals more than 10 years older than the process that made them autistic cannot separate the specific effect of pathological process per se from those of reaction to these processes. The cognitive profile of adults with autism plausibly results from the combined effects of expertise and over-training linked to special interests, adaptive compensatory strategy, or, in contrast, under-training linked to impoverishment of external stimulation. Accordingly, the use of adults is probably not the best strategy to document primary deficits even if this represents an indispensable source of information on the cognitive condition of adults with autism. The second finding of this literature survey was that CA and IQ/level, mostly verbal but also performance or FSIQ, are the dominant matching variables in current cognitive research on PDDs. This is due to the use of high-functioning individuals that allow CA matching a practice impossible with the use of individuals with mental retardation. However, the use of IQ or verbal level as the main matching variable puts a heavy burden on the instruments used to measure this IQ. This study also establishes that although Wechsler scales are largely dominant (approximately one-half of the studies), two other instruments, BPVS (one-quarter) and RPM (one-sixth) are also used in a significant

5 Matching Issues and IQ-Instrument Biases 23 proportion of studies. Therefore, possible differences in sensitivity and lack of comparability among studies may result from this use of multiple instruments, which may not be correlated similarly in comparison and clinical populations. This issue will be addressed in the following study. STUDY 2: ARE THE DIFFERENT INSTRUMENTS USED FOR IQ OR LEVEL MATCHING EQUIVALENT? A survey of recent cognitive literature on autism indicates that partial or general measures of IQ are the variables most frequently used for matching. Notwithstanding, researchers used as equivalent several instruments to measure the intelligence level on which the matching is performed. For example, verbal IQ or verbal level is obtained through Wechsler VIQ or BPVS, and nonverbal IQ is obtained through RPM or Wechsler PIQ. The purpose of Study 2 was therefore to determine if these instruments were equivalent and, more precisely, the type of biases (over- or underestimation) possibly introduced by these instruments. To answer these questions, we decided to compare the intelligence level of a group of individuals with PDD, as measured by the two most frequently used measures of verbal performance (Wechsler VIQ vs. EVIP score) and nonverbal performance (Wechsler PIQ vs. RPM). EVIP is a French translation of PPVT-R, valid for French-Canadian as well as European French populations. Method All individuals (a) with a diagnosis of autism, using both Autism Diagnostic Interview (ADI) or ADI-R and Autism Diagnostic Observation Schedule- Generic (ADOS-G) module 3 or 4, (b) having received a full-scale intelligence measurement with WISC-III or WAIS-III, (c) who were administered both a RPM and a EVIP assessment, and (d) with a FSIQ > 60 were selected in the PDD population of the clinique spécialisée de l autisme of Rivière-des-Prairies s hospital, using Datafinder database (Digimed systems). Among this group, individuals who were diagnosed with autism, but without language delay, echolalia, pronoun reversal, or stereotyped language at ADI administration were considered as having Asperger syndrome. This resulted in a group of 26 individuals (autism subgroup: 14 individuals, 11 M, 3 F; mean age, 12.5; SD, 8.2; range, 6 39; Asperger syndrome Fig. 1. Percentiles equivalent obtained by EVIP, RPM, and Wechsler subscales. subgroup: 12 individuals, 8 M, 4 F; mean age, 15.3; SD, 6.5; range, 7 29). PIQ, VIQ, and FSIQ obtained by WISC/WAIS and IQs or levels from EVIP and RPM were transformed into percentiles according to the tables provided with each instrument and compared using Student t test for paired samples (two-tailed). Because of relatively small samples, Wilcoxon matched-pairs test was also carried out as validation process of Student t-test results (Fig. 1). Results Relative IQ/levels (percentile equivalent) obtained by EVIP, RPM, and Wechsler subscale are represented in Figure 1 and statistical analyses among values provided by these different instruments in Table IV. For comparison purposes, Wechsler block-design and vocabulary subscale were included in the analyses. Autism Subgroup Wechsler FSIQ, VIQ, and PIQ comparison revealed a nonsignificant PIQ > VIQ pattern, thus replicating the finding of Siegel et al. (1996) of unremarkable differences between Wechsler sub- and full-scale among sufficiently large groups of persons with autism. In contrast, differences in the percentile measure obtained by Wechsler VIQ and EVIP level were statistically significant [Student t test for paired samples (two-tailed): t(13) = 4.55, p = 0.001; Wilcoxon matched-pairs: z = 3.30, p = 0.001]. This indicates that the EVIP considerably overestimates verbal abilities in comparison to the Wechsler verbal subscale. Differences between Wechsler PIQ and RPM were not significant, but differences between Wechsler FSIQ and RPM were near significance or significant according to analyses [Student t test for paired samples

6 24 Mottron Table IV. Consistency Among Percentiles Equivalent Obtained by Different Instruments (t-test, p-values) Asperger 1. EVIP 2. RPM.0048 c 3. Block design a.05 b Vocabulary a.013 b PIQ a.0084 c c VIQ a.0021 c FSIQ a.0033 c b Autism 1. EVIP 2. RPM Block design a Vocabulary a.0020 c PIQ a b VIQ a d b FSIQ a d b Asperger or autism 1. EVIP 2. RPM.0053 c 3. Block design a Vocabulary a d PIQ a.0032 c d VIQ a d FSIQ a d c a Wechsler Intelligence Scale. b p <.05. c p <.01. d p <.001. (two-tailed): t(13) = 2.06, p = 0.060; Wilcoxon matched-pairs: z = 2.10, p = 0.036] showing that the RPM overestimates general intelligence level in comparison to the Wechsler Full-Scale IQ. Asperger Subgroup Comparisons among Wechsler FSIQ, VIQ, and PIQ revealed a nonsignificant VIQ > PIQ pattern that replicates Miller & Ozonoff s (2000) findings with this group. In contrast, EVIP level was significantly superior to Wechsler VIQ [Student t test for paired samples (two-tailed): t(11) = 4.01, p = 0.002; Wilcoxon matched-pairs: z = 2.85, p = 0.004]. Differences between Wechsler PIQ and RPM and between Wechsler FSIQ and RPM were not significant (See Fig. 1). Findings from the Combined Groups of Persons with Autism and Persons with Asperger Syndrome In order to validate the findings in the face of possible disagreements concerning the strategy for diagnostically distinguishing between persons with autism and those with Asperger syndrome, a single analysis was conducted on both groups. According to this analysis, the scores were highest on the EVIP, which were superior to those on the three Wechsler subscales. The scores on both the EVIP and block-design were superior to those on the Wechsler VIQ and PIQ, thereby indicating that they may be considered as peaks of ability. At an individual level, EVIP level was greater than Wechsler VIQ for 24 out of 26 individuals with autism or Asperger syndrome. In contrast, differences among RPM and Wechsler PIQ in the combined group were not significant. Discussion Type of Bias Introduced by EVIP and RPM The purpose of Study 2 was to compare the intelligence level, expressed in percentile equivalent of a typically developing population, of the tests most frequently used for measuring IQ or level of performance used for matching. The finding indicates that EVIP, RPM, and Wechsler scales result in important differences in performance when applied to the same population of high-functioning PDD individuals. For the subgroup of individuals with autism, the performance discrepancies are especially important between EVIP and RPM/Wechsler scales and between RPM and Wechsler FSIQ. In the subgroup of individuals with Asperger syndrome and in the combined autism Asperger group, the discrepancies are maximum between EVIP and Wechsler scales. As a consequence, findings based on matching with vocabulary-based scales, RPM, and Wechsler measures are not compatible, and the comparisons across studies need to involve corrections for differences in sensitivity. The concern that correlations between vocabularybased scales and Wechsler scale might be substantially lower in clinical populations than in typically developing individuals is supported by prior evidence. In a MEDLINE literature review, findings from 17 studies conducted with various clinical populations (mostly learning disabilities and mental retardation at large) indicated that PPVT-R underestimates intelligence level compared to Wechsler VIQ or FSIQ. In contrast, evidence from three studies suggested that PPVT-R overestimates intelligence level in comparison with Wechsler VIQ or FSIQ, mostly in neuropsychiatric patients and elderly individuals with average to high IQ (Mangiaracina & Simon, 1986; Price, Herbert, Walsh, & Law, 1990; Snitz, Bieliauskas, Crossland, Basso, & Roper, 2000). When individuals with low intelligence are used as a comparison group for research conducted with persons with PDD, the additive effect

7 Matching Issues and IQ-Instrument Biases 25 of these two opposite biases creates a high risk of comparing populations with important differences in level of general intelligence. The bias introduced by the use of RPM as matching instrument is quantitatively smaller than that associated with the EVIP, but was still significant in the subgroup of persons with autism. This difference in measures for persons with autism is discrepant with high correlation between RPM and FSIQ or VIQ which is evidenced among typically developing individuals (Burke, 1985; Jensen, Saccuzzo, & Larsen, 1988; O Leary, Rusch, & Gaustello, 1991) as well as among a heterogeneous population of patients in psychiatric hospitals (O Leary et al., 1991). These findings from the analyses of the EVIP and the RPM are inconsistent with the assumption that instruments that measure intelligence in persons with PDD reflect general intelligence in the same way as in the comparison groups. The matching of persons with autism to typically developing individuals with vocabulary-based scales or with RPM results in an overestimation of the general intelligence of the former group which would not be manifested with a Wechsler scale. A more radical conclusion is that the concept of VIQ for persons with autism, and even more so for persons with Asperger syndrome, consists of two very different measures that cannot be used one for the other. One is an average of measures involving language use and language mechanisms, for which the Wechsler VIQ might be one source, and the other is a measure of the peak of ability in labeling objects as measured by vocabulary-based scales. Possible Explanation of the Differences in Intelligence Level Between Wechsler Scales and the Two Other Instruments Under Study My position is that vocabulary-based instruments, and to a lesser extent the RPM, are tasks that rely on a more limited range of cognitive abilities than the corresponding verbal and nonverbal Wechsler subscales and the resultant FSIQ. The consequence of this difference is that the extreme scores characterizing peaks of abilities of persons with autism influence the performance on these instruments to a larger extent than on Wechsler VIQ, PIQ, or FSIQ. This is evident when the Wechsler standard scores for the subjects in Study 2 are averaged with and without the peaks and valleys. The peaks were defined according to WISC-III manual (Canadian supplement) as a significant difference (p < 0.05) of a specific subtest in relation to the average of six other subtests. This resulted in a mean of 2.7 classical peaks (e.g., block design, vocabulary) and 2.15 valleys (e.g., comprehension, digit-span) per subject. Despite the prevalence of these widely discrepant levels of ability, the averaged standard scores with and without the extreme values were similar. In contrast, strengths in the abilities that are the basis of EVIP and RPM scores are not compensated by other tasks in which performance is average or inferior. A second position is that vocabulary-based scales and RPM happen to tap into cognitive systems that correspond to peaks of ability among persons with autism. Both instruments involve tasks in which the overfunctioning of low-level perceptual cognitive operations that characterize autism (Mottron & Burack, 2001) is an advantage. Accordingly, the association of a verbal label and a picture (EVIP) may be performed at a low perceptual level, as the similarity between a series of visually presented patterns (RPM) may be detected by extraction of perceptual regularities. Consistent with this interpretation and as indicated in Figure 1, performance on vocabulary-based scales by persons with PDDs is even higher than on the Wechsler blockdesign task, the most documented source of a peak of ability in this group. Post hoc Reinterpretations of Data Based on the Use of These Instruments One of the major goals of current research on autism is the search for cognitive hyper- or hypofunctioning. Hyperfunctioning refers to peaks of performance on a certain operation or in processing a certain type of material that is higher than the performance on other operations or with other materials. In terms of cognitive operations, most hyperfunctioning is related to tasks based on perception. In terms of the material that is processed, most hyperfunctioning in individuals with autism is found with nonsocial materials (Mottron & Burack, 2001). In contrast, hypofunctioning refers to valleys in performance, when performance on a given task or with a specific material is inferior to the average performance of the participant. This is the case for theory of mind tasks, executive tasks, or more generally complex tasks (Minshew, Sweeney, & Luna, 2002). In terms of material, most hypofunctioning is evident in the processing of social material. The evidence from Study 2 establishes that the peaks of abilities associated with the EVIP and RPM result in an overestimation of general intelligence level in persons with autism. In other words, and keeping in mind the WISC IQ profile characterizing autism, EVIP and RPM percentile value are closer to the vocabulary and the block-design peaks than to the FSIQ. In contrast, FSIQ is closer to the baseline of IQ subtests. As

8 26 Mottron a consequence, the use of the levels provided by EVIP and RPM for matching purposes results in a possible discrepancy between target and comparison group level according to the task under study. If the task relies on the same type of cognitive operations than those on which EVIP and RPM are based, the comparison group will be at the same level as the persons with autism. When searching for possible hyperfunctioning, matching with these instruments will likely result in a negative finding because the related baseline abilities of the persons with autism will be overestimated. This is a false negative finding (or type-2 error: a failure to reject the null hypothesis when it is in fact false) when looking for hyperfunctioning of the autism group. Thus, the peaks of abilities among persons with autism may not be sufficiently reported in the characterization of cognitive profiles. In the case where the task under study is not related to the matching variable (e.g., looking for an executive deficit when matching on a vocabulary-based instrument), the overestimation of the general intelligence of the clinical group introduced by the matching procedure introduces a heightened risk for a false positive finding (or type-1 error, which occurs when a true null hypothesis is incorrectly rejected) when looking for deficits. As a consequence, studies matching target and comparison group using vocabulary-based instrument or RPM may have inflated deficits characterizing autism. This is especially important in the field of face and emotion processing for which verbal performance is typically used for matching (Ozonoff, Pennington, & Rogers, 1991). This may also lead to the reconsideration of some of the findings related to impairment in theories of mind tasks, as BPVS is frequently used to match groups in theses tasks, following the notion that they are correlated to language ability (see Burack, et al., this issue). Similarly, some group differences (e.g., group effect in reaction time; Mottron, Burack, Stauder, & Robaey, 1999) may result from overestimation of participants with autism when they are matched on RPM. Role of Matching Instruments in the Validation of the Autism/Asperger Distinction The evidence from Study 2 suggests that performance on tasks like the EVIP and RPM (see Fig. 1) differs for persons with autism as compared to those with Asperger syndrome. The level of performance on the block-design task is clearly the highest peak among the persons with autism as it exceeds even performance on the EVIP, which is enhanced in relation to other areas of functioning. In contrast, the performance of the persons with Asperger syndrome was high on the EVIP, but average on the RPM and block-design task. When these two subgroups are combined, blockdesign and EVIP appear as the two main peaks of performance, although the contribution of participants with autism and Asperger syndrome differs. These results are discrepant with the recent arguments that autism and Asperger syndrome are not different at a neuropsychological level (Miller & Ozonoff, 2000). Rather, the findings presented here indicate that the peaks of abilities of persons with autism and Asperger syndrome are not similar and that this difference needs to be considered when choosing matching instruments. GENERAL RECOMMENDATIONS Some recommendations follow from these two studies. The first recommendation would be to focus on people with autism with mental retardation as well as on higher functioning persons. According to the current trend revealed by Study 1, matching issues, availability of high-functioning individuals and other practical concerns, results in an emphasis on crosssectional studies involving adult, intelligent individuals with PDDs. Besides the positive aspect of increasing our understanding of high-functioning individuals, one may question whether this research strategy will also be associated with conceptual and empirical insights about persons with autism who function in the range of mental retardation. Moreover, the use of adults leads to difficulties in disentangling nuclear cognitive deficits what we are supposed to look for from the effects of complex compensatory mechanisms and training or deprivation (or life experiences, see Burack et al., this issue) which are measured. The second recommendation is that the same level of concern regarding standardization for the diagnosis of PDD should be applied to the use of intelligence instruments. Although the scientific community considerably reduced the noise associated with the diagnoses of PDD by the use of standardized diagnosis instruments, the choice of tools for the assessment of intelligence level for matching purposes are quite discrepant. In order to address this problem, the use of the Wechsler scale should be considered as standard scientific practice whereas the other instruments that are commonly used, EVIP BPVS PPVT and RPM, should not be employed as matching instruments because they introduce major risks for type-1 and type-2 errors.

9 Matching Issues and IQ-Instrument Biases 27 The last recommendation is related to the choice of a general versus specific matching variable. The data and arguments presented here lead to two clearly separate types of cognitive studies with regard to matching strategies. In one scenario, errors related to use of over- and underestimation biases are prevalent when searching for main effects of group, as is the case when studying peaks or valleys among cognitive performance. In this situation, the use of the Wechsler instruments is recommended to avoid type-1 or type-2 errors. In contrast, when studying group-by-conditions interactions, an over- or underestimation of the general level of clinical group is less influential. This is the case in fine-tuned cognitive tasks in which the purpose is to disentangle the relative role of two mechanisms (or the influence of one operation on another), rather than main effects of superiority or inferiority. In such cases, group main effects (i.e., possibly resulting from an imprecise matching between groups) may be left uninterpreted (e.g., Mottron et al., 1999). In this case, a task-specific matching variable is preferable to a general one. This variable needs to be justified for each different experiment, thereby precluding a single solution that is valid for all the types of experiments (Burack et al., this issue). ACKNOWLEDGMENTS We thank Geneviève Martel, Patricia Jelenic, and Marie-Josée Caron for research assistance and Jake Burack, Michelle Dawson, and Oriane Landry for editing help on an earlier version of this manuscript. This work was supported by a grant from the Canadian Institute for Health research, Characterizing cognitive deficit in Autism and Asperger Syndrome. REFERENCES Burack, J. A., Iarocci, G., Bowler, D., & Mottron, L. (2002). Benefits and pitfalls in the merging of disciplines: The example of developmental psychopathology and the study of persons with autism. Development and Psychopathology, 14, Burke, H. R. (1985). Raven s Progressive Matrices: More on norms, reliability and validity. Journal of Clinical Psychology, 41, Datafinder 3.5. Digimed Systems, Montréal, Canada <info@digimedsystems.com>. Dunn, L. M., & Dunn, E. S. (1981). Peabody Picture Vocabulary Test-Revised. Circle Pines, MN: American Guidance Services. Dunn, L. M., Dunn, L. M., Whetton, C., & Pintilie, D. (1982). British Picture Vocabulary Scale. Windsor: NFER-NELSON. Dunn, L. M., Thériault-Whalen, C. M., & Dunn, L. M. (1993). Échelle de vocabulaire en image Peabody. Toronto, Ontario: Psycan. Hermelin, B., & O Connor, N. (1970). Psychological experiments with autistic children. Oxford: Pergamon Press. Jensen, A. R., Saccuzzo, D. P., & Larsen, G. E. (1988). Equating the standard and advanced forms of the Raven Progressive Matrices. Educational and Psychological Measurement, 48, Mangiaracina, J., & Simon, M. J. (1986). Comparison of the PPVT-R and WAIS-R in state hospital psychiatric patients. Journal of Clinical Psychology, 42, Miller, J. N., & Ozonoff, S. (2000). The external validity of Asperger disorder: Lack of evidence from the domain of neuropsychology. Journal of Abnormal Psychology, 109, Minshew, N. J., Sweeney, J., & Luna, B. (2002). Autism as a selective disorder of complex information processing and underdevelopment of neocortical systems. Molecular Psychiatry, 7, S Mottron, L., Burack, J. A., Stauder, J. E., & Robaey, P. (1999). Perceptual processing among high-functioning persons with autism. Journal of Child Psychology and Psychiatry, 40, Mottron, L., & Burack, J. (2001). Enhanced perceptual functioning in the development of autism. In J. A. Burack, T. Charman, N. Yirmiya, & P. R. Zelazo (Eds.), The development of autism: Perspectives from theory and research (pp ). Mahwah, NJ: Lawrence Erlbaum. O Leary, U. M., Rusch, K. M., & Guastello, S. J. (1991). Estimating age-stratified WAIS-R IQs from scores on the Raven s Standard Progressive Matrices. Journal of Clinical Psychology, 47, Ozonoff, S., Pennington, B. F., & Rogers, S. J. (1991). Executive function deficits in high-functioning autistic individuals: Relationship to theory of mind. Journal of Child Psychology and Psychiatry, 32, Price, D. R., Herbert, D. A., Walsh, M. L., & Law, J. G. (1990). Study of WAIS-R, Quick Test and PPVT IQs for neuropsychiatric patients. Perceptual and Motor Skills, 70, Raven, J. C. (1938, 1996). Progressive Matrices: A perceptual test of intelligence. Individual form. Oxford: Oxford Psychologists Press Ltd. Raven, J. C. (1947, 1995). Colored Progressive Matrices Sets I and II. Oxford: Oxford Psychologists Press Ltd. Snitz, B. E., Bieliauskas, L. A., Crossland, A., Basso, M. R., & Roper, B. (2000). PPVT-R as an estimate of premorbid intelligence in older adults. The Clinical Neuropsychologist, 14, Wechsler, D. (1974). Wechsler Intelligence Scale for Children- Revised. New York: The Psychological Corporation. Wechsler, D. (1981). Wechsler Adult Intelligence Scale-Revised. New York: The Psychological Corporation. Yirmiya, N., Erel, O., Shaked, M., & Solomonica-Levi, D. (1998). Meta-analyses comparing theory of mind abilities of individuals with autism, individuals with mental retardation, and normally developing individuals. Psychological Bulletin, 124,