Dissertation. Sarah L. Leone, M.A. Psychology Graduate Program. The Ohio State University. Dissertation Committee: Luc Lecavalier, Advisor

Transcription

1 Evaluation of the Ghuman-Folstein Screen for Social Interaction (SSI) Dissertation Presented in Partial Fulfillment of the Requirements for the Degree Doctor of Philosophy in the Graduate School of The Ohio State University By Sarah L. Leone, M.A. Psychology Graduate Program The Ohio State University 2009 Dissertation Committee: Luc Lecavalier, Advisor Michael Aman David Hammer

2 Copyright by Sarah L. Leone 2009

3 Abstract The current diagnostic model of autism spectrum disorders (ASD) is characterized by qualitative impairments in social interaction and communication, and the presence of restricted or repetitive behaviors. There are very early signs of social-communication delays that indicate risk for ASD. Several screening instruments have been developed to detect risk for ASD, but few have focused on very young children. The Ghuman-Folstein Screen for Social Interaction (SSI) is a 54-item instrument developed to detect problems with social interactions in very young children. Initial evaluation of the SSI revealed promising reliability estimates, and preliminary support for construct validity. The current study goals included (a) evaluating the SSI s psychometric properties with a large heterogeneous sample of young children with and without ASD, and (b) shortening the instrument. The SSI was completed in developmental and pediatric clinics by caregivers of children with autism or PDD-NOS, non-asd developmental or psychiatric disorders, and for typically-developing children. The sample consisted of 450 children between the ages of 24 and 61 months (mean 43.4 months; SD = 10.6). Diagnostic validity analyses were conducted separately for younger (24-42 months) and older participants (43-61 months). In order to evaluate the tool s factor validity, SSI ratings (n=207) were submitted to exploratory factor analysis (EFA). The relationship of SSI scores to ASD diagnostic measures, age, and developmental level was examined. Diagnostic validity was assessed by comparing SSI scores across different groups. In order to determine ii

4 optimal sensitivity and specificity, Receiver Operative Characteristic (ROC) curves were used in a number of comparisons between different diagnostic groups with the original and shortened scales. A subset of participants from ASD and non-asd clinical groups was matched on age and developmental level. Results of the EFA indicated that a fourfactor solution best fit the data. The factors had good internal consistency and were labeled Connection to Caregiver, Interaction/Imagination, Social Approach/Interest, and Agreeable Nature. The SSI showed moderate convergence with other ASD diagnostic measures, and little relation to age and developmental level. The SSI and its subscales significantly differentiated diagnostic groups. The Interaction/Imagination subscale produced the highest level of group discrimination, and the Agreeable Nature subscale produced the lowest. Binary logistic regression and item mean scores were used to identify critical items, resulting in the 26-item SSI-T ( Toddler version, for younger children), and the 21-item SSI-P ( Preschool version, for older children). In the ROC curve analyses, sensitivity and specificity levels were all >.79, and >.64, respectively, even in the comparisons that controlled for age, developmental level, and verbal ability. The SSI-T and SSI-PS performed better than the full 54-item scale, with less than half the number of items. Scoring recommendations were made based on the ROC results. The SSI is a unique parent-completed Level 2 screen that measures social-communication behaviors in very young children with ASD. This study suggest that the SSI may be a useful screen to discriminate very young children with significant social interaction delays associated with ASD from those with non-asd developmental delays. iii

5 Acknowledgements I wish to thank my advisor and committee members, Dr. Luc Lecavalier, Dr. Michael Aman, and Dr. David Hammer, for their expert contributions and support. Dr. Lecavalier has provided enthusiastic support and advice throughout this process. I wish to thank Dr. Betsey Benson for her guidance and mentorship in the earlier stages of my graduate studies. All of these individuals have taught me a great deal about conducting research and providing evidence-based clinical services. I am indebted to Dr. Jaswinder Kaur-Ghuman for allowing me to conduct this research study on her participant database, and for her advice and feedback along the way. Finally, I am grateful to my husband, family, and friends for supporting me in countless ways throughout this process. iv

6 Vita Edon High School, Edon, Ohio B.A. Psychology, Bowling Green State University Research Associate, The Ohio State University Nisonger Center Graduate Teaching Associate, Department of Psychology, The Ohio State University Summer Graduate Research Associate, The Ohio State University Nisonger Center M.A., Psychology, The Ohio State University Behavior Support Specialist, The Ohio State University Nisonger Center Applied Behavior Specialist, The Arc of Monroe County, Rochester, NY Behaviorist, Rise Services, Inc., Tucson, AZ Publications Aman, M.G., Hollway, J.A., Leone, S., Masty, J., Lindsay, R., & Nash, P. (2009). Effects of risperidone on cognitive-motor performance and motor movements in chronically medicated children. Research in Developmental Disabilities, 30, Aman, M.G., Leone, S., Lecavalier, L., Park, L., Buican, B., & Coury, D. (2008). The Nisonger Child Behavior Rating Form Typical IQ Version. International Clinical Psychopharmacology, 23, v

7 Lecavalier, L., Leone, S., Wiltz, J. (2006). The impact of behaviour problems on caregiver stress in young people with autism spectrum disorders. Journal of Intellectual Disability Research, 50, Lindsay, R. L., Leone, S., Aman, M. G. (2004). Discontinuation of risperidone and reversibility of weight gain in children with disruptive behavior disorders. Clinical Pediatrics, 43, Major Field: Psychology Fields of Study vi

8 Table of Contents Abstract... ii Acknowledgements...iv Vita...v List of Tables... viii List of Figures...x Chapter 1: Introduction...1 Chapter 2: Methods...22 Chapter 3: Results...31 Chapter 4: Discussion...50 References...63 Appendix A: SSI Instrument - Full Scale...70 Appendix B: Tables...76 Appendix C: Figures Appendix D: SSI-T Instrument Appendix E: SSI-PS Instrument vii

9 List of Tables Table 1. Selected Participant Characteristics (Ages months)...77 Table 2. ASD Diagnoses and Recruitment Source for Study Participants...78 Table 3. Item Score Distribution for All 198 Typically-developing Participants Months of Age...79 Table 4. Item Score Distribution for 252 Clinical Participants Months of Age...81 Table 5. Item Score Distribution for all 114 participants with ASD months of age...83 Table 6. Four-factor Solution Using MWL/CF-Varimax Rotation (n=207)...85 Table 7. Pearson s Coefficient of Correlation between the Screen for Social Interaction (SSI) Total Scores and ASD Assessment Tools (Autism Diagnostic Inventory [ADI or ADI-R] and the Childhood Autism Rating Scale [CARS])...88 Table 8. Correlations between the Screen for Social Interaction (SSI) Total Score, Adaptive Behavior scores (Vineland Adaptive Behavior Scales), and Cognitive/Developmental scores (Intelligence or Developmental Quotient)...89 Table 9. Group Comparisons on Participant Characteristics of Interest...90 Table 10. Group Comparisons of SSI Total and Subscales...91 Table 11. SSI items with Significant Difference between Younger and Older Children Presented by Diagnostic Group...92 Table 12. Binary Logistic Regression Analyses for 73 Younger Participants (24-42 months of age) to Predict Membership in ASD (autism or PDD-NOS) versus Non-ASD Clinical Group...93 Table 13. Binary Logistic Regression Analyses for 109 Older Participants (43-61 months of age) to Predict Membership in ASD (autism or PDD-NOS) versus Non-ASD Clinical Group...94 viii

10 Table 14. Mean SSI Scores and one-tailed t-tests between ASD and Non-ASD Clinical Participants...95 Table 15. Characteristics of Matched Subsample...98 Table 16. Sensitivity and Specificity for Discriminating between Autism and Typically-developing Participants...99 Table 17. Sensitivity and Specificity for Discriminating between ASD and Typically-developing Participants Table 18. Sensitivity and Specificity for Discriminating between Autism and Non-Spectrum Clinical Participants Table 19. Sensitivity and Specificity for Discriminating between ASD and Non-Spectrum Clinical Participants Table 20. Sensitivity and Specificity for Matched Groups Table 21. Sensitivity and Specificity for Verbal and Nonverbal Participants (for Those with Classification Available) ix

11 List of Figures Figure 1. SSI Score Distributions for Typically-Developing Participants: A Months of Age, B Months of Age, C Months of Age (full sample) Figure 2. SSI Score Distributions for Combined Clinic sample (including ASD): A months of age, B months of age, C months of age (full sample) Figure 3. SSI Score Distribution for Participants with ASD: A months of age, B months of age, C months of age (full sample) Figure 4. Score Distribution and Recommended Cutoff Score for 26 Item SSI-Toddler (SSI-T) (n=73) Figure 5. Score Distribution and Recommended Cutoff Score for 9 Critical SSI Items for Younger Children (n=73) Figure 6. Score Distribution and Recommended Cutoff Score for 21 Item SSI-Pre-School (SSI-PS) (n=109) Figure 7. Score Distribution and Recommended Cutoff Score for 7 Critical SSI Items for Older Children (n=109) x

12 Chapter 1: Introduction A. Diagnosing Autism Spectrum Disorders (ASD) The current diagnostic model of autism and other autism spectrum disorders (ASD) is characterized by the following three symptom domains: impaired reciprocal social interaction, impaired or delayed communication, and presence of restricted and repetitive behaviors/interests. The social interaction domain includes impairments in: (1) nonverbal behaviors, expressions, and use of gestures, (2) development of peer relationships, (3) shared enjoyment, and (4) social or emotional reciprocity. The communication domain includes: (1) delayed or lack of spoken language, (2) impaired conversational abilities, (3) stereotyped or repetitive language, and (4) lack of spontaneous or pretend play or imitation. The final domain, restricted and repetitive behaviors, includes: (1) circumscribed or stereotyped interests, (2) inflexible adherence to routine, (3) stereotyped or repetitive motor mannerisms, and (4) preoccupation with parts of objects. The Diagnostic and Statistical Manual of Mental Disorders, 4 th edition, Text Revision (DSM-IV-TR; APA, 2000) and the 10 th edition of the International Classification of Diseases (ICD-10; WHO; 1992) require some level of impairment in each of the three separate symptom domains in order to diagnose a person with autism. When an individual does not meet full criteria for autistic disorder or another ASD, a diagnosis of pervasive developmental disorder - not otherwise specified (PDD-NOS) is 1

13 commonly given. In the eyes of parents and clinicians, it is essential to identify children with social and communication delays as early as possible, regardless of the particular diagnosis, in order to begin treatment planning. Evidence that intensive early intervention for ASD is most efficacious when children are four years of age or younger (Harris & Handleman, 2000) makes early detection critical to attaining the most promising outcome for any child with an ASD. While the search for biological markers continues for the detection of ASD, clinicians currently rely upon behavioral, developmental, and historical information. There are a variety of diagnostic instruments available to help clinicians determine the presence of an ASD, ranging from parent checklists to structured interviews to observational tools. The current gold standard for diagnosing autism in children over the age of three years, the Autism Diagnostic Interview-Revised (ADI-R, Rutter, LeCouteur, & Lord, 2003), uses a semi-structured parent interview to assess each core symptom domain in great detail. The ADI-R scoring algorithm assists the clinician to identify the presence of autism, with special emphasis on the behaviors occurring in the four-to-five year age range. The instrument currently provides only a scoring algorithm for autism. The ADI-R may be unreliable for use with children with a mental age under 24 months (Rutter, LeCouteur et al., 2003). Use of the ADI-R requires specialized training, clinical expertise, and up to three hours or more to administer and score. A partner in the family of established diagnostic instruments, the Autism Diagnostic Observation Schedule - Generic (ADOS-G; Lord et al., 2000), allows for standardized assessment via direct interaction and observation. It can be administered with children or adults, with or without presence of phrase speech. One of four available 2

14 modules is administered by the clinician based on the expressive language level and chronological age of the individual being assessed. Interactive play activities allow the examiner to present social scenarios to elicit behaviors that will indicate the presence of social or communication abnormalities as well as repetitive or restricted behaviors. The scoring algorithm is designed to allow the examiner to diagnose autism differentially from other ASD. The ADOS-G has been deemed more accurate valid than the ADI-R when used in diagnosing children under the age of three (Gray, Tonge, & Sweeney, 2008; Chawarska, Klin, Paul, & Volkmar, 2007). The two instruments may be most useful when used in tandem, since they tap into different time periods and sources of information (de Bildt, Sytema, & Ketelaars, 2004). Another well-known instrument is the Childhood Autism Rating Scale (CARS; Schopler, Reichler, & Renner, 1988). It can be administered by a clinician in a relatively short amount of time. The CARS contains 15 items representing typical features of ASD, rated on 7-point Likert scales. A higher total score is interpreted as higher likelihood of having an ASD. Because it is unidimensional, it may have a tendency to over-identify individuals with language and/or developmental delays as having autism, making it useful for identification of individuals in need of service, but less appropriate for discrete discrimination of diagnostic group membership (Lord & Corsello, 2005). Additional instruments specifically designated for preschoolers are discussed in more detail below. A relatively new instrument, the Social Responsiveness Scale (SRS; Constantino, 2002) was designed to assess deficits in the capacity for reciprocal social behavior among older children (ages four to fourteen years) (Constantino, Przybeck, Freisen, & Todd, 2000). The SRS consists of 65 items reflective of social deficits, language difficulties, 3

15 and repetitive behaviors/restricted interests, but factor analysis and latent class analysis suggested one single broad category of deficits (characterized as reciprocal social behavior) (Constantino et al., 2003). While there are no data yet available on discriminative ability, SRS scores were found in one study to be highly correlated with ADI-R algorithm scores (Constantino et al., 2000) and in another study, children with autism, Asperger s Disorder, or PDD-NOS scored significantly higher on the SRS than typical and non-asd clinical samples (Constantino et al., 2003). The SRS is not deemed a diagnostic instrument, but shows promise in helping to identify core features for older children with ASD. Existing diagnostic measures tend to rely upon discrete caseness of autism versus no autism. For the great number of individuals who have clinically-significant symptoms, but who fail to meet the strict criteria for autism, a diagnosis of PDD-NOS is commonly given. The DSM-IV-TR and the ICD-10 define PDD-NOS (or atypical autism) in terms of severe impairment in reciprocal social interaction, with either late onset (after age three) and/or atypical or sub-threshold symptomatology. The social impairment is required, along with impairment in either the communication or restricted/repetitive behavior domain. There is little research on this diagnostic class, and it is generally viewed as a mild case of autism, rather than as a unique clinical entity (Volkmar & Klin, 2005; Lord & Corsello, 2005). Precise diagnosis of autism is easiest and most likely with children who are school-age, verbal, and with milder or no cognitive delay, but there is a greater need for diagnostic discrimination of those who fall outside this category (Lord & Corsello, 2005). Verbal ability is a factor that should be considered in the diagnostic process, especially 4

16 when children are being compared to developmental cohorts by age and developmental level. The presence or absence of verbal ability has been handled in various ways in scored assessment instruments. Some instruments have lower cutoffs for nonverbal individuals (such as in the ADI-R), whereas others, for simplicity of scoring, do not adjust for the number of possible items available to verbal vs. nonverbal children (Rutter, LeCouteuret al., 2003). The great variability in symptom presentation (because of age, level of functioning, and/or verbal ability) makes diagnosis challenging. The field is debating whether the three-domain conceptualization of ASD is optimal. For instance, the current symptom separation of social and communication domains has been questioned (Frazier, Youngstrom, Kubu, Sinclair, & Rezai, 2008; Snow, Lecavalier, & Houts, 2009). Further, the current symptom domain of restricted repetitive behaviors (RRBs) has been better characterized in empirical studies by separation into sensory-motor repetitive behaviors and cognitive rigidity/insistence on sameness (Cuccaro et al., 2003; Szatmari et al., 2006; Carcani-Rathwell, Rabe-Hasketh, & Santosh, 2006). RRBs appear to have different profiles based on age spans, with restricted interests as the most common, and self-injury the least common RRBs across age groups (Esbensen, Seltzer, Lam, & Bodfish, 2009). Likewise, individuals with typical IQ levels at all ages with milder forms of autism often exhibit restricted interests rather than repetitive behaviors (Berument, Rutter, Lord, Pickles, & Bailey, 1999). In keeping with the clinical features of ASD, diagnostic instruments generally include a mix of items that assess both the presence of abnormalities and absence of typically-developing features (Lord & Corsello, 2005). Seven out of eight of the current 5

17 social and communication diagnostic features in the DSM-TR reflect absences or delays, whereas all four of the restrictive and repetitive diagnostic features (by definition) reflect abnormalities. Abnormalities might include presence of hand-flapping, restricted interests, or delayed echolalia, whereas absence of normally developing features might include lack of eye contact or absence of empathy/offering comfort to others. Although the presence of repetitive abnormalities may be easier to describe and report upon, there is evidence that clinical outcome is affected more by developmental absences, especially those of social relatedness (Lord & Corsello, 2005). For all ages and developmental levels, regardless of the instrument or type of items assessed, social impairment remains the central, universally accepted, hallmark of autism. Regardless of the current debates over item content or diagnostic nosology, the autism diagnosis is viewed as culturally universal and remains similar to Kanner s 1943 original description in many ways. Changes in definitions of the autism diagnosis over the last 20 to 30 years have primarily reflected changing consensus over age of onset and the number of behavioral features required to meet diagnostic criteria in each of the three original behavioral domains (Volkmar & Klin, 2005). Diagnostic uncertainties arise mostly with individuals at very low or high levels of (1) chronological age, (2) verbal ability, and (3) general cognitive skills (Lord & Corsello, 2005). B. ASD in infants and toddlers Developing reliable ASD diagnostic criteria for infants and toddlers is difficult due to a host of factors, including rapid developmental change and growth in that period, the fact that clinical concern over absences or abnormalities depends upon chronological and developmental age, change of some symptoms with age, and the effect of testing 6

18 environment and procedures for such young individuals (Chawarska & Volkmar, 2005). Developmental regression (the early loss of skills, such as spoken words) is sometimes cited as another complicating factor, though its scope and prevalence is debated (Volkmar & Klin, 2005). Each contributing factor adds a surprising amount of complexity to efforts to sort out ASD from other developmental disabilities. This diagnostic ambiguity fades as children age, because autistic-like symptoms fade among children with non-asd disabilities, whereas true autism symptoms often become more intense with time (Chawarska & Volkmar, 2005). Children between ages two and three years diagnosed using the ADI-R are overwhelmingly more likely to fall out of the spectrum rather than into it (Lord, 1995; Kleinman, Ventola, et al., 2008). The need for good discrimination between clinical diagnoses (and thus treatment trajectory) is extremely important in the youngest, most complex populations. Diagnostic instruments have traditionally focused on the four- to five-year age range. There are several reasons for this (Rutter, LeCouteur, & Lord, 2003). First, ASD symptomatology is presumably most recognizable and prototypical during this age span. Second, there are communication and social abnormalities that children with autism over age five may have outgrown but are none the less key to deciding whether the disorder should be diagnosed. Third, a focus on delays in behaviors of the very early years (under the age of three, for instance), would necessarily be much more confounded by level of cognitive and language development. Further, children under the age of 24 months may lack the necessary skills to display typical symptoms of autism. Thus, clinicians need to 7

19 use considerable caution in ensuring that apparent autistic-like features are not a consequence of developmental delays. Despite all the challenges, detection and diagnosis are nevertheless progressing in this young age group (e.g., around 18 to 36 months of age). Traditional clinical reliance upon symptoms present during the ages of four and five years does not make sense for this age group. Since infants under age three cannot demonstrate the breadth of behaviors required in the current DSM-IV-TR diagnostic criteria for autism, new criteria may be required (Lord, 1995). In the meantime, diagnosis before the age of three is possible, but some flexibility may be in order with the use of existing criteria and practices. Clinical judgment should always be favored over absolute reliance on cutoff scores in standardized instruments. This is especially true when assessing infants and toddlers around the age of two or three, where diagnostic instruments can produce unstable results (Chawarska & Volkmar, 2005). Lord (1995) assessed children at age two and again at age three, and found that clinical diagnosis was much more accurate than the ADI-R in providing a stable diagnosis. Several ADI-R items distinguished children with and without autism at age three but did not do so at age two; specifically, scores for the children with autism increased, and scores for the group without autism decreased. In a validity study of diagnostic instruments in young children, Gray, Tonge, and Sweeney (2008) found that the ADOS predicted clinical diagnosis better than ADI-R for children between the ages of two and six years. Kleinman, Ventola et al. (2008) conducted assessments at two time points (at approximately age two and then at age three or older) and found somewhat more stable diagnoses between the two points using the 8

20 ADOS and the CARS than with the ADI-R. However, clinical diagnosis using DSM-IV criteria resulted in the best overall stability. For diagnosis of children under three or four years of age, clinical judgment is required in scoring for the restricted repetitive behaviors (RRBs) domain. It has already been noted that factor analytic studies of RRBs suggest two sets or types of behaviors (i.e., sensory-motor repetition vs. cognitive rigidity), and these most likely change with age and developmental level. It is also the case that RRBs can be hard to detect in very young children. Parents of very young children may miss or underreport abnormal behaviors (like RRBs) in comparison with a clinician s observation (Wiggins & Robins, 2008). However, around age two, typically measured repetitive behaviors aren t usually expressed yet, such that current diagnostic cutoffs and criteria for this domain cannot be expected to be relevant to this young age (Volkmar & Klin, 2005). For example, among a large sample of children age 16 to 37 months, the ADI-R had better sensitivity, specificity, and agreement with other ASD instruments when the RRB domain was ignored (Wiggins & Robins, 2008). Similarly, Cox et al. (1999) found that RRBs did not emerge at 20 months but only by 42 months. In addition, RRBs failed to differentiate the ASD groups at either time point. The authors pointed out, however, that it is unclear if RRBs have a significant presence before the age of three or four, or if current diagnostic instruments simply do not include the relevant behaviors for such young children. Indeed, researchers have been advocating expansion and re-tooling of RRB domains in diagnostic instruments and/or scoring algorithms for young children (Lecavalier et al., 2006; Gray, Tonge, & Sweeney, 2008). Using a new set of RRB interactive assessment procedures, Ozonoff et al. (2008) recently found differences in 9

21 atypical object exploration (spinning objects, and especially unusual visualizations) at 12 months of age that significantly predicted prospective ASD symptoms and diagnosis at 36 months. Despite the difficulty in identifying certain ASD symptoms in very young children, there are a number of reliable early signs being identified that are specific to infants and toddlers. Most of these early signs manifest as delays in basic social and/or communication behaviors. Specifically, joint attention, play skills, motor imitation, and use of gestures repeatedly emerge in independent studies of delays or deficits in very young children with autism (Chawarska & Volkmar, 2005). Deficits or delays in motor imitation and joint attention may help discriminate two- or three-year-olds with autism from peers with developmental disabilities, as well as being predictive of language development (Baron-Cohen, Allen, & Gillberg, 1992; Stone, Coonrod, & Ousley, 2000). In preschool children, impairments in motor imitation, sharing and responding to affective information, and joint attention have emerged in comparison to developmentally-matched controls (Coonrod & Stone, 2005). Lack of spontaneity in functional and symbolic play and marked insensitivity to the sound of one s own name have been identified by others (Chawarska & Volkmar, 2005). Of these early signs of ASD, impairment in joint attention is the most cited as a reliable marker (e.g., Ventola et al., 2007). Joint attention typically emerges between 6 and 18 months of age, and has been defined as the young child s ability to coordinate visual attention with his or her social partner (Mundy & Burnette, 2005). Mundy and Burnette described these attentional behaviors as manifesting through areas such as eye contact; affect; gestures like pointing/showing; head turns/gaze following; which are 10

22 especially important interactions because they seem to support much of the incidental and informal development of language and understanding of the world. When these skills eventually develop among children with ASD, there may be lack of flexibility and an atypical developmental sequence (Chawarska & Volkmar, 2005). There is a new module of the ADOS that is under development for ages 12 to 30 months, the ADOS Toddler Module, which places emphasis on joint attention, imitation, and play (Luyster et al., 2009). In this initial study, the authors were able to validate the instrument down to 15 months of age. Initial predictive validity of proposed algorithm scores to best estimate clinical diagnoses was very promising, with accurate categorization of approximately 90% of children with ASD and non-asd diagnoses. Because of the uncertainty and instability of diagnosis with such young children, the authors suggested a unique range of concern approach to interpreting individual scores, such that different scoring cutoffs indicate little-to-none, mild-to-moderate, or moderateto-severe concern for presence of clinically-significant social and expressive language deficits. About 75% of children were classified as being in the same range of concern after approximately two months time. Interrater reliability was reportedly better for older and verbal children than for younger and nonverbal children. According to the authors, the ADOS Toddler Module provides a standardized description of the behaviors that are observed at a certain point in time for very young children who are diagnosed with ASD by expert clinicians. This qualification highlights the ongoing complexity of diagnosing these very young children. Regardless, this new ADOS module makes an important and promising contribution to these efforts. 11

23 Clinical diagnosis is required for children to be identified and referred to necessary treatments. As outlined in this discussion, the differences between the ways in which very young children manifest symptoms and the current ASD diagnostic criteria have resulted in modified approaches to assessing this population. Higher rates of false negatives can occur when diagnosing toddlers at 18 months of age using current diagnostic approaches, in part because behaviors required for ASD diagnoses do not usually manifest as a clinical concern until around 24 months of age. However, even with improved item content for children below the age of four, the specialized training, time commitment, and/or financial expense required to use most diagnostic instruments render them neither feasible nor appropriate for general practitioners and caregivers who wish to identify children at risk of ASD at the earliest possible stage of development. Thus, brief, sensitive, and reliable screening instruments are needed for early detection of ASD to occur in non-specialist settings. A number of instruments have been developed for young children and are described below. C. Screening Instruments for Early Detection of ASD Key considerations: Purpose. The American Academy of Pediatrics (2006) recently recommended that all children be screened for autism at 18 months. Screening instruments differ from diagnostic instruments in that they reveal risk of disability rather than certainty of a specific diagnosis (Coonrod & Stone, 2005). Autism screens vary with regard to their purpose, in that some aim to identify children at risk for autism alone, while others focus on ASD in general. Some focus on qualitative deviance, and others focus more on the absence of behaviors found in typical development. Regardless of specific purpose or 12

24 content, all items should reflect behavioral markers that are easily identified by a caregiver and/or professional with little direct experience or knowledge of autism. Level. Screening instruments for disorders or diseases are commonly classified in two levels (Stone, Coonrod, Turner, & Pozdol, 2004). Level 1 instruments are intended for use with the general population (e.g., in a primary care or pediatrician office), whereas Level 2 instruments are intended for use in developmental clinic or evaluation program settings to help differentiate children at risk for a particular disorder (such as autism) from children at risk for more global developmental problems or other specific disorders (Coonrod & Stone, 2005). Most ASD screening instruments currently available are Level 2 screens. Administration. Screens are designed to be filled out independently by a caregiver or with assistance from a clinician or medical professional. Most rely upon the former method. Caregiver-completed instruments take less time and money to administer, especially in situations where there is relatively limited time with clinicians. It has been suggested that parents may concentrate on the absence of typical behaviors, whereas clinicians may notice the atypical behaviors that parents may miss (as long as they are not so subtle or context-dependent that they cannot be observed in the clinical setting) (Wiggins, Bakeman, Adamson, & Robins 2007; Stone et al., 2004). Outcome. Following initial screening, a child may be referred to a diagnostic clinic, followed by needed services or therapy (Coonrod & Stone, 2005). Corsello et al. (2007) discussed the complexity of choosing cutoffs for making decisions about referral to follow-up treatment, in light of different instrument purposes, ages, and samples used. 13

25 Knowing the purpose and limitations of each instrument is important in helping clinicians make appropriate decisions regarding follow-up. Besides the usual reporting of reliability and validity, psychometric properties of screens include the instrument s ability to discriminate diagnostic groups of interest accurately. Sensitivity is the proportion of individuals correctly identified as being at risk for a disorder, and specificity refers to the proportion of individuals correctly identified as not being at risk. As these are simply proportions of cases, values range between 0 and 1. Higher values indicate greater accuracy of identification in a particular sample. Scoring cutoffs or thresholds are chosen in order to maximize each of these indices; however, a rise in one necessarily lowers the other. Two other proportions that are usually reported, namely the Positive Predictive Value (PPV) and Negative Predictive Value (NPV), indicate more specifically the number of individuals identified by the screen as at risk (or not) who actually have (or do not have) the disorder of interest. Since PPV and NPV are directly related to the prevalence or base rate of the disorder, each of these indices is dependent upon the sample used (Coonrod & Stone, 2005). Relatively accurate identification is important to avoid over- and under-referral to full diagnostic assessment. The goal of screening is to catch as many children as possible who actually have the disorder, while avoiding an undue number of false positives. More importance is generally given to sensitivity than specificity in screening instruments (Stone et al., 2004). False negatives (diagnosed ASD but screened negative) can occur for several reasons. For example, higher- functioning children with milder ASD symptoms can easily be missed by screens because their symptoms may be more 14

26 subtle, and lower functioning, nonverbal children can easily miss the cutoff score due to inability to demonstrate certain symptom classes (Eaves, Wingert, & Ho, 2006). Current ASD screening instruments One of the most widely-used screening instruments that have been developed to help detect suspected ASD in very young children is the Modified Checklist for Autism in Toddlers (M-CHAT; Robins, Fein, Barton, & Green, 2001). The M-CHAT includes 23 items across the three main symptom domains and is designed as a Level 1 instrument for use by parents and professionals to complete at a toddler s 18-month pediatric checkup. The M-CHAT is still under investigation with regard to appropriate cutoff scores to achieve the optimal ability to predict ASD diagnosis within different settings (Robins et al., 2001; Snow & Lecavalier, 2008). Robins and Dumont-Mathieu (2006) found that most studies of the M-CHAT show the different choices of cutoff criteria established thus far produce good sensitivity but less acceptable specificity (i.e., they tend to produce too many false positives). In response to this, the developers of the MCHAT introduced the use of a structured telephone follow-up to clarify written responses provided by the parent(s) (Robins et al., 2001). A recent large-scale replication and follow-up study found similar predictive validity to the 2001 study, with special emphasis on low- and high-risk samples of children (Kleinman, Robins, et al., 2008), In addition, the scale demonstrated good ability to predict diagnosis at age four, especially with the addition of the telephone follow-up screening. A second screening instrument that has gained widespread use is the Social Communication Questionnaire (SCQ; Rutter, Bailey, & Lord, 2003). The SCQ is a 40- item Level 2 screen validated for age four and older, based on items from the ADI-R 15

27 (Lord, et al., 1994) and is intended to be completed independently by caregivers. One criticism of the SCQ is that since its items were developed from the ADI-R (a clinicianadministered instrument), some items may not be well suited to a screening instrument that is completed independently by caregivers, without guidance or item context (Snow & Lecavalier, 2008). In addition, the SCQ validation sample included only individuals four years of age and older (Berument, et al., 1999), which follows from the fact that the source of the SCQ items, the ADI-R, is especially focused upon the behaviors present between the ages of four and five. Snow and Lecavalier (2008) assessed a clinic sample of young children between the ages of 18 and 70 months of age using both the M-CHAT and the SCQ. The study assessed how both screens performed in the month range, which is outside the original validation age range for each instrument. For selected scoring criteria, they found only slightly decreased accuracy in diagnostic classification for both the M-CHAT and the SCQ for this in between age range compared to the full age strata. Both the M- CHAT and the SCQ had much higher sensitivity than specificity, and they seemed best suited to classify children with ASD at lower adaptive and intellectual levels. A third instrument, the Screening Tool for Autism in Two-Year-Olds (STAT) (Stone & Ousley, 1997) is a 12-item interactive Level 2 screening tool intended for use with children 25 to 35 months of age. The authors designed the instrument with a broad range of items that do not require presence of language comprehension abilities (Stone, Coonrod, & Ousley, 2000). Emphasis is placed on play, imitation, and joint attention, with the idea that the absence of behaviors in these areas may be early signs of ASD, as well as being predictive of language development. The STAT differs from other 16

28 screening instruments in that it relies upon clinician interaction and observation. It was designed to aid in differentiating autism from non-asd developmental delay. In an initial study of a clinic-based sample of 2-year-olds suspected of developmental disability, using an algorithm of failing (i.e. score of <2) in two of the three areas, a sensitivity of.83 and a specificity of.86 were obtained (Stone et al., 2000). The authors also examined a subsample of the children matched on age and developmental level, and found that the chosen algorithm differentiated those with autism from those with non- ASD developmental delay at nearly the same levels as the full sample. A more recent study revealed slightly higher levels of predictive ability (sensitivity.92, specificity.85) (Stone et al., 2004). There is also suggestion of clinical utility for children under 24 months of age (Stone, McMahon, & Henderson, 2008). The STAT shows promise as an interactive Level 2 screen for two-year olds, where resources and referral status make this a feasible choice. A fourth screen that bears mentioning is another emerging instrument, the Quantitative Checklist for Autism in Toddlers (Q-CHAT) (Allison, Baron-Cohen, Wheelwright, Charman, Richler, Pasco, et al., 2008). The Q-CHAT has 25 items, scored on a 5-point scale, intended as a Level 1 screen for identifying toddlers at risk for ASD in the general population. In the 2008 study, the Q-CHAT was administered to a large unselected sample of toddlers ages 18 to 24 months along with toddlers and preschool children with ASD ages 19 to 63 months. An approximately normal distribution of scores was found for the general population sample. The ASD sample scored significantly higher (more autism symptoms) than the general population sample. Good test-retest reliability was found. The Q-CHAT is still in under development. 17

29 There is a need for reliable and valid screening instruments for very young at-risk children. As discussed previously, the behavioral repertoires of children under age three or four may not be broad enough to display some of the DSM-IV-TR symptoms (like impairment in the ability to sustain a conversation, or presence of circumscribed interests). However, impaired social interaction is the cardinal feature of ASD for all age ranges. Indeed, across ASD subtypes, the social domain of the ASD diagnostic model is the one non-negotiable core feature (APA, 2000). Screening efforts for very young children should attend to very basic social interaction abilities with emphasis on the earliest signs of impairment that seem to distinguish ASD from general developmental delay (i.e., joint attention, imitation, gestures, and play behaviors). D. The Ghuman-Folstein Screen for Social Interaction (SSI) The Ghuman-Folstein Screen for Social Interaction (SSI) (Ghuman, Freund, Reiss, Serwint, & Folstein, 1998) is a 54-item instrument developed for use as a comprehensive Level 2 screen in very young children. See Appendix A for the SSI instrument. The purpose of the instrument is to provide a quick method for healthcare professionals to determine if there are social delays that indicate a need for further assessment or intervention. The SSI defines the basic capacity for social interaction in terms of a child s ability to initiate and respond to social interactions across a variety of situations. The focus is on the child s attempts to interact and responses to social interactions, rather than on how successful their attempts are. The screen items assess mostly typical social developmental milestones rather than deviant behaviors. Items were developed by consulting the ADI-R, Vineland Adaptive Behavior Scales (VABS) 18

30 Socialization subscale items (Sparrow, Balla, & Cicchetti, 1984), and from clinical experience. The items are positive (prosocial) and are scored on a four-point scale of frequency (0 to 3; the child displays the behavior almost never to almost all the time ). Thus lower scores reflect a slower or delayed course of development, and higher scores reflect more normative development. The items are based on observed behaviors, so that completion requires little inference or expertise from the caregiver. The continuous scoring method potentially allows for detection of differences among children with varying diagnoses and detection of changes within a single child s social behaviors over time. The authors of the SSI endeavored to develop a screen that focused on the basic capacity and purpose of social interaction without being confounded by verbal ability. Emphasis was placed on joint attention skills, which are of particular concern for those with ASD (Ghuman, Peebles, & Ghuman, 1998; Mundy & Burnette, 2005). The developers felt that an instrument with this focus could be useful for detecting social interaction problems associated with other developmental disability besides just those associated with the ASD. In a prior publication of the SSI, psychometric properties were reported in a sample of 51 clinic children and 60 typical controls between the ages of 24 and 61 months matched on age, ethnicity, and socio-economic status (Ghuman, Freund et al., 1998). Internal consistency for all items was.76. Test-retest reliability was fair to good depending upon sample (r =.88 for whole sample; r =.91 for clinical group; and r =.51 for controls). Interrater reliability for the clinical and control groups were.67, and.51, 19

31 respectively. One SSI scoring method was Total % score, which was a ratio of the child s total score to the maximum score possible for the age group (based on typical developmental milestones for each age group). Group comparisons by age and clinical diagnosis showed that, as expected, the control group scored significantly higher on the SSI Total score, SSI Total % score, and VABS Socialization subscale, and obtained lower scores on the ADI-R. SSI Total score converged with VABS-Socialization scores (r =.67) and SSI Total scores and Total % scores for the clinical group converged with algorithm items of ADI-R categories and Total scores (r range = -.55 to -.88, within age groups). Finally, the SSI Total % scores significantly differentiated the subjects in the clinical group who also met all or part of the ADI-R algorithm criteria. At present, there are no established cutoff scores for the SSI. In the Ghuman, Freund et al. (1998) publication, a criterion of one standard deviation above the mean was chosen as the best cutoff to balance false positives and false negatives. Initial sensitivity and specificity estimates using this cutoff were.83 and.81, respectively, for the younger children (ages months), and.75 and.86, respectively, for the older ones (ages months). Of note, these estimates were based on discrimination of a heterogeneous clinical group and a typically-developing control group. The SSI is appealing for a number of reasons. First, the instrument was developed to minimize the impact of language ability on the assessment of basic social interaction. This is an important consideration for a screening instrument, given the wide developmental range of the clinical populations for whom screens are especially relevant, as well as the difficulty in differentially diagnosing children at lower cognitive levels. Second, the items assess social interactions, the most important group of symptoms for 20

32 infants and toddlers. Third, the items are rated on an ordinal scale instead of a dichotomous scale like many other screens, which should increase the ability to conduct fine-tuned assessment of social behaviors across different clinical groups and ages. Finally, parent informants have previously rated the instrument favorably in terms of length, relevance, and comprehensible nature of the items (Ghuman, Freund et al., 1998). E. Study Objectives The current study objectives include an evaluation of the SSI s psychometric properties with a large, heterogeneous sample of young children with and without ASD. It will assess the tool s factor structure and diagnostic validity. For analyses of diagnostic validity, the sample will be evaluated in 18-month age bands, specifically 24 to 42 months of age, and 43 to 61 months of age. These procedures will help to establish age-specific scoring algorithms, and may lead to refinement of the instrument with regard to length. Evaluation of the SSI with a large heterogeneous sample will not only help establish its utility as a screening instrument, but also will add to the body of empirical investigations that are attempting to refine the autism phenotype. Given the primacy of social interaction deficits in discriminating all children with ASD (but especially infants and toddlers), this screen is potentially very useful. Szatmari et al. (2002) called for analysis of social behaviors to help determine which behaviors reflect developmental disability in general versus specific deficits of autism or ASD. Doing so with a continuous measure of developmental items, like the SSI, is especially relevant. The SSI, which was designed to assess a broad sample of basic social interaction skills without the precondition of verbal skills could be a very valuable tool for clinicians. 21

33 Chapter 2: Methods A. Participants The participants were recruited from three separate sources: (1) referrals to various infant/child autism or developmental clinics at the Kennedy Krieger Institute (KKI), (2) public and private pediatrician offices in a large city in the northeastern U.S., and (3) specialty research clinics at KKI for Fragile X syndrome, Rett s Disorder, and Smith-Lemli-Opitz Syndrome (SLOS). Inclusion criteria. Due to the nature and purpose of the SSI, the age span was limited to 24 to 61 months of age. Participants with greater than 5% of the SSI items left blank (4 or more items) were excluded. Of the 450 participants between 24 and 61 months of age, 75 participants were missing between one and three SSI items. The majority of these (n= 58) were missing only one item, while 17 individuals were missing two or three items (of which 7 were typical controls). The missing items appeared to be missing at random, as there was no detectable pattern by age, gender, race, or informant. Missing items are always a risk with independently-completed instruments. The process we used for replacement of these values is described in the Results section with the factor analysis procedures. Table 1 summarizes the characteristics of the study participants. The participants from the autism, developmental, and preschool clinics (n=164) included children with ASD (autism or PDD-NOS), general developmental delay, a developmental language 22

34 disorder, ADHD, and/or other developmental or psychiatric concerns. The SSI was also completed for 88 individuals from specialty research clinics in the following categories: individuals at risk for Fragile X (n=57), Smith-Lemli-Opitz Syndrome (SLOS) (n=18), and Rett s Disorder (n=13). A total of 198 typically-developing children had the SSI administered in pediatrician offices. Each participant group ranged from 24 to 61 months of age, but the typically-developing children were younger (mean 40.6 months) than the other groups, and the individuals from the Rett s Disorder clinic were the oldest (mean 48.7 months). The higher proportion of boys in most of the clinic samples was expected, given the high number of children with ASD from these settings. Rett s Disorder is exclusively diagnosed in females. The typically-developing children were split evenly by gender. The indices of socio economic status ranged from 26.4 (Fragile X clinic) to 55.0 (developmental clinics sample). In all cases, the SSIs were completed by a caregiver familiar with the child, who was most typically the mother, father, grandparent, or foster parent. Mothers completed the SSI in approximately 82% of cases. With regard to the specialty clinic participants, the diagnoses assigned warrant a brief description. Fragile X Syndrome is caused by a mutation of the FMR1 gene on the X chromosome. Symptoms can include general cognitive delay, some characteristic physical features, social abnormalities, and speech delay. Males are more affected by the disorder than females, and a significant portion of individuals with Fragile X meet diagnostic criteria for ADHD or an ASD (Hayes & Matalon, 2009). SLOS is a genetic disorder that causes deficiency of cholesterol in the body. It is accompanied by intellectual disability, along with a host of physical and sensory problems. Most individuals with SLOS also have an ASD (Sikora, Pettit-Kekel, Penfield, Merkens, & 23

35 Steiner, 2006). Rett s Disorder is an ASD exclusive to females, characterized by decelerated cranial growth, loss of purposeful hand movements, and overall developmental loss, including loss of social engagement (Chahrour & Zoghbi, 2007). Previous publication of study data. Data from a portion of the clinical and control groups (up to 111 participants) from the current dataset were used in the original reliability and validity pilot study reported by Ghuman, Freund et al. (1998). The objectives, procedures, and analyses in the current study are sufficiently different from those in the 1998 report to make reasonable the use of these participants as a part of the current, larger dataset. B. Procedures and Measures: Study Exemption. This research study was given a determination of Exempt from Review on January 29, 2009 by The Ohio State University Office of Responsible Research Practices under Category 4 Exemption, the study of de-identified existing data. The SSI was administered to parents or caregivers of clinically-referred participants either before or after diagnostic assessment was completed. Some caregivers chose to take the instrument home to fill it out and send it back to the study administrator. For the participants recruited in pediatrician s offices, the SSI was administered as parents or caregivers waited for appointments with their primary health care provider. A demographic form was included with each SSI. Requested information included the child s date of birth, gender, and ethnicity, along with education and occupational information about the informant and her/his spouse, as applicable. For portions of the clinical groups, diagnostic evaluation and measurement of adaptive or cognitive/developmental level were obtained. Approximately 72% of the 24

36 children in the clinical groups received expert diagnosis for presence of an ASD by a child psychiatrist, psychologist, or multidisciplinary team, with approximately 80% of these having the ADI, ADI-R, and/or the CARS administered as part of the evaluation process. Diagnostic assignment of autism or PDD-NOS was ultimately determined via clinical judgment using DSM-IV criteria (APA, 1994), though there was high agreement with available ADI or ADI-R scores with regard to assignment of autism versus other disorder. Assessments of adaptive behavior and/or intellectual/developmental levels were conducted for approximately 49% of the clinical referrals. Adaptive behavior was evaluated using the VABS Composite Score and/or its Social and Communication subscales. The two most common modes of assessing intellectual/developmental level were the Stanford-Binet Intelligence Scales, 4 th edition (Thorndike, Hagen, & Sattler, 1986) and the Bayley Scales of Infant and Toddler Development (Bayley, 1993). Table 2 shows the number of participants with and without ASD diagnoses by recruitment source. A total of 172 diagnostic evaluations for the presence of an ASD resulted in 114 confirmed diagnoses of autism or PDD-NOS, and 68 non-asd diagnostic determinations. The 50 clinical participants who were not assessed for an ASD had high probability of deficits in social interaction, and/or other developmental or psychiatric concerns. C. Data Analyses: Data analyses of the SSI were conducted in the following three stages: tests of data integrity, assessment of construct validity, and assessment of diagnostic validity. Each is described below. 25

37 Tests of Data Integrity Group distributions were analyzed with regard to centrality, normality, and presence of outliers. Item score distributions and item-total correlations were calculated separately for clinical referrals and for typically-developing participants. Construct Validity Factor structure. Exploratory factor analysis (EFA) was conducted with SSI items from all clinical referrals to determine the tool s structure and help establish construct validity. All clinic-referred participants were used in these analyses (i.e., typically-developing children were excluded). This was thought to be appropriate for assessing the factor structure of the SSI given the overlapping concerns about general development along with delays or abnormality in social interaction among many of these children. The statistical program Comprehensive Exploratory Factor Analysis (CEFA; Browne, Cudeck, Tateneni & Mels, 2004) was used to assess potential factor structures. Oblique rotation with Maximum Wishart Likelihood (MWL) discrepancy function and Crawford-Ferguson equivalent of Varimax (CF-Varimax) rotation criteria were chosen. Oblique rotation is preferable to orthogonal since it allows (but does not force) factors to correlate with one another. MWL discrepancy function provides some opportunity for tests of model fit, such as the root mean square error of approximation (RMSEA). Varimax (as opposed to Equimax) rotation criteria tend to push factor loadings toward one and zero, which can help interpretation and determination of factors. These procedures adhere to current standards and recommended procedures for the use of EFA in psychological research (Norris & Lecavalier, 2009). The SSI data met the assumption 26

38 of multivariate normal distribution for the exploratory factor analysis and for the use of MWL. Measures of internal consistency using Cronbach s alpha were calculated for the retained items in each derived subscale. Convergent validity. The SSI s convergent validity was assessed separately for younger (24-42 months of age) and older participants (43-61 months of age) via correlations with available scores on ASD diagnostic measures, which included the ADI- R, its subscales, and the CARS. The impact of participants age, intellectual/developmental level, and adaptive level on SSI scores was analyzed via Pearson s and Spearman s correlations. Diagnostic validity Group comparisons among diagnostic groups (autism, PDD-NOS, non-asd clinical, typically-developing) were conducted on the SSI Total Score and on its derived subscales using factorial analysis of variance. This was followed by evaluation of how individual item means changed between age groups (younger versus older). Identification of critical items. In order to identify critical items, binary logistic regressions were conducted with SSI items as predictor variables and diagnostic membership (ASD vs. non-asd) as the binary outcome variable. Analyses were conducted separately for younger (ages 24 to 42 months of age) and older participants (ages 43 to 61 months of age). Since the discriminatory performance of individual items would not necessarily depend upon their contribution to the determined factor structure, it was decided that all items would be used in the logistic regressions. Due to the relatively small number of positive cases (ASD diagnoses) (52 younger and 62 older participants) 27

39 to the number of predictors (54 SSI items), items were entered in seven separate blocks ranging from six to nine items. This brought the cases-to-predictors ratio closer to the recommended level of approximately 10 or higher (Peduzzi, Concato, Kemper, Holford, & Feinstein, 1996). Stepwise procedure was chosen in light of the exploratory nature of the analyses and because of the relative number of cases and predictors (Menard, 2002). Backward stepwise procedure was chosen, which begins with a saturated model (i.e., all variables are initially included). This type of model (backward stepwise procedure) is superior to forward stepwise in uncovering suppressor effects or identifying significant variables that depend on another independent variable for expression (Menard, 2002). Significance testing in logistic regression is similar to linear regression but it uses the likelihood function for a dichotomous outcome variable (Hosmer & Lemeshow, 2000). One choice for significance testing of a model, the -2 log likelihood statistic, is a measure of model fit or likelihood of the fitted model, and is the basis for the likelihood ratio test. The researcher s model is compared to a reduced model of the likelihood of dependent binary group membership without additional predictors (Hosmer & Lemeshow, 2000). The method chosen for this study was the significance of the change in -2 log likelihood if each item was removed from the model. Thus SSI items whose removal would significantly impact the regression model were viewed as critical items. The outcomes were compared to the results of forward stepwise analyses. The results were expected to be very similar to the backward model. When differences do occur, it is usually due to the backward model uncovering relationships missed by forward selection (Menard, 2002). 28

40 Assessment of item discrimination and redundancy. SSI item means were compared between ASD and non-asd clinical participants using independent samples t- tests. Significant items, alongside critical items, were considered for inclusion in a brief SSI scale for each age group. Inter-item correlations for SSI items were also examined to identify any redundant items that could be removed within each age group. Determination of cutoff scores. Receiver Operating Characteristic (ROC) curve analyses, using various group divisions (autism, combined ASD, non-asd clinical, and typically-developing) were conducted within each age band to determine cutoff scores for optimal sensitivity and specificity. The comparisons progressed from least stringent (autism versus typically-developing) to most stringent (combined ASD versus non-asd clinical). Comparing children with ASD (including PDD-NOS) to those with non-asd clinical diagnoses is stringent because these two sets of individuals can share many clinical characteristics and developmental concerns, often making differential diagnosis challenging. ROC curve analyses were also conducted for a subset of ASD and non-asd participants matched on age and intellectual/developmental level, as well as for verbal and nonverbal participants. For most sets of comparisons, three SSI scoring methods were assessed, as follows: (1) all 54 items, (2) a brief version of the SSI for each age group, and (3) critical items identified in the binary logistic regressions for each age group. Scoring recommendations based on ROC curve analyses were then developed separately for younger and older participants for the brief SSI scales and for the total score of critical items identified in the binary logistic regression. 29

41 Summary of subsamples used in the different analyses: (A) For tests of data integrity: 450 participants with < 3 missing SSI items (all 252 clinical referrals, 198 typically-developing participants). (B) For determination of factor structure: 207 clinical referrals (252 clinical referrals less the 45 children with between one and three missing SSI items). (C) For assessments of diagnostic validity: 350 participants (114 with ASD, 68 non-asd clinical [formally assessed for ASD], 168 typically-developing participants [i.e., with no missing SSI items]). 30

42 Chapter 3: Results A. Tests of Data Integrity Before conducting any of the inferential analyses, the data were examined for normality and for presence of outliers. SSI total scores for the typically-developing participants are depicted in Figure 1. Although there was some slight negative skewness and leptokurtosis (i.e., a peaked shape) to the data, the Kolmogorov-Smirnov test was non-significant, indicating the distribution to be fairly normally distributed (Z value =.895). There were two outliers (defined as being more than three standard deviations from the mean) among the older typically-developing participants. Upon inspection, these scores were not markedly lower than the next higher scores in the distribution. Further, these scores came from two of the six participants reporting Native American ethnicity. Since the scores for typically-developing participants would be used only for basic group comparisons, the two outliers were retained in the analyses. The SSI data for all combined clinic-referred participants are depicted in Figure 2. The distributions for these participants had no apparent abnormalities with regard to shape, and the Kolmogorov-Smirnov Z value was.728, indicating appearance of a normal distribution. These findings indicate multivariate normal distributions for the younger, older, and combined samples of clinic-referred children. This was a favorable finding for the use of these participants scores in subsequent analyses. The final SSI score distributions in Figure 3 depict the scores for a subset of the clinically-referred 31

43 participants, i.e., those diagnosed with an ASD (autism or PDD-NOS). Perhaps not surprisingly, the distributions for younger, older, and combined samples of children with ASD were found likewise to be normally distributed, with no significant outliers. Individual SSI item score distributions for the typically-developing participants, combined clinical referrals, and ASD subset of participants are shown in Tables 3, 4, and 5, respectively. The tables show the item-total correlation, mean score, and percent of endorsement at each response level. A visual comparison of the tables suggests that there are far fewer endorsements of zero ( Almost never ) for the typically-developing participants in Table 3 than the combined clinical and ASD-diagnosed subsets in the other two tables. With regard to mean item scores, the typically-developing children had mean scores of 2 or higher for 41 of the 54 SSI items, and no items with means less than 1. This pattern was different for the combined clinical and ASD-diagnosed subsets of participants, who had means of greater than 2 for 17 and 11 items, respectively, and means less than 1 for 3 items and 15 items, respectively. (To avoid misrepresentation of these results, it bears repeating that the ASD sample in Table 5 is a subset of the combined clinically-referred children in Table 4.) There was more similarity among the groups with regard to which items had nonsignificant item-total Pearson s correlations (i.e., p >.05). Item 28 (upset when left with unfamiliar babysitter) was non-significant for all three samples. For both the typicallydeveloping participants and for those with ASD, item 31 (shy around people s/he doesn t know well) also failed to reach significance. For the participants with ASD, one additional item correlated poorly with the rest of the SSI items (item 47, aggressive with siblings or other children). (See Tables 3-5.) 32

44 No items were removed from the SSI at this point for at least two reasons. First, it was unknown if the items with low item-total correlations would express a separate but clinically important construct in the factor analysis. Second, each item still needed to be assessed for its ability to discriminate between clinical groups of interest before deciding to remove any from the instrument. B. Construct validity Factor structure. SSI items for 207 clinically-referred participants (i.e., all but the typically-developing participants) were submitted to exploratory factor analysis using the procedures described in the previous section. The sample size was 207 because individuals with missing items (n=45) could not be included in the analysis. Three, four, five, and six factor solutions were analyzed. The four-factor solution was chosen as the best fit for the data. This solution had highly interpretable factors and small amounts of shared variance among factors. The three-factor solution was not chosen due to relatively high overlap among factors. The five-factor and six-factor solutions each contained factors that were difficult to interpret. Table 6 shows the four-factor solution. The criterion used for assigning and retaining items was a loading of.33 or higher on a particular factor. This criterion resulted in retaining 48 items and leaving six items unassigned to any factor. The unassigned items (7, 20, 27, 28, 32, and 33) are italicized in Table 6. The four resulting factors were labeled as follows: I. Connection with Caregiver (21 items), II. Interaction/Imagination (13 items), III. Social Approach/Interest (7 items), and IV. Agreeable Nature (7 items). 33

45 Mean factor loadings for factors I through IV were 0.45, 0.59, 0.62, and 0.61, respectively. With regard to goodness-of-fit of the model, the RMSEA of the model was 0.09 [90% confidence interval (CI) = ], which is just above the cutoff determined to be an acceptable model fit by Browne and Cudeck s (1993) standards. Hereafter, the four factors will be referred to as subscales. Items were weighted onto their subscales by using the Likert score endorsed by the caregiver. Cronbach s alpha coefficients were calculated for each subscale. They were as follows: Connection with Caregiver,.92; Interaction/Imagination,.93; Social Approach/Interest,.81; and Agreeable Nature,.82. Mean alpha across the four subscales was.87. These are good levels of internal consistency (Cicchetti, 1994). Replacement of missing values. For the 45 clinically-referred individuals with three or fewer missing SSI items, the individual s mean score for the relevant subscale was used to replace the missing item(s). When the missing item was one of the six unfactored items, the item was left blank. This occurred for 13 individuals. With regard to the 30 typically-developing participants with missing items, using the mean of item groupings derived from the factor analysis of the clinically-referred participants to replace missing items was deemed inappropriate. Therefore at this point it was decided that the 30 typically-developing participants with missing items would be excluded from the rest of the analyses. Convergent validity. The relationship of the SSI to scores on ASD diagnostic measures (the ADI or ADI-R, its subscales, and the CARS) is depicted in Table 7. The Pearson s coefficient and the available number of participants are shown for each relationship. All correlations were significant at p <.01. The negative directions of the 34

46 relationships reflect the fact that the SSI scores increase with typicality, whereas the diagnostic measures decrease with typicality. The strongest correlation occurred with the ADI Social Interaction subscale (r = -.71), and the weakest correlation, though still significant, occurred with the ADI Restrictive/Repetitive subscale (r = -.36). Interestingly, the correlation of the SSI with the ADI Total Score (r = -.67) was much higher than for the CARS (r = -.43). Relationship of SSI with other subject characteristics. Table 8 shows the correlations of the SSI Total Score of each diagnostic group with measures of age, adaptive behavior, and IQ. Pearson s r correlations were used to assess the relationship of age with the SSI. Spearman s rho (ρ) was used to assess the relationship of ordinal ranges of adaptive behavior and IQ with the SSI scores. Adaptive behavior and IQ levels were ordinally coded into eight bands for these analyses, as follows: <24, 25-39, 40-54, 55-69, 70-84, 85-99, , >115. Across all three diagnostic groups, no statistically significant relationships emerged for these variables. While all relationships were statistically non-significant, it bears mentioning that a few of the relationships were particularly low, approaching ρ or r values of 0. Specifically, for the group with autism, the SSI correlation with the VABS Composite score (overall adaptive behavior) was ρ =.064. In the PDD-NOS group, the SSI correlation with IQ level was ρ = -.063, and for the non-asd clinical group, SSI correlation with age was r =.097. C. Diagnostic validity: The following groups of participants were used to assess diagnostic validity: 114 participants diagnosed with an ASD (48 with autism and 66 with PDD-NOS), 68 clinicalreferred participants determined to have no ASD, and 168 typically-developing children. 35

47 Group comparisons of participant characteristics for these groups are depicted in Table 9. Comparisons were conducted using either Chi Square tests or analyses of variance. With regard to age, the typically-developing group was found to be significantly younger than the other groups (F = 9.2; post-hoc p-value <.01). Differences in gender distribution followed the expected demographic patterns of approximately 80% male in the ASD groups, about 50% in the typically-developing group, and an intermediate proportion in the non-asd clinical group. The ASD groups had a higher proportion of Caucasian participants (X 2 = 10.68; p <.05). With regard to informant characteristics, Table 9 shows that no significant differences emerged on mother s education level, but the level of socio-economic status (SES) for the typically-developing children was significantly higher than for the two ASD groups (F = 7.55; p <.01). SES was calculated using the Hollingshead (1975) four-factor index, which is based on a composite of occupational status and educational level, with higher scores reflecting higher SES. Cognitive and adaptive measurements were also compared for the first three (clinically-referred) sets of participants in Table 9. The available number of participants for each variable of interest is shown in the table. Both ASD groups had a significantly lower proportion of verbal participants than the non-asd group (X 2 = 8.97; p <.05). The autism group had significantly lower mean IQ and VABS adaptive behavior composite scores than the non-asd group (F = 6.14, and 8.59, respectively). With regard to available VABS subscales, both ASD groups scored significantly lower on the Social domain than the non-asd group, and the autism group scored significantly lower on the Communication domain than both the PDD-NOS and non-asd groups (see Table 9). 36

48 Group comparisons. Table 10 shows the diagnostic group comparisons for SSI Total and subscales scores. Of note is that the subscales do not include the six items that did not load on any subscale in the EFA. A one-way analysis of variance with a priori contrasts was conducted for the Total Score comparisons. The a priori contrasts predicted significant differences between each and every one of the four groups, with the autism group predicted to be lowest, and the typically-developing group to be highest. The main effect was significant (F = ; p <.001), and the specific a priori predictions were confirmed. Each group s SSI Total score was significantly different from every other group, even between the autism and PDD-NOS participants. With regard to the subscale scores, a multivariate analysis of variance was conducted (four subscales by four diagnostic groups). The main effect for each subscale was highly significant (F values are listed in Table 10; all p-values <.001). Post-hoc analyses were conducted in order to determine which particular groups were significantly different from one another. Scheffé (for equal variances) and Tamhane (for unequal variances) comparisons were chosen, with a significance level of p <.05 used. Two of the subscales (Interaction/Imagination and Social Approach/Interest) showed the same strong pattern as found for the Total Score, in that each group s score was significantly different from every other group. The Connection to Caregiver subscale performed nearly as well, in that the ASD groups together were significantly lower than the non- ASD clinical group, which was in turn significantly lower than the typically-developing group. For the final subscale, Agreeable Nature, only the typically-developing group was significantly different from any other group. 37

49 Two subsequent multivariate analyses of variance were conducted for SSI Total and subscale scores in order to examine the impact of certain subject characteristics. First, SSI scores were assessed by gender and by diagnostic group. Diagnosis was entered as a covariate to evaluate any potential interaction effects with gender. However, no main effect for gender was found. Second, scores were assessed by age group and verbal ability. Verbal participants had significantly higher scores on the Interaction/Imagination subscale (mean of 18.1; SD = 10.1) than nonverbal participants (mean of 12.3; SD = 10.2) (F = 9.48; p =.003). Older children (43-61 months of age) among this subset had significantly higher scores than younger participants (24-42 months of age) on Interaction/Imagination (means of 18.0 and 12.6, respectively; F = 8.20; p =.005), and on Social Approach/Interest (means of 11.6 and 9.5, respectively; F = 5.69; p =.019). No interaction was found for verbal ability and age group. Item score change by age group. For the remaining analyses, the participants were examined in two 18-month age bands, i.e., months of age, and months of age. Table 11 depicts the SSI items with significant mean differences in independent samples t-tests between these younger and older age groups. Results are presented by diagnostic group. Twenty-one SSI items showed significant differences between the age groups (p <.05). At least three patterns can be detected in these results. First, the autism group had only three items with significant mean differences between age groups, and all three were from the Connection to Caregiver subscale. In contrast, the PDD-NOS and non-asd clinical groups had six and seven significant items, respectively, almost exclusively coming from the Interaction/Imagination, Social Approach/Interest, and Agreeable Nature subscales. Second, only three items, (concern when other children are 38

50 upset [41], liked by other children [45], and other children want to play with him/her [46]), were significant for more than one diagnostic group. Finally, nine items had lower endorsement (i.e., the behavior was reportedly exhibited less frequently) by older versus younger participants, a phenomenon that was much more common in the non-asd groups than in either ASD group. Identification of critical items. The results of the binary logistic regressions are presented in Table 12 and Table 13, for the younger and older participants, respectively. The level of each beta (ß) weight, the change in -2 log likelihood (if the item were removed from the model), and the significance of the potential change are listed in each table. As noted previously, the SSI items were entered in separate blocks, for a total of seven analyses in each age group. The items presented are only those with p-values under.05, but the critical items (in bold) were chosen using the stricter p <.01 criteria. While less strict criteria are recommended for exploratory analyses (Menard, 2002), in this case, the p value was set at.01, given the high number of comparisons. Moreover, a subsequent analysis would serve to test further the discriminative ability of each SSI item in a different way, to expand the view of the diagnostic utility of items. With regard to the critical items, nine items were identified for the younger participants, and seven items were identified for the older participants (see Tables 12 and 13). Each of the four subscales had critical items represented for both age groups, but only one or two critical items emerged from the Agreeable Nature subscale as important predictors. As noted beneath both tables, item 49 (takes toys from siblings or other children), although it reached statistical significance, it was not considered a critical item 39

51 since it did not discriminate between ASD and non-asd clinical groups in a subsequent analysis. A follow-up set of forward stepwise binary logistic regressions, using the same procedures, resulted in slightly more conservative findings. Specifically, for the younger participants, three of the items (1, 32, and 34) did not meet the same threshold of statistical significance. The results for the older participants were the same as the backward stepwise results. As mentioned previously, backward models can uncover relationships between predictors and dependent variables that forward models can miss, as was the case for the younger participants in this study. Assessment of item discrimination and redundancy. At this point, further examination of the individual SSI items was necessary in order to refine the instrument. One-tailed independent samples t-tests within each age group for all 54 items were done to assess mean differences between ASD and non-asd clinical groups. Table 14 shows the results of these analyses. The means, standard deviations, and test values are displayed for each item comparison. Some explanation of the inclusion and exclusion criteria for SSI items is necessary for the reader to properly interpret Table 14. Two initial criteria were used to decide which items to retain for shortened versions of the SSI for each age group: (1) the t-value for the item mean comparison was significant at p <.01 (and the mean difference was in the anticipated direction), and/or (2) the item was identified as a critical item in the previous analyses, and had a mean difference of p <.05. Several items across both age groups were eliminated based on the first criterion. Only item 49 (takes toys from siblings or other children) was eliminated based on criterion 2. 40

52 These two selection criteria resulted in retaining 32 SSI items for the younger participants and 27 SSI items for the older participants. However, since brevity is central to the utility of screening instruments, a third (exclusionary) criterion was used. Item pairs with Pearson inter-item correlations at or above r =.70 were examined, and one of the items in each pair was considered for removal. For the younger individuals, this high intercorrelation was found for eight item pairs. For the older individuals, it was found for nine item pairs. The average of these high intercorrelations was r =.79 (range ). Clinical judgment was used to determine which items to remove. For instance, one pair related to joint attention (items 5 and 25) so they were both retained. In most cases, the item with the lesser discriminative power was dropped (e.g., it had a lower t-statistic). In a few cases, the two items performed similarly and had similar apparent clinical value (see items 34 and 35 in the younger participants for an example), so the decision was more arbitrary. Overall, six items were eliminated for the younger individuals, and six items were eliminated for the older individuals. These items are denoted in Table 14. The retained items are depicted in boldface. These refinements resulted in a 26-item instrument for the younger participants, and a 21-item instrument for the older participants. For an easier view of which items were retained for the younger and the older children, Appendices D and E contain the shortened instruments, labeled SSI-Toddler (SSI-T), and SSI-Preschool (SSI-PS), respectively. Matched participants. A subset of participants from the younger and older ASD and non-asd clinical groups were individually matched on age and IQ level. 41

53 Participants in different groups were matched within 6.5 months of age and 12 full scale or developmental quotient points of each other. Table 15 depicts the resulting matched groups (20 younger participants; 34 older participants) and the characteristics of the samples, along with significance tests of the group differences on each characteristic. Significance tests were conducted via independent samples t-tests or Chi Square tests, as applicable. Across all characteristics, including age, IQ, verbal ability, gender, ethnicity, and SES indices, only one significant difference emerged. Specifically, the SES mean score for older non-asd participants was found to be significantly higher than the mean score for the ASD group (t= -2.89; p <.05). The equivalence of nearly all of the unmatched characteristics was serendipitous, although admittedly some of the statistical equivalence should be attributed to the small sample sizes of the groups. Determination of cutoff scores. Next, a series of Receiver Operating Characteristic (ROC) curve analyses with different comparison groups were conducted to determine cutoff scores for optimal sensitivity and specificity. The results for each of these analyses are presented in Tables 16 through 21. These are ordered from least to most stringent, i.e., from discriminating autism from typical development (least stringent) to discriminating ASD (autism or PDD-NOS) from non-asd clinical referrals (most stringent). In almost all tables, three screen scoring alternatives are presented: (1) the full SSI (54 items) for younger and older children, (2) the shortened SSI for each age group (SSI-Toddler and SSI-Preschool), and (3) the critical items previously identified for the younger and older children. Scoring alternatives (2) and (3) were derived from the discrimination of ASD vs. non-asd clinical groups, but were used to test the other group discriminations (depicted in Tables 16 to 18) as well. All six tables show the SSI 42

54 scores under investigation, the sample size of each subgroup, screen score descriptive statistics, the area under the curve (AUC) estimation, and sensitivity and specificity values for different screen scores. For each comparison, the score judged to have the optimal balance of both sensitivity and specificity was placed in the middle, in boldface. The scores immediately preceding and following are nearby cutoffs which optimized the specificity and sensitivity, respectively. Autism vs. Typically-developing. Table 16 shows the discrimination of participants with autism versus those who are typically-developing, which was the least stringent comparison. For the younger participants months of age, the shortened SSI scale (26-item SSI-Toddler) produced the highest levels of sensitivity and specificity (.93, and.82, respectively, for a score of 50). Similarly, for the older participants months of age, the shortened SSI scale (21-item SSI-Preschool) produced the best balance of sensitivity and specificity (.97 and.94, respectively, for a score of 37). The use of the other two scoring methods (the full 54-item SSI score and the score for critical items only) resulted in slightly lower specificity (i.e., correct identification of typicallydeveloping participants). ASD vs. Typically-developing. Table 17 shows the discrimination of participants with ASD (either autism or PDD-NOS) versus typically-developing participants. For the younger participants, again the scoring method that produced the best outcome was the 26-item SSI-Toddler (sensitivity of.94 and specificity of.91, for a score of 50). For the older participants, however, the use of the critical item total score narrowly produced the best levels of discrimination (sensitivity of.89 and specificity of.83, for a score of 16). 43

55 Autism vs. non-asd clinical. Table 18 depicts a more stringent diagnostic discrimination, namely participants with autism versus non-asd clinical participants. Again, the best choice of method to discriminate the younger children was the use of the SSI-Toddler (sensitivity of.87, and specificity of.71, for a score of 45). For the older children, the critical items total score again narrowly produced the best group discrimination (sensitivity of.97, and specificity of.72, for a score of 13). ASD vs. non-asd clinical. Tables 19 and 20 depict the ROC discriminations of greatest clinical interest. Both represent the scores of participants with ASD (both autism and PDD-NOS) versus non-asd clinical participants. Table 19 shows results for all of these individuals; Table 20 shows results for the subset of matched participants. In Table 19, it was found that the SSI-Toddler score produced the best discrimination for the younger participants (sensitivity of.87 and specificity of.71, for a score of 45). There was only a slight advantage, however, as the optimal sensitivity and specificity for the next best method, critical items score, were.85 and.71, respectively. For the older participants, as before, the score on the critical items provided the best discrimination (sensitivity of.82 and specificity of.72, for a score of 13). As with the younger children, the critical items method performed only slightly better than the next best method, SSI- Preschool score, which produced a sensitivity and specificity of.81, and.70, respectively. Positive predictive value (PPV) and negative predictive value (NPV) for the younger and older participants from Table 19 were calculated next, using the optimal cutoff scores. The PPV represents the proportion of children screened positive for being at risk for an ASD who actually have an ASD, and NPV represents the proportion of children screened negative for being at risk for an ASD who actually do not have an 44

56 ASD. For the 26-item SSI-Toddler (using a score cutoff of < 45), PPV was.87, and NPV was.70. For the 9 critical items for younger participants (using a score cutoff of < 17), PPV was.86, and NPV was.76. For the 21-item SSI-Preschool (using a score cutoff of < 37), PPV was.78, and NPV was.76. For the 7 critical items for older participants (using a score cutoff of < 13), PPV was.76, and NPV was.77. These values were very close, and in some cases, higher than the corresponding sensitivity and specificity levels. This is not surprising for a Level 2 screening situation, where the number of cases (i.e., base rate within the sample) is relatively high in the sample being assessed (Coonrod & Stone, 2005). The ROC results for the matched subsamples are shown in Table 20. The best method for the younger children was the score on critical items (sensitivity of.90 and specificity of.70, for a score of 18), but the SSI-Toddler score performed nearly as well (sensitivity of.80 and specificity of.70, for a score of 48). For the older children, the score on critical items (sensitivity of.88 and specificity of.65, for a score of 15) somewhat outperformed the SSI-Preschool scoring method (sensitivity of.82 and specificity of.71, for a score of 42). These indices are encouraging, as they are comparable to the results for the larger unmatched sample in Table 19. However, some small differences did emerge between the results in Table 19 and 20 with regard to the magnitude of the screen scores. The unmatched groups optimal cutoffs were between one and five points lower than the matched groups optimal cutoffs (for the critical item scoring method, and the shortened scale scoring method, respectively). This outcome may have been due to the increased stringency of the matched group comparisons, and/or it may relate to the smaller sample size of the matched groups. 45

57 One final set of ROC discriminations, found in Table 21, shows the breakdown of results by verbal ability and by age group. The SSI full scale score was not included in this set of analyses in order to keep the number of scoring comparisons from being unwieldy. Verbal ability was defined in the same terms as done previously in this study, namely the ADI/ADI-R classification of presence or absence of functional phrase speech. It follows that the participants were limited to those who had full ADI/ADI-R assessment. Upon inspection of the results, the nonverbal participants, both younger and older, had higher levels of specificity than the verbal participants in almost all cases. The average difference on the AUC between verbal and nonverbal participants was approximately.08. However, the optimal cutoffs were within the range of typical differences found in the discriminations throughout the rest of the ROC tables. A series of independent samples t-tests were done for each mean comparison of SSI-Toddler, SSI-Preschool, and critical item scores depicted in Tables Across all comparisons, the mean score was found to be significantly lower for ASD participants than for non-asd clinical participants at p <.001 or better. The only exception to these significance levels occurred with the younger matched participants in Table 20 (n= 10 ASD; n=10 non-asd clinical), where the critical items mean score t-value was 2.98 (p <.01), and the SSI-T mean score t-value was 2.81 (p <.05). Evaluation of False Negatives. Next, the false negatives (diagnosed ASD but screened non-asd by SSI) were examined for the individuals represented in Table 19. (False negatives were examined for the shortened scale and for the critical items; the SSI full scale was not considered here.) For the younger children, the optimal screen scores for the SSI-T and the critical items together failed to identify 7 of the 52 individuals with 46

58 ASD. These false negatives had lower ADI/ADI-R total scores (mean = 27.6; SD = 6.3) compared to the true positives (mean = 33.0; SD = 12.0). Each of the ADI/ADI-R subscale means was lower for the false negatives, with the exception of the Restrictive/Repetitive subscale. Likewise, the false negatives had somewhat lower IQ scores (mean = 69.4; SD = 12.7) compared to the true positives (mean = 77.1; SD = 25.2). Significance testing was not done due to the relatively small number of false negatives. No notable differences were observed for age, ethnicity, presence of verbal ability, type of ASD diagnosis, or proportion of referral sources. For the older sample, the optimal screen scores for the SSI-Preschool and for the critical items together failed to identify 11 of the 62 individuals with ASD. These false negatives had lower ADI/ADI-R total scores (mean = 23.8; SD = 5.9) versus the true positives (mean = 34.3; SD = 8.6). Each of the ADI/ADI-R subscale means was lower for the false positives, with the exception of the Restrictive/Repetitive subscale. Type of ASD diagnosis and referral source also differed. Nearly all of the false negatives (10 out of 11 children) had a diagnosis of PDD-NOS, versus 16 out of 51 for the true positives. (In other words, 67% of true positives had a diagnosis of autism, compared to 9% of false negatives.) Over half of the false negatives (6 of the 11 children) were referred from the Fragile X clinic, compared to 14% of the true positives (7 of 51 children). No notable differences were found for age, ethnicity, IQ level, or presence of verbal ability. Scoring Recommendations The SSI Total Score was never the best scoring method for discriminating diagnostic groups. Thus it was not considered with regard to cutoff scores. In determining recommended cutoff scores, the discrimination of interest was between the 47

59 individuals with ASD and the non-asd clinical referrals, reflected in Table 19 (the full unmatched sample), and Table 20 (the matched subsample). As previously noted, the optimal cutoff scores for each method did not differ markedly between the two sets of results. It was decided that the mean of the unmatched and the matched optimal cutoff score for each scoring method would be used as the recommended clinical cutoff. Using the mean of the optimal cutoff scores recognizes the greater experimental control of the matched sample discrimination, together with the potentially greater statistical power of the larger unmatched sample. Moreover, using the mean of the comparable optimal score cutoffs from Tables 19 and 20 generally represented little to no change in the levels of sensitivity and specificity for each scoring method. For example, the optimal SSI-Toddler cutoff in Table 19 was 45, and in Table 20 it was 48. The mean of these scores is 47 (rounded to a whole integer). An SSI-Toddler cutoff of 47 for the participants in Table 19 would reflect a sensitivity of.90 and a specificity of.67. For the participants in Table 20, an SSI-Toddler cutoff of 47 would reflect a sensitivity of.75 and a specificity of.70. In practical terms, since the cutoff scores for matched participants were slightly higher than for unmatched participants, using the mean of each pair of values would potentially increase the sensitivity levels for the individuals in Table 19 at the expense of specificity, and would potentially increase specificity levels for the individuals in Table 20 at the expense of sensitivity. Thus, for younger children, recommended clinical cutoffs are as follows: a score of < 47 on SSI-T items, or a score of < 18 on the 9 Critical items. For older children, the recommended clinical cutoffs are as follows: a score of < 40 on SSI-Preschool items, or a score of < 14 on the 7 Critical items. Figures 4 through 7 depict the histograms of each 48

60 score distribution, with vertical reference lines representing the recommended cutoff scores. Participants on and to the left of the vertical reference line in each histogram represent those who would be positively identified as having clinically-significant delays in basic social interaction by their screening score. The ASD group has been divided into autism and PDD-NOS diagnoses so that their relative presence in the scoring continuum can be seen. Participants with autism are depicted in black, participants with PDD-NOS are depicted in gray, and non-asd clinical participants are depicted in white. 49

61 Chapter 4: Discussion The SSI was developed to screen for delays in the basic capacity for social interaction. Items are scored in a positive direction, i.e., higher SSI scores reflect greater presence of social behaviors and thus more typical development. This study found that the SSI instrument performed very well in discriminating children diagnosed with autism or PDD-NOS from those with other developmental and/or psychiatric concerns who were also assessed for ASD. Diagnostic validity was found to be equally strong for both the unmatched and matched samples of toddler-age (24-42 months old) and preschool-age (43-61 months old) participants. The SSI performance also did not appear to be much affected by presence or absence of verbal ability. The instrument was refined and two brief versions of the SSI were developed, the SSI-T (Toddler) and the SSI-PS (Preschool). The SSI-T and the SSI-PS differ from other screening instruments (such as the MCHAT, the SCQ, or the STAT) in a number of important ways. First, the SSI-T and SSI-PS place the focus on the basic capacity for social interaction. Special emphasis is placed on presence of joint attention and other social initiation and response behaviors which have been found to discriminate very young children with ASD from those with other developmental delays much more reliably than language delay, sensory processing issues, or repetitive behaviors (Ventola et al., 2007). Second, the SSI-T and SSI-PS are Level 2 screens that can be filled out independently by caregivers, such that expert 50

62 administration is not required. Third, most of the items tap into typical behaviors with a positive valence, which may be easier for caregivers to recognize and report upon. Fourth, the four-point scoring method allows for detection of subtle differences that dichotomously-scored instruments like the MCHAT or the SCQ could potentially miss. Finally, the two versions for younger and older participants (the SSI-T and the SSI-PS) allow for capture of age-specific social communication behaviors. This is especially relevant given that symptom profiles and severity seem to change significantly for a portion of children with ASD between the ages of 24 and about months of age (as either improvement or worsening of symptoms) (e.g., Cox et al., 1999; Chawarska, Klin, Paul, Macari, & Volkmar, 2009). The Study Samples Evaluation of participant characteristics within the four main groups (autism, PDD-NOS, non-asd clinical, and typically-developing) revealed that the typicallydeveloping participants were younger and had higher SES than the other groups. With regard to the three clinical groups, differences were found that one would expect for adaptive behavior, verbal ability, and IQ (e.g., lower VABS-S and fewer verbal participants in ASD groups compared to non-asd clinicals; lower VABS composite and IQ level in the autism group). These findings lend more credibility to the samples being representative of each diagnostic group and also highlight the importance of creating matched groups for subject characteristics when possible. This study incorporated a heterogeneous clinic sample of participants, such that participants were recruited from several types of clinics, with a variety of suspected delays or abnormalities. This richness of the sample avoids some of the potential bias 51

63 associated with single-clinic referrals, and it likely increases generalizability to many types of clinical populations. Commonly-identified conditions among both the ASD and the non-asd clinical groups were mood or disruptive disorders, hyperactivity, various language or communication disorders, cognitive or motor delays, and auditory problems. Finally, while it is not always important to establish a perfect Gaussian distribution of scores on a clinical instrument, the question of multivariate normality is an important issue for an instrument that measures behaviors that are purportedly normallydistributed. All subsets of participants (typically-developing, combined clinical, and ASD only) displayed normally-distributed SSI scores. This was especially important for the scores of the combined clinical groups (i.e., all non-typically developing participants) that were used in the factor analyses. Construct Validity The exploratory factor analysis (EFA) contributed to establishing the construct validity of the scale. The derived subscales had high internal consistency, and each was conceptually diverse from the others. The first and largest subscale, the 21-item Connection to Caregiver, consisted of items relating to shared attention, reading and expressing basic emotions, display of affection, and interpersonal use of gestures. The second subscale, the 13-item Interaction/Imagination subscale, had the highest level of internal consistency, and was made up of items relating to interactive and pretend play, basic communication, and seeking attention. This subscale made remarkable contributions to diagnostic validity, as all of its items significantly discriminated diagnostic groups for both age bands. The third subscale, the 7-item Social Approach/Interest, included items relating to attempts to play and interact with other 52

64 children. The fourth and final subscale, the 7-item Agreeable Nature, consisted of items relating to how the child gets along with and is perceived by others. In this study, the ratio of study participants to the number of SSI items was approximately 4:1. This falls short of a commonly recommended ratio of approximately 5:1 in order to obtain adequately-stable factor solutions (MacCallum, Widaman, Zhang, & Hong, 1999). However, as these authors discuss, this also depends in part upon the average communality levels (i.e., the squared multiple correlations for each variable) and whether there is a high level of overdetermination (i.e., high magnitude on at least three loadings per factor, along with a simple structure). The average of the communalities for the four-factor solution for the SSI was.45 (which is not considered high), but the factors do appear to be highly overdetermined. The RMSEA estimate for the four-factor solution was just above the cutoff considered an acceptable fit by Browne and Cudeck (1993). Together, these lines of evidence indicate that an adequate factor solution with moderately-high loadings was obtained with the current sample size. Future studies will help determine the stability of the structure. The convergent validity of the SSI with the ADI-R and its subscales was moderately high, as would be expected. The correlation with the ADI-R Reciprocal Social Interaction (RSI) subscale and the Total Score were the highest (r = -.71); the weakest relationship was with the ADI-R Restricted/Repetitive (RR) subscale (r = -.36). The moderately-high correlation between the SSI and the ADI-R RSI indicate that the two measurements share variance, but that there is also unique variance, which indicates the relevance of the SSI items as targeted for younger individuals. The relatively low correlation with the ADI-R RR subscale corroborates with the general findings that 53

65 restrictive and repetitive behaviors are either not as manifest or not as predictive of the presence of ASD in very young children compared to older children (e.g., Wiggins & Robins, 2008). The relationship of the SSI to the CARS score was notably lower than for the ADI-R Total Score, which is likely explained by much greater focus on communication, sensory issues, and repetitive behaviors in the CARS. Thus the SSI Total Score correlated moderately with diagnostic measures. However, it was not found to correlate significantly with adaptive behavior, IQ, or age. This implies that the behaviors measured by the SSI may not be tapping into development or delay as much as autistic symptomatology. However, some of the very low correlations on these indices may be due to restriction of range in the data to be able to detect relationships. Specifically, cognitive and adaptive functioning were ordinally scaled, and individuals in the severe/profound or high average range of functioning were not well represented. Diagnostic Validity The SSI 54-item Total Score discriminated the diagnostic groups very well. In fact, there were significant mean differences among the scores for those with autism, PDD-NOS, non-asd clinical, and typically-developing participants. The subscale means varied in their ability to discriminate these groups, with the Interaction/Imagination and Social Approach/Interest subscales as the best performers, followed closely by Connection to Caregiver. The Agreeable Nature subscale was not good at discriminating diagnostic groups. SSI scores were also evaluated with regard to two potentially-relevant subject characteristics: gender and verbal ability. No significant differences in SSI scores were found by gender. However verbal children were found to have significantly 54

66 higher Interaction/Imagination subscale scores compared to nonverbal children. It is notable that only one of the four subscales showed this effect, indicating relatively small impact of verbal ability on the screening outcome. However, it does highlight the importance of considering how presence of speech affects symptom presentation. With regard to cross-sectional change in item means (i.e., relative frequency of behaviors by age), some noteworthy trends emerged. First of all, only three items in the autism group had significant mean changes (all increases) between young and older participants. A significant decrease in item endorsement for older versus younger children, though cross-sectional, suggests surpassing of milestones. Even though this did not occur with any items in the autism group, this occurred with increasing frequency from the PDD-NOS group, to the non-asd clinical group, to the typically-developing group. This is suggestive of the aforementioned findings of changes in symptom manifestation (either through improvement or worsening of symptoms) for some individuals between approximately 24 and about 44 months of age (e.g., Lord, 1995; Cox et al., 1999). However, further conjecture about the autism or PDD-NOS diagnostic group changes by age group is not appropriate with the current cross-sectional data. Turning to identification of critical items, some recommend a ratio of the number of positive cases to the number of independent variables to be 10 or higher in order to obtain robust logistic regression results (Peduzzi et al., 1996). In order to approximate this ratio more closely, the SSI items were submitted to the regression procedure in small groups, with the resulting ratio of cases to independent variables ranging between approximately 6 and 10. In addition, a stepwise procedure was chosen, as recommended by Menard (2002) in these cases. With regard to identified items, all items discriminated 55

67 diagnostic groups in subsequent t-tests, except for one item, #49. It is unclear why this item was identified by the binary logistic regression procedures for both younger and older participants. On the other hand, item 49, takes toys from siblings or other children, is reverse scored. As such, it can be interpreted clinically as doesn t take toys from other children, which is similar to a retained critical item, 54, shares favorite items or toys with other children. One is cautioned when using exploratory stepwise procedures to interpret the findings with regard to clinical importance, as these types of anomalies can occur (Hosmer & Lemeshow, 2000). Such paradoxical associations (finding significance in the wrong direction) are a risk with relatively low variable-to-item ratio (Peduzzi et al., 1996). In light of the number of independent variables submitted to logistic regression (i.e., 54 SSI items), it may seem reasonable to have decreased the number of items submitted by leaving out items that were not included in the factor structure, and/or items that had significantly low item-total correlations. However, out of the six items unassigned to factors, two were found to be critical items (has a blank face [32] and looks distant or removed [33]), and three items (interacts with sounds or talking at meals [20], 32, and 33) significantly discriminated ASD from non-asd clinical participants in subsequent item mean t-tests for one or both age groups. Moreover, the problematic critical item (#49) had both a significant factor loading and item-total correlation. The nine critical items for the younger participants reflected the expression and detection of emotion, praise-seeking, and some play-related behaviors. The seven critical items for the older participants reflected display of greeting, reading emotions, pretend play, praise-seeking, and some play-related behaviors. Three common critical items 56

68 emerged for both the younger and older participants: seeks praise by showing what s/he did (#25), joins play when invited by other children (#40), and shares favorite items or toys with other children (#54). Item 25 reflects joint attention and the other two items relate to development of peer relationships, which are both specifically noted as clinically-significant deficits in the DSM-IV-TR. Lack of joint attention, as discussed previously, is increasingly recognized as an early hallmark of ASDs (Mundy & Burnette, 2005; Coonrod & Stone, 2005). With regard to representation across the scale, critical items for younger and older children emerged from all four subscales. This was undoubtedly due in large part to the fact that items were entered into the binary logistic regressions by subscales or in smaller sets of subscale items. However, the performance varied with regard to proportion of significant diagnostic discriminators of ASD from non-asd clinical participants in the t-tests of individual items. Remarkably, every item from the Interaction/Imagination subscale significantly discriminated the ASD from non-asd participants. A high proportion of the Social Approach/Interest items also were significant diagnostic discriminators. The Connection to Caregiver subscale had fewer good discriminators, and the Agreeable Nature subscale had almost none. Agreeable Nature items tap into how the child gets along with and is perceived by others, including presence of tantrums or other irritable behaviors. One-way independent samples t-tests were conducted on each item within each age band. Specific selection criteria were used to determine which items to retain for the shortened SSI instruments for each age group. These criteria resulted in the 26-item SSI- T (Toddler-age version) and the 21-item SSI-PS (Preschool-age version). 57

69 With regard to the ROC curve analyses, the SSI-T, SSI-PS, and the mean of the critical items held up to stringent tests of diagnostic validity, especially with regard to assessment of screening accuracy. As discriminations increased in stringency (from autism vs. typical to ASD vs. non-asd clinical), sensitivity and specificity held up nicely. The levels of accuracy for younger and older discriminations were similar to one another, as the average AUC was not systematically higher or lower for one age group over the other. The levels of accuracy for unmatched and matched samples were likewise high and remarkably similar to one another. Some small differences were found for verbal ability, such that optimal cutoff scores for older nonverbal participants were somewhat higher than for older verbal participants. Along with this, the nonverbal participants had slightly higher sensitivity levels for both age groups and for all scoring methods. Evaluation of false negatives (i.e., individuals diagnosed ASD but screened non- ASD by the SSI) revealed that both younger and older cases had lower autism symptomatology on the ADI-R and its subscales (with the exception of restrictive/repetitive behaviors). The younger false negatives also had lower IQ than true positives. The older false negatives were almost exclusively diagnosed with PDD-NOS, and they were disproportionately Fragile X clinic referrals. Most of these patterns are not unexpected. However, individuals with Fragile X and ASD may display different patterns or severity of social interaction deficits (Hagerman, 2006), which should possibly be considered when these children are included in ASD diagnostic groups in studies of instrument development or symptom presentation. 58

70 The levels of sensitivity and specificity found across nearly all comparisons in this study met or surpassed many of the levels that have been found for other screening instruments. For instance, the SSI performed much better with respect to specificity levels than another emerging parent-completed screening instrument, the Developmental Behaviour Checklist: Early Screen (DBC-ES), which recently differentiated ASD from non-asd clinical participants with sensitivity of.83 and specificity of.48 (Gray, Tongue, Sweeney, & Einfeld, 2008). The SSI performed much better than either the SCQ (sensitivity.79; specificity.38) or the MCHAT (sensitivity.85; specificity.40) obtained in a recent clinic study of young children (Snow & Lecavalier, 2008). One evaluation of the SCQ within a small sample of children 17 to 45 months of age found optimal sensitivity of.89 and.89 (Wiggins et al., 2007). The participants had all previously received clinical diagnosis, and were returning to complete the SCQ (after presumably becoming somewhat familiar with the ASD diagnostic criteria). The SSI did not reach the same level of sensitivity and specificity that has been found for the STAT screening instrument for two-year-olds (sensitivity.92, specificity.85) (Stone et al., 2004) or for the ADOS Toddler Module (sensitivity and specificity for two age bands approximately.90 and.91, respectively) (Luyster et al., 2009). One would expect greater accuracy for these types of instruments, which utilize a clinicianadministered interactive evaluation, and in the case of the STAT, a discrimination of autism versus non-asd diagnosis. The optimal levels of sensitivity and specificity found in the current study were also slightly lower than those found in the Ghuman, Freund et al. (1998) pilot study of the SSI. However, the diagnostic group discrimination in the 59

71 current study was more stringent (ASD vs. non-asd clinical referrals) than in the 1998 study (which compared a heterogeneous clinical sample to typically-developing controls). Strengths The methodological strengths of the study included a heterogeneous clinical sample, expert clinical diagnosis, and stringent diagnostic discriminations (i.e., matched samples on age and level of cognitive functioning). The use of heterogeneous referrals from different clinic settings allows for potentially broader generalization of the findings. Every determination for presence or absence of an ASD was done via expert clinical diagnosis. Comparing ASD (both autism and PDD-NOS) with non-asd clinical referrals represented a very stringent test of diagnostic validity. This is especially true because the individuals in the clinical control group were assessed for presence of autism due to actual concern for the presence of the disorder. It can be assumed that these individuals, like the ASD group, had some types of developmental concerns akin to the social, communication, and/or repetitive/ritualistic behaviors shared by individuals with ASD. Finally, the matched subjects assessed as a part of the ROC analyses provided some control for the impact of participant age and developmental level. Those participants also happened to be well-matched on other skills and demographic characteristics. Limitations One limitation of the study related to the sample size available for the exploratory factor analyses. As mentioned previously, a ratio of participants to items less than 5 to 1 may have led to a somewhat unstable factor structure. Second, it would have been ideal if matched samples could have been used to identify critical items, but the regression 60

72 procedures called for a greater number of participants than the matched groups could provide. Third, there were few participants with severe or profound intellectual disability in the study sample, such that the findings may not generalize to these groups. Conclusion The SSI successfully discriminated very young children with autism or PDD-NOS from those with other developmental, psychiatric, or behavior disorders who were also assessed for ASD. The four subscales derived through exploratory factor analysis represented clinically-meaningful and diverse constructs. The items in the Interaction/Imagination subscale in particular were remarkable discriminators of ASD and non-asd groups. Scoring recommendations were provided based on stringent diagnostic comparisons, using the relative benefits of both the small matched and the larger unmatched groups. The shortened scales (SSI-T and SSI-PS) and the critical items for each age group seemed to discriminate groups with about the same level of accuracy. However, at this time, the use of the SSI-T and SSI-PS instruments are recommended over the use of the critical items only, until the identification of the critical items can be replicated with a different sample of children. There are several avenues for future research on the SSI. Replication of the factor structure is needed, with a larger sample of participants, and/or with a smaller set of SSI items. As mentioned, identification of critical items also needs to be replicated, perhaps controlling for language level and/or verbal ability. More investigation of the effect of presence of verbal ability on SSI scores is needed, even though the results of this study seemed to indicate that the instrument performed as well, if not better, for nonverbal children. The utility of the instrument for individuals with severe or profound delays 61

73 should be evaluated. The SSI may be useful in gauging improvement over time with individuals receiving targeted therapies to improve social interaction and social communication behaviors (Ghuman, Freund, et al., 1998). The SSI can add to the body of research on the autism phenotype. It provides a picture of the relative presence of many social interaction behaviors that appear to be continuously distributed within different samples of individuals with and without social or general developmental delay. 62

74 References Allison, C., Baron-Cohen, S., Wheelwright, S., Charman, T., Richler, J., Pasco, G., & Brayne, C. (2008). The Q-CHAT (Quantitative Checklist for Autism in Toddlers): A normally-distributed quantitative measure of autistic traits at months of age: Preliminary report. Journal of Autism and Developmental Disorders, 38, American Academy of Pediatrics, Council on Children with Disabilities, Section on Developmental Behavioral Pediatrics, Bright Futures Steering Committee, Medical Home Initiatives for Children with Special Needs Project Advisory Committee (2006). Identifying infants and young children with developmental disorders in the medical home: An algorithm for developmental surveillance and screening. Pediatrics, 118, American Psychiatric Association (1994). Diagnostic and statistical manual of mental disorders (4 th ed.), Washington, DC: Author. American Psychiatric Association (2000). Diagnostic and statistical manual of mental disorders (4 th ed., Text Revision), Washington, DC: Author. Bayley, N. (1993). The Bayley scales of infant development (2 nd ed.), San Antonio, TX: Harcourt Brace. Baron-Cohen, S., Allen, J., & Gillberg, C. (1992). Can autism be detected at 18 months? The needle, the haystack, and the CHAT. British Journal of Psychiatry, 161, Berument, S. B., Rutter, M., Lord, C., Pickles, A., & Bailey, A. (1999). Autism Screening Questionnaire: Diagnostic validity. British Journal of Psychiatry, 175, Browne, M. W., & Cudeck, R. (1993). Alternative ways of assessing model fit. In K. A. Bollen & J. Scott Lang (Eds.), Testing structural models (pp ). Newbury Park, CA: Sage. Browne, M. W., Cudeck, R., Tateneni, K., & Mels, G. (2004). CEFA: Comprehensive Exploratory Factor Analysis, Version 2.00 [Computer software and manual]. Retrieved from 63

75 Carcani-Rathwell, I., Rabe-Hasketh, S., & Santosh, P. (2006). Repetitive and stereotyped behaviours in pervasive developmental disorders. Journal of Child Psychology and Psychiatry, 47, Chahrour, M., & Zoghbi, H. Y. (2007). The story of Rett Syndrome: From clinic to neurobiology. Neuron, 56, Chawarska, K., Klin, A., Paul, K., Macari, S., & Volkmar, F. (2009). A prospective study of toddlers with ASD: Short-term diagnostic and cognitive outcomes. Journal of Child Psychology and Psychiatry, 50, Chawarska, K., Klin, A., Paul, K., & Volkmar, F. (2007). Autism spectrum disorder in the second year: Stability and change in syndrome expression. Journal of Child Psychology and Psychiatry, 48, Chawarska, K., & Volkmar, F. R. (2005). Autism in infancy and early childhood. In F. R. Volkmar (Ed.), Handbook of Autism and Pervasive Developmental Disorders, Volume 1: Diagnosis, Development, Neurobiology, and Behavior (3 rd Edition) (pp ). Hoboken, NJ: John Wiley & Sons, Inc. Cicchetti, D. V. (1994). Guidelines, criteria, and rules of thumb for evaluating normed and standardized assessment instruments in psychology. Psychological Assessment, 6, Constantino, J. N. (2002). The Social Responsiveness Scale. Los Angeles: Western Psychological Services. Constantino, J. N., Davis, S. A., Todd, R. D., Schindler, M. K., Gross, M. M., Brophy, S. L., Metzger, Shoushtari, C. S., Splinter, R., & Reich, W. (2003). Validation of a brief quantitative measure of autistic traits: Comparison of the Social Responsiveness Scale with the Autism Diagnostic Interview-Revised. Journal of Autism and Developmental Disorders, 33, Constantino, J. N., Przybeck, T., Friesen, D., & Todd, R. D. (2000). Reciprocal social behavior in children with and without pervasive developmental disorders. Developmental and Behavioral Pediatrics, 21, Coonrod, E. E., & Stone, W. L. (2005). Screening for autism in young children. In F. R. Volkmar, R. Paul, A. Klin, & D. Cohen (Eds.), Handbook of Autism and Pervasive Developmental Disorders, Volume II: Assessment, Interventions, and Policy (3 rd Edition) (pp ). Hoboken, NJ: John Wiley & Sons, Inc. Corsello, C., Hus, V., Pickles, A., Risi, S., Cook, E. H. Jr., Leventhal, B. L., & Lord, C. (2007). Between a ROC and a hard place: decision making and making decisions about using the SCQ. Journal of Child Psychology and Psychiatry, 48,

76 Cox, A., Klein, K., Charman, T., Baird, G., Baron-Cohen, S., Swetteham, J., et al. (1999). Autism spectrum disorders at 20 and 42 months of age: Stability of clinical and ADI-R diagnosis. Journal of Child Psychology and Psychiatry and Allied Disciplines, 40, Cuccaro, M. L., Shao, Y., Grubber, J., Slifer, M., Wolpert, C. M., Donnelly, S. L., et al. (2003). Factor analysis of restricted and repetitive behaviors in autism using the Autism Diagnostic Interview-R. Child Psychiatry and Human Development, 34, de Bildt, A., Sytema, S., & Ketelaars, C. (2004). Interrelationship between Autism Diagnostic Observation Schedule-Generic (ADOS-G), Autism Diagnostic Interview-Revised (ADI-R), and the Diagnostic and Statistical Manual of Mental Disorders (DSM-IV-TR) Classification in Children and Adolescents with Mental Retardation. Journal of Autism and Developmental Disorders, 34, Eaves, L. C., Wingert, H. D., Ho, H. H. (2006). Screening for autism: Agreement with diagnosis. Autism, 10, Esbensen, A., J., Seltzer, M. M., Lam, K. S. L., Bodfish, J. W. (2009). Age-related differences in restricted repetitive behaviors in autism spectrum disorders. Journal of Autism and Developmental Disorders, 39, Frazier, T., Youngstrom, E., Kubu, C., Sinclair, L., & Rezai, A. (2008). Exploratory and confirmatory factor analysis of the Autism Diagnostic Interview Revised. Journal of Autism and Developmental Disorders, 38, Ghuman, J. K., Freund, L., Reiss, A. Serwint, J., & Folstein, S. (1998). Early detection of social interaction problems: Development of a social interaction instrument in young children. Developmental and Behavioral Pediatrics, 19, Ghuman, J. K., Peebles, C. D., & Ghuman, H. S. (1998). Review of social interaction measures in infants and preschool children. Infants and Young Children, 11, Gray, K, Tonge, B., & Sweeney, D. (2008). Using the Autism Diagnostic Interview- Revised and the Autism Diagnostic Observation Schedule with young children with developmental delay: Evaluating diagnostic validity. Journal of Autism and Developmental Disorders, 38, Gray, K., Tonge, B., Sweeney, D., & Einfeld, S. (2008). Screening for autism in young children with developmental delays: An evaluation of the Developmental Behaviour Checklist: Early Screen. Journal of Autism and Developmental Disorders, 38,

77 Hagerman, R. (2006). Lessons from Fragile X regarding neurobiology, autism, and neurodegeneration. Developmental and Behavioral Pediatrics, 27, Harris, S., & Handleman, J. (2000). Age and IQ at intake as predictors of placement for young children with autism: A four- to six-year follow-up. Journal of Autism and Developmental Disorders, 30, Hayes, E. W., & Matalon, R. (2009). Fragile X Syndrome. Pediatrics, 124, Hollingshead, A. B. (1975). Four factor index of social status. Unpublished manuscript, Yale University, New Haven. Hosmer, D., & Lemeshow, S. (2000). Applied Logistic Regression, 2 nd edition. New York: Wiley & Sons. Kanner, L. (1943). Autistic disturbances of affective contact. The Nervous Child, 2, Kleinman, J. M., Robins, D. L., Ventola, P. E., Pandey, J., Boorstein, H. C., Esser, E. L., et al. (2008). The Modified Checklist for Autism in Toddlers: A follow-up study investigating the early detection of autism spectrum disorders. Journal of Autism and Developmental Disorders, 38, Kleinman, J. M., Ventola, P. E., Pandey, J., Verbalis, A. D., Barton, M., Hodgson, S. (2008). Diagnostic stability in very young children with autism spectrum disorders. Journal of Autism and Developmental Disorders, 38, Lecavalier, L., Aman, M.G., Scahill, L., McDougle, C. J., McCracken, J. T., Vitiello, B., et al. (2006). Validity of the autism diagnostic interview-revised. American Journal of Mental Retardation, 111, Lord, C. (1995). Follow-up of two-year-olds referred for possible autism. Journal of Child Psychology and Psychiatry, 36, Lord, C., & Corsello, C. (2005). Diagnostic instruments in autistic spectrum disorders. In F. R. Volkmar, R. Paul, A. Klin, & D. Cohen (Eds.), Handbook of Autism and Pervasive Developmental Disorders, Volume II: Assessment, Interventions, and Policy (3 rd Edition). (pp ). Hoboken, NJ: John Wiley & Sons, Inc. Lord, C., Risi, S., Lambrecht, L., Cook, E. H., Jr., Leventhal, B. L., DiLavore, P. C., Pickles, A., & Rutter, M. (2000). The Autism Diagnostic Observation Schedule- Generic: A standard measure of social and communicative deficits associated with the spectrum of autism. Journal of Autism and Developmental Disorders, 30,

78 Lord, C., Rutter, M., Goode, S., Heemsbergen, J., Jordan, H., Mawhood, L., & Schopler, E. (1989). Autism Diagnostic Observation Schedule: A standardized observation of communicative and social behavior. Journal of Autism and Developmental Disorders, 19, Luyster, R., Gotham, K., Guthrie, W., Coffing, M., Petrak, R., Pierce, K., et al. (2009). The Autism Diagnostic Observation Schedule Toddler Module: A new module of a standardized diagnostic measure for autism spectrum disorders. Journal of Autism and Developmental Disorders, 39, MacCallum, R., Widaman, K., Zhang, S., & Hong, S. (1999). Sample size in factor analysis. Psychological Methods, 4, Menard, S. (2002). Applied logistic regression analysis, 2 nd Edition. Thousand Oaks, CA: Sage Publications. Series: Quantitative Applications in the Social Sciences, No First ed., Mundy, P., & Burnette, C. (2005). Joint attention and neurodevelopmental models of autism. In F. R. Volkmar (Ed.), Handbook of Autism and Pervasive Developmental Disorders, Volume 1: Diagnosis, Development, Neurobiology, and Behavior (3 rd edition) (pp ). Hoboken, NJ: John Wiley & Sons, Inc. Norris, M., & Lecavalier, L. (2009). Evaluating the use of exploratory factor analysis in developmental disability psychological research. Journal of Autism and Developmental Disorders. Advanced online publication. doi: /s Ozonoff, S., Macari, S., Young, G. S., Goldring, S., Thompson, M., & Rogers, S. J. (2008). Atypical object exploration at 12 months of age is associated with autism in a prospective sample. Autism, 12, Peduzzi, P., Concato, J., Kemper, E., Holford, T., & Feinstein, A. (1996). A simulation study of the number of events per variable in logistic regression analysis. Journal of Clinical Epidemiology, 49, Robins, D. L., & Dumont-Mathieu, T. M. (2006). Early screening for autism spectrum disorders: Update on the modified checklist for autism in toddlers and other measures. Journal of Developmental and Behavioral Pediatrics, 27, Robins, D. L., Fein, D., Barton, M. L., & Green, J. A. (2001). The modified checklist for autism in toddlers: An initial study investigating the early detection of autism and pervasive developmental disorders. Journal of Autism and Developmental Disabilities, 31,

79 Rutter, M., Bailey, A., & Lord, C. (2003). Social Communication Questionnaire. Los Angeles, CA: Western Psychological Services. Rutter, M., LeCouteur, A., & Lord, C. (2003). Autism Diagnostic Interview-Revised. Los Angeles, CA: Western Psychological Services. Schopler, E., Reichler, R., & Renner, B. (1988). The Childhood Autism Rating Scale (CARS). Los Angeles: Western Psychological Services. Sikora, D. M., Pettit-Kekel, K., Penfield, J., Merkens, L. S., & Steiner, R. D. (2006). The near universal presence of autism spectrum disorders in children with Smith- Lemli-Opitz Syndrome. American Journal of Medical Genetics, Part A, 140A, Snow, A., & Lecavalier, L. (2008). The sensitivity and specificity of the M-CHAT and the SCQ in preschoolers suspected of having pervasive developmental disorders. Autism, 12, Snow, A., Lecavalier, L., & Houts, C. (2009). The structure of the Autism Diagnostic Interview-Revised: Diagnostic and phenotypic implications. Journal of Child Psychology & Psychiatry, 50, Sparrow, S., Balla, D., & Cicchetti, D. (1984). Vineland Adaptive Behavior Scales. Circle Pines, MN: American Guidance Service. Stone, W. L., Coonrod, E.E., & Ousley, O. Y. (2000). Screening tool for autism in twoyear-olds (STAT): Development and preliminary data. Journal of Autism and Developmental Disorders, 30, Stone, W. L., Coonrod, E. E., Turner, L. M., & Pozdol, S. L. (2004). Psychometric Properties of the STAT for early autism screening. Journal of Autism and Developmental Disorders, 34, Stone, W. L., Lee, E. B., Ashford, L., Brissie, J., Hepburn, S. L., Coonrod, E. E., & Weiss, B. H. (1999). Can autism be diagnosed accurately in children under three years? Journal of Child Psychology and Psychiatry, 40, 219, 236. Stone, W. L., McMahon, C. R., & Henderson, L. M. (2008). Use of the Screening Tool for Autism in Two-Year-Olds (STAT) for children under 24 months: An exploratory study. Autism, 12, Stone, W. L., & Ousley, O. Y. (1997). STAT Manual: Screening tool for autism in twoyear-olds. Unpublished manuscript, Vanderbilt University. 68

80 Szatmari, P., Mérette, C., Bryson, S. E., Thivierge, J., Roy, M., Cayer, M., & Maziade, M. (2002). Quantifying dimensions in autism: A factor-analytic study. Journal of the American Academy of Child and Adolescent Psychiatry, 41, Szatmari, P., Georgiades, S., Bryson, S., Zwaigenbaum, L., Roberts, W., Mahoney, W., Goldberge, J., & Tuff, L. (2006). Investigating the structure of the restricted, repetitive behaviours and interests domain of autism. Journal of Child Psychology and Psychiatry, 47, Thorndike, R. L., Hagen, E. P., & Sattler, J. M. (1986). The Stanford-Binet Intelligence Scale (4 th ed.), Chicago, IL: Riverside Publishing Company. Ventola, P., Kleinman, J., Pandy, J., Wilson, L., Esser, E., Boorstein, H., et al. (2007). Differentiating between autism spectrum disorders and other developmental disabilities in children who failed a screening instrument for ASD. Journal of Autism and Developmental Disorders, 37, Volkmar, F. R., & Klin, A. (2005). Issues in the classification of autism and related conditions. In F. R. Volkmar (Ed.), Handbook of Autism and Pervasive Developmental Disorders, Volume 1: Diagnosis, Development, Neurobiology, and Behavior (3 rd edition) (pp. 5-41). Hoboken, NJ: John Wiley & Sons, Inc. Wiggins, L. D., Bakeman, R., Adamson, L. B., & Robins, D. L. (2007). The utility of the Social Communication Questionnaire in screening for autism in children referred for early intervention. Focus on Autism and other Developmental Disabilities, 22, Wiggins, L. D., & Robins, D. L. (2008). Brief report: Excluding the ADI-R behavioral domain improves diagnostic agreement in toddlers. Journal of Autism and Developmental Disorders, 38, World Health Organization (1992). The ICD-10 Classification of Mental and Behavioral Disorders: Clinical descriptions and diagnostic guidelines. Geneva, Switzerland: Author. 69

81 Appendix A: SSI Instrument - Full Scale 70

82 GHUMAN-FOLSTEIN SCREEN FOR SOCIAL INTERACTION (SSI)* (INCLUDES SCORING TEMPLATE) Demographic Information: Name Today s Date Child s Birth Date Child s Age Child s Gender M F Your relationship: Child s Ethnic Group: Mother [ ] Father [ ] African American [ ] Other [ ] specify Caucasian [ ] Primary Caregiver s relationship (if Native American [ ] different than parents) Asian or Pacific Islander [ ] Head of Household s relationship (if Other (specify) different than primary caregiver): Highest Grade Completed: Mother Mother s type of work Father Father s type of work Primary Caregiver Primary Caregivers s work Head of Household Head of Household s work Instructions: On the next pages, there is a list of items that describes how children interact in social situations. Please read each question carefully and check the box that best describes your child NOW (in the past three months). ALMOST NEVER: SOME OF THE TIME: MOST OF THE TIME: Check this box if your child RARELY or NEVER does what is described in the question. Check this box if your child SOMETIMES does what is described in the question. Check this box if your child USUALLY does what is described in the question. ALMOST ALL THE TIME: Check this box if your child VERY OFTEN or ALMOST ALWAYS does what is described in the question. PLEASE ANSWER ALL OF THE QUESTIONS AS WELL AS YOU CAN, EVEN IF SOME OF THE QUESTIONS DON T SEEM TO APPLY TO YOUR CHILD. 71

83 SSI Full Scale Screen with Scoring Template Page 2 YOUR OBSERVATION ALMOST NEVER SOME OF THE TIME MOST OF THE TIME ALMOST ALL OF THE TIME When you talk to your child, does s/he: 1. look at you? smile at you? Does your child try to get your attention: 3. to get things for him/her? when s/he can t do something by himself/herself? to show you things? Does your child interact with you back and forth over a few turns using: 6. speech or sounds? gestures? Does your child greet you when you return home by: 8. looking at you? making sounds or talking to you? wanting to be picked up or coming to you? hugging you? Does your child show affection by: 12. cuddling up to you? When you show affection, does your child return your affection? Does your child want you to hug or pick him/her up when s/he gets hurt? Does your child respond playfully by: 15. laughing when you make silly sounds? playing games like peek-a-boo, pattycake, rolling ball, blowing bubbles? 17. playing dressing up games? playing pretend games with dolls, cars, action figures, dollhouse?

84 SSI Full Scale Screen with Scoring Template Page 3 YOUR OBSERVATION 73 ALMOST NEVER SOME OF THE TIME MOST OF THE TIME ALMOST ALL OF THE TIME During meals does your child: 19. look at you? interact with you by making sounds or talking? During meals does your child show what food s/he likes by: 21. making sounds or talking? pointing or other gestures? speech or making sounds? Does your child copy you by washing dishes, pretending to cook or mow the lawn, etc.? 25. Does your child show you things that s/he had done and wants you to praise? 26. Does your child smile when you praise him/her? 27. Does your child have a favorite toy, blanket, stuffed animal or another object that s/he wants to take with him/her everywhere? Specify the kind of toy: 28. Does your child get upset when you leave him/her with an unfamiliar babysitter? 29. Does your child follow you around in the house? 30. Does your child come looking for you if you are not in the same room with him/her? Is your child shy around people s/he does not know well? Does your child: 32. have a blank face? look distant or removed? Can your child tell from the look on your face or the tone of your voice that you are: 34. happy? angry?

85 SSI Full Scale Screen with Scoring Template Page 4 DOES YOUR CHILD SHOW INTEREST IN OTHER CHILDREN BY: 36. watching them? moving towards them? staying close to them? trying to play with them? joining in the play if they invite him/her? showing concern if another child is upset? Is your child able to: 42. start a social exchange with other children? join in when other children start the social exchange? keep a social exchange going back and forth with other children over a few turns? Do other children (of his/her age): 45. like him/her? want to play with him/her or want to be around him/her? Does your child turn off siblings or other children his/her age because he/she: 47. is aggressive with them? avoids other children? takes toys away from them? yells and screams? Is your child able to take turns in play? Does your child have playmates who s/he prefers to play with? Does your child share toys and favorite objects with: 53. you or other adults? siblings or other children his/her age? *Screen developed by Jaswinder Kaur-Ghuman, M.D., and Susan Folstein, M.D. (1992; 1994) 74

86 SSI Full Scale Screen with Scoring Template Page 5 COMMENTS: PLEASE WRITE ANY COMMENTS OR QUESTIONS YOU MAY HAVE REGARDING THIS QUESTIONNAIRE. THANK YOU. 75

87 Appendix B: Tables 76

88 *Recruitment Source: N Mean age in months (SD) % male % minority % with mother as informant SES level** mean (SD) Full sample (10.6) 64% 32% 82% 40.5 (22.8) KKI Infant & Preschool clinic (10.7) 79% 36% 54% 38.9 (21.8) KKI other developmental clinics (10.5) 68% 50% 82% 55.0 (22.6) KKI CARD Autism Clinic Fragile X research clinic Rett s Disorder research clinic SLOS research clinic Pediatric clinics (Typical controls) (9.1) 77% 20% 86% 30.4 (18.0) (8.7) 91% 12% 91% 26.4 (10.8) (10.0) 0% 15% 77% unknown (10.5) 56% 17% 78% unknown (10.9) 51% 40% 86% 45.8 (23.1) *Recruitment source abbreviations: (KKI Kennedy Krieger Institute; CARD Center for Autism and Related Disorders; SLOS Smith-Lemli-Opitz Syndrome) **Hollingshead four-factor index criteria for socioeconomic status (mother/father occupational and educational status). Table 1. Selected Participant Characteristics (Ages months) 77

89 ASD Diagnosis Recruitment Group PDD-NOS Autism No ASD No ASD Assessment Totals CARD clinic IPC clinic Other devel. clinic Fragile X clinic Rett s clinic SLOS clinic Typical controls Totals Table 2. ASD Diagnoses and Recruitment Source for Study Participants 78

90 Score % Item # Item Description Item-Total Mean (SD) SSI_1 looks at you when you talk to her (.67) SSI_2 smiles at you when you talk to her (.78) SSI_3 gets your attn to obtain things (.89) SSI_4 gets your attn to help her do something (.80) SSI_5 gets your attn to show you things (.65) SSI_6 interacts with you via speech or sounds (.81) SSI_7 interacts with you via gestures (1.03) SSI_8 greets your return by looking at you (.72) SSI_9 greets your return with sounds or talking (.60) SSI_10 greets your return by seeking proximity (.72) SSI_11 greets your return by hugging you (.57) SSI_12 shows affection by cuddling (.62) SSI_13 returns your affection (.65) SSI_14 wants hugged or picked up when hurt (.61) SSI_15 laughs when you make silly sounds (.55) SSI_16 plays games like peek-a-boo (.97) SSI_17 plays dress-up games (1.17) SSI_18 plays pretend games (dolls, cars) (.82) SSI_19 looks at you during meals (.84) SSI_20 interacts with sounds or talking at meals (.84) SSI_21 shows food preference by look on her face (.97) SSI_22 shows food preference by pointing/gestures (1.07) SSI_23 shows food preference by speech or sounds (.75) SSI_24 copies you by pretending to do chores (.90) SSI_25 seeks praise by showing what she did (.62) SSI_26 smiles when you praise her (.56) SSI_27 has favorite toy/object she takes everywhere (1.21) SSI_28 upset when left with unfamiliar babysitter -.010^ 1.31 (1.19) SSI_29 follows you around house (.91) SSI_30 looks for you in another room of house (.91) Table 3. Item Score Distribution for All 198 Typically-developing Participants Months of Age (Continued) 79

91 Table 3 Continued SSI_31 shy around people she doesn't know well.098^ 1.61 (.97) SSI_32* has a blank face (.55) SSI_33* looks distant or removed (.60) SSI_34 can see you are happy by your face or voice (.60) SSI_35 can see you are angry by your face or voice (.76) SSI_36 watches other children (.74) SSI_37 moves toward other children (.85) SSI_38 stays close to other children (.91) SSI_39 tries to play with other children (.81) SSI_40 joining play when invited by other children (.76) SSI_41 shows concern if another child is upset (.83) SSI_42 can start social exchange with other children (.88) SSI_43 can join in social exchange with other children (.80) SSI_44 can maintain social exchange with other children (.85) SSI_45 is liked by other children her age (.66) SSI_46 other children her age want to play/be around her (.67) SSI_47* is aggressive with siblings or other children (.92) SSI_48* avoids other children (.70) SSI_49* takes toys from siblings or other children (.87) SSI_50* yells and screams (.84) SSI_51 can take turns in play (.81) SSI_52 has preferred playmates (1.01) SSI_53 shares favorite items or toys with adults (.82) SSI_54 shares favorite items or toys with children (.86) *indicates item is reverse scored ^item-total correlation non-significant (p >.05) (Note: 0 = Almost Never; 1 = Some of the time; 2 = Most of the time; 3 = Almost all the time) 80

92 Score % Item # Item Description Item-Total Mean (SD) SSI_1 looks at you when you talk to her (.92) SSI_2 smiles at you when you talk to her (.92) SSI_3 gets your attn to obtain things (.93) SSI_4 gets your attn to help her do something (.99) SSI_5 gets your attn to show you things (1.18) SSI_6 interacts with you via speech or sounds (1.10) SSI_7 interacts with you via gestures (1.08) SSI_8 greets your return by looking at you (.97) SSI_9 greets your return with sounds or talking (1.17) SSI_10 greets your return by seeking proximity (1.00) SSI_11 greets your return by hugging you (1.12) SSI_12 shows affection by cuddling (.95) SSI_13 returns your affection (.90) SSI_14 wants hugged or picked up when hurt (.90) SSI_15 laughs when you make silly sounds (.99) SSI_16 plays games like peek-a-boo (1.07) SSI_17 plays dress-up games (1.18) SSI_18 plays pretend games (dolls, cars) (1.21) SSI_19 looks at you during meals (.98) SSI_20 interacts with sounds or talking at meals (1.07) SSI_21 shows food preference by look on her face (1.12) SSI_22 shows food preference by pointing/gestures (1.21) SSI_23 shows food preference by speech or sounds (1.17) SSI_24 copies you by pretending to do chores (1.11) SSI_25 seeks praise by showing what she did (1.23) SSI_26 smiles when you praise her (.98) SSI_27 has favorite toy/object she takes everywhere (1.28) SSI_28 upset when left with unfamiliar babysitter.095^ 1.10 (1.21) SSI_29 follows you around house (.99) SSI_30 looks for you in another room of house (1.01) Table 4. Item Score Distribution for 252 Clinical Participants Months of Age (Continued) 81

93 Table 4 Continued SSI_31 shy around people she doesn't know well (1.11) SSI_32* has a blank face (.81) SSI_33* looks distant or removed (.82) SSI_34 can see you are happy by your face or voice (.94) SSI_35 can see you are angry by your face or voice (.98) SSI_36 watches other children (.98) SSI_37 moves toward other children (1.03) SSI_38 stays close to other children (1.02) SSI_39 tries to play with other children (.99) SSI_40 joins play when invited by other children (1.07) SSI_41 shows concern if another child is upset (1.12) SSI_42 can start social exchange with other children (1.03) SSI_43 can join in social exchange with other children (1.03) SSI_44 can maintain social exchange with other children (1.05) SSI_45 is liked by other children her age (.93) SSI_46 other children her age want to play/be around her (.95) SSI_47* is aggressive with siblings or other children (.95) SSI_48* avoids other children (.95) SSI_49* takes toys from siblings or other children (.98) SSI_50* yells and screams (1.05) SSI_51 can take turns in play (.95) SSI_52 has preferred playmates (1.06) SSI_53 shares favorite items or toys with adults (1.04) SSI_54 shares favorite items or toys with children (.97) *indicates item is reverse scored ^item-total correlation non-significant (p >.05) (Note: 0 = Almost Never; 1 = Some of the time; 2 = Most of the time; 3 = Almost all the time) 82

94 Score % Item # Item Description Item-Total Mean (SD) SSI_1 looks at you when you talk to her (.85) SSI_2 smiles at you when you talk to her (.91) SSI_3 gets your attn to obtain things (.92) SSI_4 gets your attn to help her do something (1.01) SSI_5 gets your attn to show you things (1.12) SSI_6 interacts with you via speech or sounds (.95) SSI_7 interacts with you via gestures (.97) SSI_8 greets your return by looking at you (1.02) SSI_9 greets your return with sounds or talking (1.21) SSI_10 greets your return by seeking proximity (1.04) SSI_11 greets your return by hugging you (1.16) SSI_12 shows affection by cuddling (.91) SSI_13 returns your affection (.91) SSI_14 wants hugged or picked up when hurt (.94) SSI_15 laughs when you make silly sounds (1.00) SSI_16 plays games like peek-a-boo (1.01) SSI_17 plays dress-up games (.97) SSI_18 plays pretend games (dolls, cars) (1.12) SSI_19 looks at you during meals (.92) SSI_20 interacts with sounds or talking at meals (.98) SSI_21 shows food preference by look on her face (1.10) SSI_22 shows food preference by pointing/gestures (1.15) SSI_23 shows food preference by speech or sounds (1.13) SSI_24 copies you by pretending to do chores (.99) SSI_25 seeks praise by showing what she did (1.01) SSI_26 smiles when you praise her (1.03) SSI_27 has favorite toy/object she takes everywhere (1.33) SSI_28 upset when left with unfamiliar babysitter.020^.99 (1.19) SSI_29 follows you around house (.96) SSI_30 looks for you in another room of house (.98) Table 5. Item Score Distribution for all 114 Participants with ASD Months of Age (Continued) 83

95 Table 5 Continued SSI_31 shy around people she doesn't know well.140^ 1.33 (1.13) SSI_32* has a blank face (.85) SSI_33* looks distant or removed (.87) SSI_34 can see you are happy by your face or voice (1.00) SSI_35 can see you are angry by your face or voice (1.03) SSI_36 watches other children (.93) SSI_37 moves toward other children (.95) SSI_38 stays close to other children (.93) SSI_39 tries to play with other children (.88) SSI_40 joining play when invited by other children (.87) SSI_41 shows concern if another child is upset (.93) SSI_42 can start social exchange with other children (.75) SSI_43 can join in social exchange with other children (.85) SSI_44 can maintain social exchange with other children (.72) SSI_45 is liked by other children her age (.88) SSI_46 other children her age want to play/be around her (.89) SSI_47* is aggressive with siblings or other children.141^ 2.19 (.99) SSI_48* avoids other children (1.00) SSI_49* takes toys from siblings or other children (1.02) SSI_50* yells and screams (1.08) SSI_51 can take turns in play (.91) SSI_52 has preferred playmates (.96) SSI_53 shares favorite items or toys with adults (.97) SSI_54 shares favorite items or toys with children (.93) *indicates item is reverse scored ^item-total correlation non-significant (p >.05) (Note: 0 = Almost Never; 1 = Some of the time; 2 = Most of the time; 3 = Almost all the time) 84

96 Item Description Factor 1 Factor 2 Factor 3 Factor 4 Connection with Caregiver (Cronbach s α =.923) 1 looks at you when you talk to him/her smiles at you when you talk to him/her gets your attention to obtain things gets your attention to help him/her do something interacts with you via gestures greets your return by looking at you greets your return with sounds or talking greets your return by seeking proximity greets your return by hugging you shows affection by cuddling returns your affection wants hugged or picked up when hurt laughs when you make silly sounds plays games like peek-a-boo looks at you during meals shows food preference by look on his/her face shows food preference by pointing/gestures smiles when you praise him/her has favorite toy/object s/he takes everywhere upset when left with unfamiliar babysitter follows you around house looks for you in another room of house * has a blank face can see you are happy by your face or voice can see you are angry by your face or voice Table 6. Four-factor Solution Using MWL/CF-Varimax Rotation (n=207) (Continued) 85

97 Table 6 Continued Interaction/Imagination (Cronbach s α =.925) 5 gets your attention to show you things interacts with you via speech or sounds plays dress-up games plays pretend games (dolls, cars) shows food preference by speech or sounds copies you by pretending to do chores seeks praise by showing what s/he did shows concern if another child is upset can start social exchange with other children can join in social exchange with other children can maintain social exchange with other children can take turns in play has preferred playmates Social Approach/Interest (Cronbach s α =.811) 20 interacts with sounds or talking at meals shy around people s/he doesn t know well * looks distant or removed watches other children moves toward other children stays close to other children tries to play with other children joins play when invited by other children * avoids other children (Continued) 86

98 Table 6 Continued Agreeable Nature (Cronbach s α =.824) 45 is liked by other children his/her age other children his/her age want to play/be around him/her 47* is aggressive with siblings or other children * takes toys from siblings or other children * yells and screams shares favorite items or toys with adults shares favorite items or toys with children *Indicates item is reverse-scored. Italicized items failed to meet criteria for inclusion. 87

99 Diagnostic Tool Total or Subscale Score: ADI RSI (Social Interaction) subscale ADI Verbal Communication subscale ADI Nonverbal Communication subscale ADI RR (Restricted/Repetitive) subscale ADI Total Score CARS Total Score **p<.01 Two-tailed Pearson s Correlations with SSI Total Score Combined Clinical Participants -.712** (n=131) -564** (n=74) -.537** (n=58) -.359** (n=131) -.671** (n=130) -.430** (n=42) Table 7. Pearson s Coefficient of Correlation between the Screen for Social Interaction (SSI) Total Scores and ASD Assessment Tools (Autism Diagnostic Inventory [ADI or ADI-R] and the Childhood Autism Rating Scale [CARS]) 88

100 Autism group (n=66) (24-61 months old) Two-tailed Correlations^ with SSI Total Score (54 items) PDD-NOS group (n=48) (24-61 months old) Non-ASD clinical group (n=68) (24-61 months old) Cognitive or Adaptive Behavior Total Score, Subscale, or Domain: VABS SOC (Social) domain.178 (n=32).252 (n=25).212 (n=33) VABS COM (Communication) domain.291 (n=20).454 (n=14).314 (n=15) VABS ABC (Composite).064 (n=19).366 (n=12).397 (n=13) Cognitive Level (IQ/DQ).200 (n=20) (n=22).254 (n=37) Age *p<.05 (none reached significance) ^Spearman s rho correlations were used to assess the relationship of ordinal ranges of adaptive and cognitive/developmental levels with screen scores. Pearson s r correlations were used to assess the relationship of age with screen scores. Table 8. Correlations between the Screen for Social Interaction (SSI) Total Score, Adaptive Behavior Scores (Vineland Adaptive Behavior Scales), and Cognitive/Developmental Scores (Intelligence or Developmental Quotient) 89

101 Characteristic: Age in months: mean, (SD) Gender (% male) Ethnicity (% Caucasian [obs.]) Autism group (n=66) (a) 45.4 (10.0) 80% 77% PDD-NOS group (n=48) (b) 44.5 (10.7) 83% 83% Non-ASD clinical group (n=68) (c) 47.4 (9.1) 69% 60% Typicallydeveloping group (n=168) (d) 40.3 (10.9) 52% 63% Significance test/test value : F= 9.20** X 2 = 26.20** X 2 = 10.68* Informant Characteristics: Mother s education: high school or beyond (% [obs.]) SES score: mean, (SD) 95% 33.2 (19.6) 87% 30.8 (19.2) 87% 41.6 (21.0) 82% 44.9 (22.9) X 2 = 6.68 F= 7.55** Cognitive and Adaptive levels: Verbal ability^: % verbal, (ratio) IQ: mean, (SD); n VABS-Soc. Domain: mean, (SD); n VABS-Com. domain: mean, (SD); n VABS ABC: mean, (SD); n 49% (23/47) 61.8 (21.9); (11.6); (13.5); (9.8); 19 43% (16/37) 70.6 (22.3); (17.7); (15.9); (11.2); 12 73% (35/48) 80.9 (17.4); (14.2); (17.5); (18.8); X 2 = 8.97* F= 6.14** F= 12.63** F= 8.08** F= 8.59** VABS-Soc. = Socialization domain of the Vineland Adaptive Behavior Scales (VABS); VABS-Com. = VABS Communication domain; VABS ABC = VABS Adaptive Behavior Composite score. ^Verbal ability as determined by ADI-R functional language definition, when available. *p<.05; **p<.01 ANOVA or Chi Square comparisons by group. Tamhane post hoc comparisons used when unequal variances among samples. ANOVA post-hoc comparisons (significance value of p <.05): Age (Scheffé): a,c>d. SES score (Tamhane): a,b<d. IQ (Scheffé): a<c. VABS-Soc (Scheffé): a,b<c. VABS-Com (Scheffé): a<b,c. VABS ABC (Tamhane): a<c. Table 9. Group Comparisons on Participant Characteristics of Interest 90

102 SSI Domain (# of items) Autism group (a) (n=66); mean (SD) PDD-NOS group (b) (n=48); mean (SD) Non-ASD clinical group (c) (n=68); mean (SD) Typicallydeveloping group (d) (n=168); mean (SD) Test value/ significance SSI Total Score (54) SSI Subscales: Connection with Caregiver (21) Interaction/Imagination (13) Social Approach/Interest (7) Agreeable Nature (7) 71.4 (25.2) 36.3 (13.0) 8.5 (6.7) 7.8 (4.2) 10.9 (4.5) 83.2 (26.4) 39.4 (12.5) 12.8 (9.3) 10.0 (4.5) 11.7 (4.7) (24.7) 45.8 (10.8) 23.7 (8.4) 13.2 (4.4) 12.6 (4.7) (16.2) 50.6 (7.1) 29.2 (6.0) 15.7 (3.4) 15.7 (3.5) F^=114.58* F =39.48* F =164.85* F =74.58* F =28.79* *p<.001 ^One-way ANOVA conducted along with three sets of a-priori contrasts. All contrasts significant: (a < b,c,d), (a,b < c,d), (a,b,c < d); contrasts controlled for unequal variance where appropriate. Multivariate ANOVA by groups and subscales. Post-hoc comparisons for subscale scores (significance level of p<.05 chosen in light of conservative tests used; Tamhane comparison used when unequal variances among samples): Connection with Caregiver - Tamhane comparison results: a,b<c<d. Interaction/Imagination - Tamhane comparison results: a<b<c<d. Social Approach/Interest - Scheffé s comparison results: a<b<c<d. Agreeable Nature - Tamhane comparison results: a,b,c<d. Table 10. Group Comparisons of SSI Total and Subscales 91

103 SSI Items: Item # and brief description Significant difference (p<.05) in t-test between age groups within each sample: Autism? PDD- NOS? Non-ASD clinical? Typicals? Item Mean Scores: Younger (24-42 mo) Older (43-61 mo) 2 smiles at caregiver Y 2.39* gets attention to get things Y ** 4 gets attention for help Y * 6 interacts w/ speech/sounds Y * 12 affection by cuddling Y 2.62* plays peek-a-boo Y 2.58** plays dress-up games Y * 22 points for food preference Y 2.01* unfamiliar babysitter Y 1.44* looks for you in home Y 1.96** shy around unfamiliar Y 1.95* ^ has blank face Y * 33^ looks distant / removed Y * 38 stays close to children Y * 40 joins play when invited Y * 41 concern when kids upset Y ** 41 (same item) Y * 44 maintain social exchange Y.27.81* 45 is liked by other children Y 1.95* (same item) Y 2.29* children want to play Y 1.82* (same item) Y 2.19* takes turns in play Y * 52 has preferred playmates Y * ^items are reverse-scored *p<.05; **p<.01 Age splits by group: (younger; older) Autism (30; 36), PDD-NOS (22; 26), non-asd (21; 47), typically-developing (103; 65). Table 11. SSI Items with Significant Difference between Younger and Older Children Presented by Diagnostic Group 92

104 Change in -2 Log Likelihood b Significance of Change c Predictors a : Item and description ß 1. looks at you when you talk to him/her smiles at you when you talk to him/her interacts with you via gestures (unfactored item) greets your return by looking at you greets your return by hugging you shows affection by cuddling laughs when you make silly sounds has a blank face (unfactored item; rev. scored) can see you are happy by your face or voice interacts with you via speech or sounds seeks praise by showing what s/he did < shows concern if another child is upset can start social exchange with other children shy around people s/he doesn t know well looks distant/removed (unfactored item; rev. scored) < stays close to other children joins play when invited by other children is liked by other children his/her age takes toys from siblings or other children (rev. scored) shares favorite items or toys with children a Items were entered using Backward Stepwise method in seven separate blocks of six to nine items: three blocks from Factor I, two blocks from Factor II, and one block each from Factor III and IV. b Change in model if item removed. c Boldface indicates critical items chosen. (Item 49 is not selected because of non-significant item mean difference between ASD and non-asd clinical.) Table 12. Binary Logistic Regression Analyses for 73 Younger Participants (24-42 months of age) to Predict Membership in ASD (autism or PDD-NOS) versus Non-ASD Clinical Group 93

105 Change in -2 Log Likelihood b Significance of Change c Predictors a : Item and description ß 1. looks at you when you talk to him/her greets your return with sounds or talking < can see you are angry by your face or voice plays pretend games (dolls, cars) seeks praise by showing what s/he did < can start a social exchange with other children < joins play when invited by other children < takes toys from siblings or other children (rev. scored) shares favorite items or toys with children <.001 a Items were entered using Backward Stepwise method in seven separate blocks of six to nine items: three blocks from Factor I, two blocks from Factor II, and one block each from Factor III and IV. b Change in model if item removed. c Boldface indicates critical items chosen. (Item 49 is not selected because of non-significant item mean difference between ASD and non-asd clinical.) Table 13. Binary Logistic Regression Analyses for 109 older participants (43-61 months of age) to Predict Membership in ASD (autism or PDD-NOS) versus Non-ASD Clinical Group 94

106 Connection with Caregiver Younger ASD (n=52) Mean (SD) Younger Non-ASD Clinical (n=21) Test value Mean (SD) t value Older ASD (n=62) Older Non-ASD Clinical (n=47) Item Description 1 looks at you when you talk to him/her a 1.46 (.90) 2.10 (1.04) 2.61** 1.58 (.82) 2.13 (.88) 3.35** 2 smiles at you when you talk to him/her 1.60 (.96) 1.76 (.77) (.86) 1.81 (.92) 1.89* 3 gets your attention to obtain things 1.88 (.92) 2.10 (.70) (.92) 2.13 (.95).17 4 gets your attention to help him/her do something 1.85 (1.04) 2.05 (.81) (.97) 2.28 (.93).54 7 interacts with you via gestures 1.00 (.95) 1.57 (1.12) 2.21* 1.06 (.99) 1.40 (1.06) 1.72* 8 greets your return by looking at you 2.13 (1.05) 2.29 (.96) (1.00) 2.45 (.90) 1.98* 9 greets your return with sounds or talking b 1.62 (1.21) 2.19 (1.03) 1.92* 1.56 (1.22) 2.47 (.91) 4.43** 10 greets your return by seeking proximity 2.04 (1.12) 2.52 (.87) 1.98* 1.94 (.97) 2.40 (.99) 2.47** 11 greets your return by hugging you 1.98 (1.18) 2.48 (.81) 2.05* 1.65 (1.13) 2.28 (.97) 3.06** 12 shows affection by cuddling 2.10 (.96) 2.62 (.59) 2.83** 2.00 (.87) 2.19 (1.04) returns your affection 2.04 (.86) 2.43 (.81) 1.78* 2.05 (.95) 2.30 (.91) wants hugged or picked up when hurt 2.37 (.86) 2.62 (.67) (1.00) 2.26 (.97) laughs when you make silly sounds a 2.04 (.97) 2.76 (.54) 4.05** 1.98 (1.03) 2.43 (.93) 2.31* 16 plays games like peek-a-boo 1.81 (1.05) 2.33 (.91) 2.01* 1.85 (.99) 2.09 (1.20) looks at you during meals 1.35 (1.01) 2.14 (.91) 3.14** 1.35 (.85) 1.77 (1.03) 2.23* 21 shows food preference by look on his/her face 1.48 (1.10) 2.29 (.85) 3.02** 1.56 (1.11) 1.81 (1.19) shows food preference by pointing/gestures 1.27 (1.16) 2.00 (1.18) 2.43** 1.35 (1.15) 1.74 (1.19) 1.73* 26 smiles when you praise him/her 1.71 (1.04) 2.52 (.81) 3.56** 2.06 (.99) 2.49 (.69) 2.64** 27 has favorite toy/object s/he takes everywhere 1.17 (1.31) 1.24 (1.22) (1.33) 1.22 (1.19) upset when left with unfamiliar babysitter 1.18 (1.24) 1.24 (1.18) (1.13) 1.21 (1.25) follows you around house 1.35 (.97) 1.62 (.92) (.96) 1.45 (.95) looks for you in another room of house 1.56 (.96) 1.17 (1.10) (1.00) 1.49 (1.04).19 Mean (SD) Test value Mean (SD) t value Table 14. Mean SSI Scores and One-tailed t-tests between ASD and Non-ASD Clinical Participants (Continued) 95

107 Table 14: Continued 32^ has a blank face a 1.85 (.98) 2.52 (.81) 2.81** 2.34 (.65) 2.47 (.78) can see you are happy by your face or voice a 1.79 (1.04) 2.57 (.68) 3.80** 2.13 (.95) 2.47 (.75) 2.09* 35 can see you are angry by your face or voice b 1.79 (1.05) 2.57 (.68) 3.77** 2.11 (.99) 2.49 (.78) 2.22* Item Interaction/Imagination Description Younger ASD (n=52) Mean (SD) Younger Non-ASD Clinical (n=21) Test value Older ASD (n=62) Older Non-ASD Clinical (n=47) Test value Mean (SD) t value Mean (SD) Mean (SD) t value 5 gets your attention to show you things 1.02 (1.11) 2.14 (.85) 4.16** 1.13 (1.12) 2.40 (.83) 6.83** 6 interacts with you via speech or sounds 1.13 (.84) 2.14 (.85) 4.62** 1.27 (1.03) 2.13 (1.08) 4.21** 17 plays dress-up games.31 (.83) 1.19 (1.21) 3.07**.61 (1.06) 1.60 (1.31) 4.20** 18 plays pretend games (dolls, cars) b.88 (1.06) 2.00 (1.05) 4.08**.98 (1.17) 2.21 (.93) 5.93** 23 shows food preference by speech or sounds 1.27 (1.11) 2.43 (.93) 4.24** 1.55 (1.14) 2.32 (.94) 3.87** 24 copies you by pretending to do chores.58 (.89) 1.67 (1.16) 4.33**.79 (1.07) 1.68 (1.07) 4.30** 25 seeks praise by showing what s/he did a,b.58 (.94) 2.10 (1.00) 6.16**.74 (1.07) 2.36 (.87) 8.47** 41 shows concern if another child is upset.52 (.70) 1.38 (.87) 4.44**.98 (1.05) 1.89 (1.11) 4.38** 42 can start social exchange with other children b.44 (.73) 1.48 (1.25) 3.56**.45 (.78) 1.60 (.99) 6.52** 43 can join in social exchange with other children.54 (.83) 1.43 (1.25) 3.01**.66 (.87) 1.64 (.97) 5.55** 44 can maintain social exchange with other children.25 (.56) 1.10 (1.41) 2.66**.44 (.82) 1.49 (1.02) 5.80** 51 can take turns in play.83 (.99) 1.48 (.98) 2.55** 1.03 (.83) 1.60 (.95) 3.24** 52 has preferred playmates.58 (.83) 1.33 (1.28) 2.51**.82 (1.05) 1.64 (.90) 4.28** (Continued) 96

108 Table 14: Continued Social Approach/Interest Younger ASD (n=52) Mean (SD) Younger Non-ASD Clinical (n=21) Test value Older ASD (n=62) Older Non-ASD Clinical (n=47) Item Description 20 Interacts with sounds or talking at meals 1.31 (.98) 2.10 (1.00) 3.09** 1.15 (.97) 1.94 (1.03) 4.10** 31 shy around people s/he doesn t know well a 1.29 (1.11) 1.95 (1.07) 2.34* 1.37 (1.13) 1.34 (1.05) ^ looks distant or removed a 1.76 (.95) 2.57 (.60) 3.60** 2.10 (.77) 2.40 (.74) 2.08* 36 watches other children 1.44 (.90) 2.10 (.94) 2.78** 1.60 (.97) 2.06 (.97) 2.50** 37 moves toward other children 1.15 (.92) 1.67 (1.11) 2.04* 1.21 (.98) 1.94 (.97) 3.87** 38 stays close to other children.92 (.93) 1.10 (1.14) (.93) 1.74 (.97) 3.98** 39 tries to play with other children.83 (.81) 1.48 (.93) 2.97**.92 (.95) 1.91 (.88) 5.61** 40 joins play when invited by other children a,b.71 (.78) 1.48 (1.08) 2.96**.85 (.94) 2.04 (1.02) 6.30** 48^ avoids other children 1.88 (.96) 2.43 (.81) 2.28* 2.13 (1.02) 2.53 (.78) 2.26* Test value Mean (SD) t value Mean (SD) Mean (SD) t value Agreeable Nature Mean (SD) Mean (SD) t value Mean (SD) Mean (SD) t value Item Description 45 Is liked by other children his/her age 1.65 (.91) 2.29 (.90) 2.70** 1.60 (.86) 1.77 (.84) other children want to play/be around him/her 1.62 (.87) 2.19 (.81) 2.61** 1.40 (.90) 1.66 (.87) ^ Is aggressive with siblings or other children 2.13 (.97) 2.29 (.90) (1.00) 2.02 (.94) ^ takes toys from siblings or other children a,b 1.81 (1.05) 1.67 (1.11) (1.00) 1.77 (.84) ^ yells and screams 2.10 (1.05) 2.00 (1.10) (1.12) 183 (.99) shares favorite items or toys with adults 1.21 (.98) 1.67 (1.07) 1.76* 1.05 (.97) 1.60 (.95) 2.96** 54 shares favorite items or toys with children a,b.81 (.89) 1.52 (.98) 3.04**.84 (.98) 1.47 (.91) 3.43** *p<.05; **p<.01; ^Indicates item is reverse-scored. Indicates item to be removed due to high intercorrelation (r >.70) with other item(s) in the subscale. a Critical item for younger children; b Critical item for older children (as determined in binary logistic regression) Notes: Italicized items did not meet previous criteria for admission into subscale. Boldface t values indicate items to be retained. 97

109 ASD Non-ASD Test statistic Age months: n=10 n=10 Age in months 1 (mean, SD) 38.5 (4.3) 37.7 (4.2) t =.466 IQ 1 (mean, SD) 78.6 (13.2) 82.3 (13.9) t = Verbal ability^2 (n, % verbal) 5 (71% of known) 7 (78% of known) X 2 =.333 Gender 2 (n, % male) 8 (80%) 8 (80%) X 2 =.000 Ethnicity 2 (n, % Caucasian) 5 (63% of known) 8 (80%) X 2 =.692 Mother s education high school or beyond 2 (n, %) 8 (89% of known) 8 (80%) X 2 =.000 SES score 2 (mean, SD) 33.1 (16.1) 41.0 (29.4) t = Age months: n=17 n=17 Age in months 1 (mean, SD) 51.6 (4.7) 51.9 (4.8) t =.157 IQ 1 (mean, SD) 66.9 (14.3) 71.7 (14.1) t =.966 Verbal ability^2 (n, % verbal) 7 (58% of known) 8 (57% of known) X 2 =.067 Gender 2 (n, % male) 15 (88%) 12 (71%) X 2 =.333 Ethnicity 2 (n, % Caucasian) 13 (77%) 11 (65%) X 2 =.167 Mother s education high school or beyond 2 (n, %) 17 (100%) 14 (100% of known) X 2 =.290 SES score 2 (mean, SD) 23.1 (17.0) 44.9 (22.8) t = -2.89* *p<.05 1 Matched characteristic. 2 Unmatched characteristic. ^Verbal ability as determined by ADI-R functional language definition, when available. SES scores available: 8 of 10 ASD younger; 10 of 10 non-asd younger; 15 of 17 ASD older; 13 of 17 non-asd older. Table 15. Characteristics of Matched Subsample 98

110 Test variable and Groups: n SSI mean (SD); range Younger all 54 SSI items: Autism Typical Younger SSI-T (26 items) : Autism Typical Younger 9 Critical SSI items : Autism Typical Older all 54 SSI items: Autism Typical Older SSI-PS (21 items) : Autism Typical Older 7 Critical SSI items : Autism Typical Discrimination: Autism vs. Typically-developing R.O.C. (AUC) 70.0 (27.6); (14.8); (14.5); (8.4); (5.0); (3.0); (23.4); (18.3); (9.8); (8.9); (3.4); (3.1); Screen Score^ Sensitivity Specificity ^Cutoffs refer to screen scores less than or equal to the score(s) listed. Reminder: these subsets of items emerged from discrimination of ASD versus non-asd clinical groups. Table 16. Sensitivity and Specificity for Discriminating between Autism and Typically-developing Participants 99

111 Discrimination: ASD (autism and PDD-NOS) vs. Typically-developing Test variable and Groups: n SSI mean (SD); range Younger all 54 SSI items: ASD Typical Younger SSI-T (26 items) : ASD Typical Younger 9 Critical SSI items : ASD Typical Older all 54 SSI items: ASD Typical Older SSI-PS (21 items) : ASD Typical Older 7 Critical SSI items : ASD Typical R.O.C. (AUC) 73.9 (24.7); (14.8); (12.8); (8.4); (4.4); (3.0); (27.6); (18.3); (13.7); (8.9); (4.8); (3.1); Screen Score^ Sensitivity Specificity ^Cutoffs refer to screen scores less than or equal to the score(s) listed. Reminder: these subsets of items emerged from discrimination of ASD versus non-asd clinical groups. Table 17. Sensitivity and Specificity for Discriminating between ASD and Typically-developing Participants 100

112 Test variable and Groups: n SSI mean (SD); range Younger all 54 SSI items: Autism NS Clinical Younger SSI-T (26 items) : Autism NS Clinical Younger 9 Critical SSI items : Autism NS Clinical Older all 54 SSI items: Autism NS Clinical Older SSI-PS (21 items) : Autism NS Clinical Older 7 Critical SSI items : Autism NS Clinical Discrimination: Autism vs. Non-Spectrum clinical R.O.C. (AUC) 70.0 (27.6); (25.7); (14.5); (13.8); (5.0); (4.4); (23.4); (24.5); (9.8); (11.0); (3.4); (4.0); Screen Score^ Sensitivity Specificity ^Cutoffs refer to screen scores less than or equal to the score(s) listed. Reminder: these subsets of items emerged from discrimination of ASD versus non-asd clinical groups. Table 18. Sensitivity and Specificity for Discriminating between Autism and Non-Spectrum Clinical Participants 101

113 Discrimination: ASD (autism and PDD-NOS) vs. Non-Spectrum Clinical Test variable and Groups: n SSI mean (SD); range Younger all 54 SSI items: ASD NS Clinical Younger SSI-T (26 items): ASD NS Clinical Younger 9 Critical SSI items: ASD NS Clinical Older all 54 SSI items: Older SSI-PS (21 items): ASD NS Clinical ASD NS Clinical Older 7 Critical SSI items: ASD NS Clinical R.O.C. (AUC) 73.9 (24.7); (25.7); (12.8); (13.8); (4.4); (4.4); (27.6); (24.5); (13.7); (11.0); (4.8); (4.0); Screen Score^ Sensitivity Specificity ^Cutoffs refer to screen scores less than or equal to the score(s) listed. Table 19. Sensitivity and Specificity for Discriminating between ASD and Non-Spectrum Clinical Participants 102

114 Discrimination: Matched ASD (autism and PDD-NOS) vs. Non-Spectrum Clinical Test variable and R.O.C. Matched Groups: n SSI mean (SD); range (AUC) Younger all 54 SSI items: Younger SSI-T (26 items): ASD NS Clinical ASD NS Clinical Younger 9 Critical SSI items: ASD NS Clinical Older all 54 SSI items: Older SSI-PS (21 items): Older 7 Critical SSI items: ASD NS Clinical ASD NS Clinical ASD NS Clinical (22.4); (26.3); (12.4); (14.3); (3.5); (4.8); (30.8); (21.2); (15.0); (8.9); (5.0); (3.0); Screen Score^ Sensitivity Specificity ^Cutoffs refer to screen scores less than or equal to the score(s) listed. Table 20. Sensitivity and Specificity for Matched Groups 103

115 Discrimination: ASD (autism and PDD-NOS) vs. Non-Spectrum Clinical Test variable and Groups: n SSI mean (SD); range R.O.C. (AUC) Screen Score^ Sensitivity Specificity Younger: Verbal SSI-T (26 items): Verbal 9 Critical SSI items: ASD NS Clinical ASD NS Clinical Nonverbal SSI-T (26 items): ASD NS Clinical Nonverbal 9 Critical SSI items: ASD Older: Verbal SSI-PS (21 items): Verbal 7 Critical SSI items: NS Clinical ASD NS Clinical ASD NS Clinical Nonverbal SSI-PS (21 items): ASD NS Clinical Nonverbal 7 Critical SSI items: ASD (14.6); (13.1); (4.4); (3.6); (10.2); (13.8); (4.4); (3.5); (13.0); (10.6); (5.0); (4.0); (14.8); (11.1); NS Clinical ^Cutoffs refer to screen scores less than or equal to the score(s) listed (5.0); (3.5); Table 21. Sensitivity and Specificity for Verbal and Nonverbal Participants (for Those with Classification Available) 104

116 Appendix C: Figures 105

117 Median = 125 Mode = 125 Range = (no significant outliers) Skewness = (S.E. =.221); somewhat negatively skewed Kurtosis = (S.E. =.438); acceptable A (Continued) Figure 1. SSI Score Distributions for Typically-Developing Participants: A Months of Age, B Months of Age, C Months of Age (full sample) 106

118 Figure 1 Continued Median = 123 Mode = 137 Range = (scores at or below 68 are > 3 SD from mean) Skewness = (S.E. =.272); negatively skewed Kurtosis = (S.E. =.538); somewhat leptokurtic (i.e., peaked) B (Continued) 107

119 Figure 1 Continued Median = 124 Mode = 137 Range = (scores at or below 72 are > 3SD from mean) Skewness = (S.E. =.173); negatively skewed Kurtosis =.761 (S.E. =.344); somewhat leptokurtic (i.e., peaked) Kolmogorov-Smirnov Z value =.895 (NS; sample appears normally distributed) C 108

120 Median = 87 Mode = 78 (multi-modal) Range = (no significant outliers) Skewness =.168 (S.E. =.237); acceptable Kurtosis = (S.E. =.469); acceptable A (Continued) Figure 2. SSI Score Distributions for Combined Clinic Sample (Including ASD): A Months of Age, B Months of Age, C Months of Age (full sample) 109

121 Figure 2 Continued Median = 95 Mode = 86 (multi-modal) Range = (no significant outliers) Skewness = (S.E. =.199); acceptable Kurtosis = (S.E. =.396); acceptable B (Continued) 110

122 Figure 2 Continued Median = 91 Mode = 104 Range = (no significant outliers) Skewness = (S.E. =.153); acceptable Kurtosis = (S.E. =.306); acceptable Kolmogorov-Smirnov Z value =.728 (NS; sample appears normally distributed) C 111

123 Median = 71 Mode = 78 Range = (no significant outliers) Skewness =.320 (S.E. =.330); acceptable Kurtosis =.151 (S.E. =.650); acceptable A (Continued) Figure 3. SSI Score Distribution for Participants with ASD: A Months of Age, B Months of Age, C Months of Age (full sample) 112

124 Figure 3 Continued Median = 75 Mode = 56 (multimodal) Range = (no significant outliers) Skewness =.113 (S.E. =.304); acceptable Kurtosis = (S.E. =.599); acceptable B (Continued) 113

125 Figure 3 Continued Median = 74 Mode = 56 Range = (no significant outliers) Skewness =.217 (S.E. =.226); acceptable Kurtosis = (S.E. =.449); acceptable Kolmogorov-Smirnov Z value =.748 (NS; sample appears normally distributed) C 114

126 115 Figure 4. Score Distribution and Recommended Cutoff Score for 26-Item SSI-Toddler (SSI-T) (n=73)

127 116 Figure 5. Score Distribution and Recommended Cutoff Score for 9 Critical SSI Items for Younger Children (n=73)