MEASURING PROGRESS WITHIN A SOCIAL THINKING CURRICULUM FOR PRESCHOOLERS WITH SOCIAL-EMOTIONAL CHALLENGES: TOOLS FOR TEACHERS.

Transcription

1 MEASURING PROGRESS WITHIN A SOCIAL THINKING CURRICULUM FOR PRESCHOOLERS WITH SOCIAL-EMOTIONAL CHALLENGES: TOOLS FOR TEACHERS A Project Presented to the faculty of the Graduate and Professional Studies in Education California State University, Sacramento Submitted in partial satisfaction of the requirements for the degree of MASTER OF ARTS in Education (Special Education) by Hayley Amanda Kercher SUMMER 2016

2 2016 Hayley Amanda Kercher ALL RIGHTS RESERVED ii

3 MEASURING PROGRESS WITHIN A SOCIAL THINKING CURRICULUM FOR PRESCHOOLERS WITH SOCIAL-EMOTIONAL CHALLENGES: TOOLS FOR TEACHERS A Project by Hayley Amanda Kercher Approved by:, Committee Chair Jean Gonsier-Gerdin, Ph.D. Date iii

4 Student: Hayley Amanda Kercher I certify that this student has met the requirements for format contained in the University format manual, and that this project is suitable for shelving in the Library and credit is to be awarded for the project., Department Chair Susan Heredia, Ph.D. Date Graduate and Professional Studies in Education iv

5 Abstract of MEASURING PROGRESS WITHIN A SOCIAL THINKING CURRICULUM FOR PRESCHOOLERS WITH SOCIAL-EMOTIONAL CHALLENGES: TOOLS FOR TEACHERS by Hayley Amanda Kercher The rate of young children being diagnosed with autism spectrum disorder (ASD) continues to climb. Interventions vary greatly among the diverse population of individuals with ASD, as one size does not fit all. For preschool-aged children with ASD or other social learning differences, who may have typical cognitive or language skills and who require less adaptive support, but who struggle with the more nuanced, complex world of the why behind social interactions and behaviors, Hendrix, Palmer, Tarshis, and Winner s The Incredible Flexible You is a popular curriculum for early childhood special education programs. This is due in part to the curriculum s solid foundation in evidence-based practices and research-proven theories and concepts (Winner, 2007; Hendrix et al., 2013; Crooke, Hendrix, & Rachman, 2007; Koning, Magill-Evans, Volden, & Dick, 2008). However, this curriculum lacks tools and strategies for measuring student progress in an objective manner and making data-driven decisions for students goals and objectives. The purpose of this project was to create observation-based data collection tools and a corresponding progress monitoring system for educators implementing this curriculum. To create the data collection and progress monitoring tools, The Incredible Flexible You (Vol. 1) was analyzed to determine observable behaviors for educators to measure, which v

6 could then be applied to student progress rubrics supplied within the curriculum. This project includes data collection tools for each of the five concepts in The Incredible Flexible You (Vol. 1) (i.e., Thinking Thoughts and Feeling Feelings, The Group Plan, Thinking With Your Eyes, Body in the Group, and Whole Body Listening) for both verbal and nonverbal behavior, corresponding progress monitoring rubrics adapted from the curriculum s manual, and a mastery sheet to measure individual student progress across time. The hope is that this set of tools for teachers will support educators as they implement this curriculum, providing insight into the progress and growth their students are making in the world of social learning, and could even be used to help determine the efficacy of this curriculum in future research., Committee Chair Jean Gonsier-Gerdin, Ph.D. Date vi

7 ACKNOWLEDGEMENTS This project would not have been possible if not for my family, my partner Marshall, my enthusiastic colleagues Korina, Lauren, and Marina, my advisor Dr. Gonsier-Gerdin, Michelle Garcia Winner and the rest of the brilliant minds involved with Social Thinking, and of course, my wonderful students. vii

8 TABLE OF CONTENTS Page Acknowledgements... vii Chapter 1. INTRODUCTION... 1 Background of the Problem... 5 Statement of the Problem... 6 Purpose of the Project... 7 Theoretical Framework... 8 Definition of Terms Assumptions Justification Limitations Organization of the Remainder of the Project REVIEW OF THE LITERATURE Introduction Social Skills Interventions for Individuals with ASD with Typical Cognitive Abilities Lack of Generalization: Clinic-Based Social Skills Trainings & Other Challenges...27 Measuring Student Progress Within Interventions Summary METHODOLOGY Analysis of Key Concepts and Identification of Observable Behaviors Development of Data Collection Tools viii

9 Progress Monitoring System DESCRIPTION OF THE PROJECT, DISCUSSION, AND RECOMMENDATIONS.. 48 Description of the Project Discussion Recommendations for Future Practice and Research Appendix A. Tools for Teachers: A Progress Monitoring System to be Utilized by Educators Implementing The Incredible Flexible You (Vol. 1) References ix

10 1 Chapter 1 INTRODUCTION Autism spectrum disorder (ASD) is a developmental disability that can cause significant challenges in social, communication, and behavioral abilities (American Psychiatric Association, 2013). Individuals with ASD may learn, problem-solve, and communicate differently than their neurotypical peers, and studies have shown that early intervention (e.g., from birth to five) can help children with ASD learn important skills and improve their development in regards to communication, social skills, and adaptive behavior (McConnell, 2002). Prior to the newest edition of the Diagnostic and Statistical Manual, published by the American Psychiatric Association, individuals displaying autistic-like characteristics could potentially receive a diagnosis of a number of separate and distinct disorders, including Asperger syndrome, Pervasive developmental disorder, not otherwise specified (PDD-NOS), or Autism disorder. In the latest (fifth) edition of the Diagnostic and Statistical Manual (DSM-5), these disorders are all incorporated into the continuum of the Autism Spectrum Disorder diagnosis, though distinctions may be made during diagnosis regarding severity (American Psychiatric Association [APA], 2013). According to the DSM-5, clinical diagnostic criteria for ASD include: (a) persistent deficits in social communication and social interaction; (b) restrictive, repetitive patterns of behavior, interests, or activities; (c) presence of symptoms in early developmental period; (d) symptoms cause clinically significant impairment in social or occupational areas of functioning; and (e) symptoms are not explained by intellectual disability or

11 2 global developmental disability. Specifications regarding severity are established in a three level system, based on the individual Requiring support (Level 1), Requiring substantial support (Level 2), or Requiring very substantial support (Level 3) (APA, 2013). Individuals with ASD may share common deficits in social communication regarding social reciprocity (e.g., sharing of interests, initiation and response to social interactions), nonverbal communication (e.g., understanding and using eye-contact, gestures, body language, facial expressions), and relationships (e.g., developing and maintaining relationships, interpreting others behavior and adjusting ones own behavior in response, engaging in imaginative or sustained play with others) (American Psychiatric Association, 2013). Individuals with ASD may also share restricted, repetitive patterns of behavior, interests, or activities in regards to repetitive or stereotyped motor behaviors or speech (e.g., echolalia, flapping of hands), insistence on sameness (e.g., rigidity in regards to routines or rituals, difficulties with transitions or small changes in schedule), highly restricted and abnormally intense interests (e.g., perseverative interest in certain objects or topics), and abnormal reaction or interest to sensory input (e.g., adverse reaction to specific sounds, textures, or images, excessive smelling/touching/tasting of objects, visual fascination with lights/movement) (APA, 2013). Children with ASD or other conditions that impact their ability to engage in typical social reciprocity, such as Attention-Deficit/Hyperactivity Disorder (ADHD), often struggle with a variety of social-cognitive tasks and abilities (Jones & Carr, 2004; Winner, 2007). The degree to which a child with ASD or another social learning

12 3 challenge is able to function successfully in our social world is impacted by the severity of his/her disability, as well as the presence of other comorbid conditions, such as intellectual disability (American Psychiatric Association, 2013). Regardless of the severity of the disability, however, research has found that individuals with ASD also struggle with tasks related to executive functioning (i.e., higher-order cognitive functions that are goal-oriented, including working memory, cognitive shifting, planning, and inhibition). Other differences common among individuals with ASD includes difficulties regarding theory of mind (i.e., the social-cognitive ability to take the perspective of another person, and to attribute complex mental states, such as beliefs, emotions, intentions, or thoughts to another person), self-regulation, social cognition, social skills, and social pragmatics (Jones & Carr, 2004; Kimhi, Shoam-Kugelmas, Agam, Ben- Moshe, & Bauminger-Zviely, 2014; Winner, 2007). In providing interventions for students with ASD, research has shown significant success in teaching children discrete social skills such as greetings, brief conversational scripts, play sequences, and other rote tasks (White, Keonig, & Scahill, 2006). Interventions utilized in teaching these skills, such as Discrete Trial Training (DTT) and Applied Behavioral Analysis (ABA), are critical elements of most early intervention programs for children with ASD who have greater support needs, and progress in learning these discrete skills is easily measured. However, for students with autism with more typical cognitive skills or who require less support, a need emerges for instruction and teaching on the more nuanced, complex facets of social interaction and communication, such as perspective taking (e.g. knowing how to share space, self-

13 4 regulation in a group, or cooperation and negotiation in a group). One example of a program that meets this need is the Social Thinking methodology, a curriculum and teaching framework developed by Michelle Garcia Winner for individuals with social learning challenges (Winner, 2007). This methodology helps bridge the gap between teaching students memorized rote social skills and the ability for students to read the current situation and adapt their social skills based on the expectations of the people and the environment (Crooke, Hendrix, & Rachman, 2007; Winner, 2007). One component of the Social Thinking methodology is the preschool and earlyelementary curriculum, The Incredible Flexible You (Hendrix et al., 2013). This curriculum, specifically designed for young children with social learning differences, such as ASD or ADHD, is often implemented within the preschool setting with service providers, such as the early childhood special education specialist or the speech/language pathologist, instructing the students and guiding the lessons (Winner, 2007). Indeed, the school district in which the author of this project is employed has been implementing The Incredible Flexible You for several years, in a special education preschool program designed for children who are at or near grade level academically, but are struggling socially and have ASD or other social learning challenges. However, for educators implementing this curriculum in their classrooms measuring progress of students may be difficult, as these skills are not taught in a concrete, trial-based lesson style, but are instead introduced through stories, discussion, and role-play (Hendrix, Palmer, Tarshis, & Winner, 2013).

14 5 Background of the Problem Measuring student progress within a teaching methodology such as Winner s Social Thinking is a challenge for educators and service providers. This instructional framework is comprised of an embedded web of concepts, strategies, lessons and tools that are all based on current research and literature related to typical development, socialemotional learning, executive functioning, problem solving, social skills, social language, behavioral principles, cognitive behavioral therapy, pragmatic language, and ASD (Winner, 2007). In particular, The Incredible Flexible You curriculum is designed to help young students develop the skills they need to understand the why behind social interactions and to be flexible social thinkers and social problem solvers. The Incredible Flexible You uses a thought-based approach to explicitly teach the concepts and vocabulary through structured play, and gives educators lessons, activities, songs, and strategies to teach these nuanced concepts to students with average (or above average) language and learning ability, but who struggle with social learning (Hendrix et al., 2013). The curriculum offers general rubrics as a measurement tool to track progress, which are helpful, but limited in their broad application and subjectivity. For example, each concept comes with a rubric intended for measuring individual student progress, which asks the teacher to measure the child s knowledge of the concept on a 0-5 scale, with 0 indicating No understanding of the concept. They are not using the vocabulary or demonstrating any of the requisite associated behavior and a 5 indicating Solid understanding of the concept and can demonstrate its use with minimal cues (Hendrix et

15 6 al., 2013, p. 62). This lack of specificity is a problem, because students need to be evaluated throughout the school year based on their progress in the classroom. In fact, special education teachers must write measureable, observable goals for each of their students and report on their progress with data on a regular basis. For students participating in special education programs, the goal is always generalization--can the students use the tools and integrate the lessons outside of the structured lesson format? However, without specific tools for data collection, an individual student s progress within the curriculum is measured based only on the teachers or parents impressions of the child s ability, which is subjective and makes decisions regarding a student s goals and services a challenge (Crooke et al., 2007). Similarly, guiding lesson development and implementation to support generalization of the targeted skills is difficult without data collection and analysis. Statement of the Problem The need at the heart of this project is the lack of a clear, easy-to-use and widely applicable method for measuring progress and tracking the generalization of skills within the classroom application of a social learning teaching methodology. As previously described, while there are rubrics for each lesson within The Incredible Flexible You (Vol. 1), these are based on the raters general impressions of the student s behavior and are not specific to individual concepts demonstrated by the student. Without tools for collecting data on student behavior and measuring progress, educators implementing such a curriculum are unable to track student development or to report accurately on student goals and, even more critically, have no way of demonstrating that the students are

16 7 generalizing the skills learned in specific lessons in more spontaneous, nonstructured activities throughout the school day. Purpose of the Project This project aims to provide educators with a set of tools to collect data on student behavior and monitor student progress to be used in the classroom in conjunction with a social learning curriculum for preschoolers with social learning challenges, Hendrix, Palmar, Tarshis, and Winner s The Incredible Flexible You (Vol. 1), a component of the Social Thinking methodology. The primary purpose was to develop a set of observation-based measurement tools to track student behavior (both verbal and nonverbal) during class and to specifically record individual application of concepts and skills during naturally occurring classroom activities. Target skills to be measured include perspective taking, thoughts/feelings in self and others, identifying the group plan and recognizing how one s behavior affects others thoughts, self-regulation, social problem solving, and other social thinking skills identified through research as major areas of need in individuals with autism (Hendrix et al., 2013). This project provides observation-based data collection tools to inform the rubrics that accompany the curriculum, which will in turn aid in progress monitoring of the students, lesson planning, and goal development. The secondary purpose of this project was to provide individuals using The Incredible Flexible You (Vol. 1) with information that could be potentially used for future evaluation of the efficacy of this particular curriculum. Winner s Social Thinking methodology, including the early-learner curriculum The Incredible Flexible You, is a

17 8 popular choice for many service providers supporting children with social learning differences and has been the focus of a number of studies regarding interventions for students with ASD with typical cognitive abilities (Crooke et al., 2007). In developing a set of tools specifically for on-going, observation-based data collection and progress monitoring in a preschool classroom environment, both educators and researchers will be able to gather more data for evaluating the use of this curriculum, both for individual students and for the program as a whole. This project will be distributed to other special education preschool teachers utilizing the curriculum within the author s school district, with the goal of providing educators with an easy method for collecting data and monitoring student progress. Theoretical Framework Several theoretical frameworks support this project. First, this project considers the conceptualization of individuals with autism spectrum disorder, in particular those who have typical cognitive ability and who require less adaptive support, as people with particular needs in the areas of executive functioning, theory of mind, self-regulation, social cognition, social skills, and pragmatics (Jones & Carr, 2004; Winner, 2007). Indeed, this is the same framework that supports The Incredible Flexible You curriculum. This particular population of children with ASD or other social learning challenges children who have acquired language, who may even display advanced language skills in regards to word recognition or reading skills, children who may be great talkers with advanced vocabularies but struggle to relate to their peers, children who may engage in play with peers but have difficulty departing from their preferred play

18 9 theme or sharing an imagination these are the children for whom this curriculum was developed (Hendrix et al., 2013). Social learning teaching frameworks such as Winner s Social Thinking methodology are used by a wide variety of educators because they address in a comprehensive, dynamic way the complex learning and processing challenges students with social learning challenges, such as ASD, may possess. The Incredible Flexible You is a helpful tool for teachers of young students with social learning challenges who have higher cognitive and language-processing skills. Second, this project is informed by the theoretical framework of applied behavioral analysis, which identifies observation of social behavior as the most ecologically valid measure of children s social skills (McMahon, Vismara, & Solomon, 2013). Applied behavior analysis is the application of the science of how learning occurs, utilizing and manipulating the consequences of a behavior to either increase or decrease the likelihood of repetition. This project focuses on developing tools based on behavioral observation as a means of measuring progress and generalization of social skills, concepts, and socialemotional learning. The direct observation of children s social-communication skills in a natural environment, such as a preschool classroom, provides a comprehensive way to monitor effects of intervention and guide the implementation of a particular curriculum (Goldstein, Kaczmarek, & English, 2002).

19 10 Definition of Terms Applied Behavioral Analysis (ABA) Applied behavior analysis is the process of systematically applying interventions based upon the principles of learning, motivation, and behavior theory. ABA is a process which is designed to bring about meaningful and positive change in behavior. ABA is an evidence-based practice for supporting individuals with autism spectrum disorder and helps solve problems of social significance (Batshaw, Roizen, & Lotrecchiano, 2013; Center for Disease Control, 2016). Autism Spectrum Disorder (ASD) Autism spectrum disorder (ASD) is a neurologically-based developmental disability that involves significant challenges in social, communication, and behavioral abilities Children with ASD may learn, problem-solve, and communicate differently than their neurotypical peers (i.e., children who are typically-developing) (American Psychiatric Association, 2013). Discrete Trial Training (DTT) Discrete trial training (DTT) is a comprehensive training package based on applied behavioral analysis, developed in part by the work of Ivar Lovaas. DTT has shown to be effective in teaching children with ASD a wide variety of behaviors, including languageand play-based skills. Complete learning trials make up DTT in which students work with a teacher to learn specific skills that have been broken down into discrete lessons with carefully controlled prompts and reinforcements. A critical hallmark of DTT is the careful fading of prompts so the target behavior or skill can then be exhibited in the

20 11 natural environment (Batshaw et al., 2013). Executive Functioning (EF) Executive functioning refers to higher-order cognitive functions that relate to goaldriven behavior. These functions may include working memory, cognitive shifting, organization, time-management, flexible thinking, planning, and inhibition (Kimhi, 2014) High-Functioning Autism (HFA) High-functioning autism is a term for individuals with ASD who possess stronger cognitive skills and intellectual and language abilities and who require less adaptive support. Asperger s syndrome is a form of HFA. However, in the interest of promoting people-first language and in recognition of the new DSM-V, which refers to autism spectrum disorder as an inclusive continuum, this project will avoid the use of this term unless citing individual studies that use it. Neurotypical Neurotypical refers to individuals who are not displaying or characterized by autistic or other neurologically atypical patterns of thought or behavior when compared to individuals with neurological differences, such as ASD. Neurotypical is a term commonly utilized among individuals on the autism spectrum (Neorutypical, 2016). Social Thinking Social Thinking is a teaching framework for individuals, aged preschool through adult, created by Michelle Garcia Winner. Social Thinking includes a vocabulary which establishes a common language for discussing social functioning, curriculum lessons and strategies. Social Thinking is a cognitive behavioral approach that breaks down social

21 12 concepts (e.g., understand the thoughts, feelings and perspectives of others; using and interpreting eye contact and body language; self-regulation as a means of impacting the thoughts others have about you) into concrete, teachable formats, and includes a curriculum for preschool and early elementary students called The Incredible Flexible You (Winner, 2007). Social Skills Training (SST) or Social Skills Training Program (SSTP) Social skills training programs (SSTP), also known as social skills training (SST), are an intervention approach for individuals with ASD. Social skills, ranging from greetings and conversations to body language and problem solving, are taught to individuals with all levels of ability. SSTs are developed for a corresponding range of participants, ranging from those with cognitive impairment who require greater levels of adaptive support to those with typical cognitive abilities and who require less adaptive support (McMahon et al., 2013). Theory of Mind (ToM) Theory of mind refers to the social-cognitive ability to take the perspective of another person and to attribute complex mental states, such as beliefs, emotions, intentions, or thoughts, to another person. Early research on ToM, including studies conducted by Baron-Cohen, Leslie, and Frith (1985), demonstrated that individuals with autism tend to have difficulty understanding and imputing beliefs to others. In other words, they may lack an understanding that others have thoughts and feelings different from their own, and may have difficulty predicting others behavior (Baron-Cohen et al., 1985; Kimhi, 2014). Theory of mind is well-established in typically-developing four-

22 13 year-olds who can then use this ability to understanding the thoughts and feelings of others as they engage in the social world (Hendrix et al., 2013). Assumptions This project assumes several premises to be true. First, it is a fact that children and adults with autism demonstrate some delays, deficits, or atypical characteristics in the frequency, type, and quality of social interactions and social relationships with other individuals (McConnell, 2002). A second assumption is that students with ASD, who have more typical intellectual abilities and may require less adaptive support, will benefit from direct instruction of nuanced social thinking concepts (e.g., perspective taking, context analysis, social problem solving) with repeated exposure to these concepts and opportunities to practice. As such, a third assumption is that curriculum such as The Incredible Flexible You, which strives to teach not only the what of social skills but the why of social cognition, is beneficial to children with social learning challenges (Hendrix et al., 2013). Finally, this project assumes that observation-based behavioral data collection is an effective way to measure progress, especially when the information being gathered might occur in a naturalistic environment. Justification This project will benefit individuals who are implementing a social learning curriculum, such as Hendrix, Palmar, Tarshis, and Winner s The Incredible Flexible You (Vol. 1) curriculum, a component of Winner s Social Thinking methodology. Such individuals might include special education preschool teachers, general education teachers, speech/language pathologists, or even parents. Data collection is an integral

23 14 component of any special education program, as the information gathered informs teachers of individual student progress, as well as provides information on the class as a whole. Currently, special education preschool teachers who may be using this or a similar teaching framework lack a clear, functional system for collection data on student progress. Developing a system of observation-based data collection tools and a progress monitoring system could facilitate educators utilizing this social learning curriculum to more effectively measure student growth and progress within naturalistic, play-based or non-structured classroom activities. The project will be disseminated to the author s colleagues within the school district, which include special and general education preschool teachers and speech/language pathologists. In addition, those wishing to measure the efficacy of this particular intervention could utilize these tools. Limitations The nuanced, complex, and multifaceted nature of social thinking and social learning does not lend itself well to the rigid nature that strict, performance-based observation requires. By using a data collection and progress monitoring system with this particular curriculum, the complete picture of how an individual student is understanding each of the concepts may not be captured. For example, a child who is utilizing the vocabulary related to thinking with your eyes at a rate of about 30% accuracy may indeed understand the concept much more, but simply has difficulty expressing herself. However, for teachers who must monitor progress and who are struggling to report on goals and objectives, a system for data collection and progress monitoring is needed.

24 15 In addition, this data collection and progress monitoring system is intended to be utilized by educators using The Incredible Flexible You (Vol. 1) curriculum, and this project is limited in that regard. Future developments could improve this by developing a set of tools for Volume 2 of the curriculum as well. Another limitation of this project is that the tools were developed based on the author s experience with a specific, limited group of students (18 four- and five-year-olds, almost all boys, with autism, ADHD, or other social learning challenges). Finally, this project is limited in that the tools developed have not been evaluated for inter-rater reliability, and the assumption that they could be used to measure the efficacy of the curriculum in the future is therefore not guaranteed. Organization of the Remainder of the Project The remainder of this project will be organized into three chapters, followed by the project product as Appendix A. Chapter two serves as a review of the literature that supports this project, including: (a) successful components of social skills and social learning interventions for individuals with ASD who have typical cognitive skills and require less adaptive support; (b) challenges with particular social skills interventions, especially regarding generalization; and (c) measuring student progress within social skills interventions. Chapter three discusses the methodology of the project, including the analysis of The Incredible Flexible You (Vol. 1) curriculum concepts, the development of the data collection tools, and the development of the progress monitoring system. Chapter four describes and discusses the project, while considering limitations and potential recommendations for future practice and research. Finally, the project itself

25 16 will be included as Appendix A: Tools for Teachers: A Progress Monitoring System to be Utilized by Educators Implementing The Incredible Flexible You (Vol. 1).

26 17 Chapter 2 REVIEW OF THE LITERATURE Introduction Individuals with autism spectrum disorder (ASD) typically demonstrate deficits, delays, or atypical characteristics in regard to the frequency, variety, and quality of social interactions and social relationships (American Psychiatric Association, 2013). These social reciprocity deficits were a component of the original description of autism in the 1940s and continue to be a common element of all diagnostic tools to date (McConnell, 2002; White et al., 2006). While interventions targeting social gains vary considerably depending on the intended population, evidence has shown that: a) social interactions can be taught and learned, b) social interaction in typical settings can be successfully accomplished, and c) substantial positive outcomes accrue (McConnell, 2002, p. 366) from receiving interventions which are focused on social skills and social awareness. This project is informed by a review of the literature that examines a variety of interventions for children with autism, focusing on those with typical cognitive abilities and who need less adaptive support, or individuals with High-Functioning Autism (HFA). In addition, this chapter reviews research supporting methods of data collection and progress measurement within social skills and social thinking interventions. Social Skills Interventions for Individuals with ASD with Typical Cognitive Abilities Interventions for children with autism tend to focus on impairments in social communication skills, especially in the development of reciprocal social interaction (Barry et al., 2003). Some interventions, collectively referred to as social skills training

27 18 programs (SSTP or SST), have been shown to be especially effective for a wide variety of individuals with ASD, ranging from those who are more significantly impacted by their autism and require focus on more concrete concepts, such as eye-contact, simple greetings, imitation, and turn-taking, to those who have more typical cognitive abilities and may require less adaptive support (McMahon, Vismara, & Solomon, 2013). Individuals with autism who have higher cognitive abilities and/or verbal IQ scores may benefit from interventions focusing on more nuanced skills and abilities, such as perspective taking, reading nonverbal and verbal cues and interpreting the behavior of others, theory of mind, understanding and using humor, and adjusting their own behavior in relation to others (Barry et al., 2003). One such intervention approach is the Social Thinking methodology, developed by Michelle Garcia Winner, which is based on a wide body of research-based theories regarding the unique challenges faced by those with social learning differences (Crooke et al., 2007; Winner, 2007). Deficits in executive functioning, theory of mind, self-regulation, social cognition, social skills, and pragmatics all contribute to social challenges for individuals with ASD, and these areas of need are addressed in Winner s methodology, as discussed in Chapter 1 (Jones & Carr, 2004; Winner, 2007). Literature Reviews of Interventions Krasny, Williams, Provencal and Ozonoff (2003) offered a review of five social skills interventions for students with ASD who were described as high functioning and outlined several key components of a successful intervention. Kransy et al. (2003) found that successful interventions include certain central principles, such as making the

28 19 abstract concrete, providing structure and predictability, providing scaffolded language support, providing multiple and varied learning opportunities, including other -focused activities (e.g., pair work), fostering self-awareness and self-esteem, selecting relevant goals, organizing the program in a logical and sequential order, and providing opportunities for programmed generalization and on-going practice. A subsequent literature review by Matson, Matson, and Rivet (2007) found similar conclusions. Matson et al. (2007) stressed the importance of offering social skills development intervention to individuals with ASD at an early age and described the target skills of already pre-existing social skills interventions, including initiation of speech, eye-contact, appropriate content of speech (e.g., greetings, niceties), appropriate speech intonation, number of words spoken, appropriate facial affect, appropriate motor movements, verbal disruptions, leaving the group, unpleasant demeanor, conversational speech, number of interactions, and an overall rating of ability and proficiency in social skills. This literature review examined 79 distinct treatment studies and then categorized the different interventions into five distinct categories, which were modeling and reinforcement, peer-mediated interventions, reinforcement schedules and activities, scripts and stories, and miscellaneous. Each category was broken down and examples of each were discussed, and modeling and reinforcement were identified as the most frequently used and most effective strategies. Sustained and varied opportunities to practice the new skill and thought processes were thought to be one of the most invaluable components of a successful intervention. Matson et al. (2007) concluded by discussing the need for interventions to offer embedded opportunities for generalization

29 20 for the participants and for studies to investigate the effectiveness of these interventions. Studies Examining Specific Interventions Theory of mind. Gevers, Clifford, Mager, and Boer (2006) conducted a study that evaluated the use of a theory of mind-based social cognition training (Steerneman, Jackson, Pelzer & Muris, 1996) for children with ASD. Gevers et al. (2006) used the ToM test (Muris et al., 1999) as a pre- and posttest measure, and also utilized parent report questionnaires as a pre- and posttest measure of child socialization and social skills. The ToM test comprises an interview for children that includes vignettes, stories, and drawings about which the child must answer a number of questions. Within the test, there are three subscales: a) precursors of theory of mind, b) first manifestations of a real theory of mind, and c) more advanced elements of theory of mind. Scores from the items from each of the subtests are totaled, ranging from Participants included 18 elementary-aged children who had pervasive developmental disorders, not otherwise specified (PDD-NOS), which is now considered part of the autism spectrum disorder diagnosis (American Psychiatric Association, 2013). The intervention included the social cognition training developed by Steerneman et al. (1996), which occurred through weekly 60-minute group sessions and targeted the development of ToM. In addition, five educational trainings were provided to the parents regarding ToM, promoting social cognition in their children, and PDD-NOS. Gevers et al. (2006) found that ToM scores improved for all participants among all subtests of the ToM test, with the exception of emotion recognition. Significant improvements for perception/imitation, first order belief (a person s ability to attribute

30 21 beliefs to another person, e.g., Matt thinks the pencil is in the box ) and irony/humor were also found. Significant improvement for adaptive functioning was found as well as in all socialization subdomains (i.e., interpersonal relationships, play/leisure, and social skills). The authors concluded that despite limitations regarding sample size, this study demonstrated that school-age children with ASD are able to benefit from ToM-based training aimed at developing social-cognitive skills. An earlier study by Ozonoff and Miller (1995) was one of the first of its kind to examine ToM training in an experimental design with a control group. Ozonoff and Miller (1995) conducted a four-and-a-half month intervention that aimed at teaching ToM skills to adolescent boys with ASD who had typical cognitive skills. Four boys were in the control group and five received the intervention. Pre- and posttest measures were given to the participants, assessing a variety of ToM tasks including: False-belief skills (e.g., Students are shown a box labeled M&Ms. The box is opened to reveal not candy, but a pencil inside. Students are asked what a peer might think is inside the box if they saw it sitting on the table). Second-order belief attribution skills (e.g., Children hear a vignette about two children, Mary and John, playing in the park with miniature toys acting out the scene. After hearing the story, children were asked to predict what Mary thought regarding John s whereabouts and where she thinks he might have gone). Justification/explanation skills (e.g., Children have to explain their logic as to why they believed what they did regarding the characters thoughts) (Ozonoff & Miller, 1995).

31 22 Each task had several levels of difficulty. For example, some had visual supports while subsequent tasks did not. In addition to the pre- and posttest ToM tasks, participants were rated using a standardized social skills rating system (Gresham & Elliott, 1990) by parents and teachers. The social skills training program consisted of 14 sessions which used a group design to teach and practice concepts such as perspectivetaking, conversational skills, reading and interpreting nonverbal communication, etc. Through pre- and post-intervention assessment, Ozonoff and Miller (1995) found significant improvement within the intervention group on ToM tasks. No change in ToM tasks was found in the control group. In light of these findings, authors suggested that the intervention had a robust affect on the ability of the participants to understand and apply theory of mind knowledge. However, parent and teacher assessment of social skills did not demonstrate significant changes which indicated a lack of skill generalization. Social skills and social cognition. Crooke, Hendrix, & Rachman (2007) examined the use of parts of Winner s social cognitive intervention methodology, Social Thinking, in order to determine if this intervention strategy resulted in effective and generalized progress in students described as having HFA, and if so, what are some of the essential components of this curriculum. Crooke et al. (2007) identified the effectiveness of teaching basic social skills through discrete trial training (e.g., greeting, eye-contact, responding to a name, waving, etc.). Such a discrete skills-based approach is useful in students who have limited or emerging language skills. The authors pointed to the vast amounts of research that supports the effectiveness of these strategies in teaching discrete social skills; however, there is a large gap in the research regarding the large-scale

32 23 improvements in enduring social proficiency (Crooke et al., 2007, p. 581) following such interventions. For students with HFA, Crooke et al. (2007) cited Winner s Social Thinking methodology as one current approach that addresses the essential ingredients for social interventions, while also targeting the underlying knowledge necessary for the expression and use of related social skills. Crooke et al. (2007) described Social Thinking as grounded in the framework and theories related to social cognition (e.g., theory of mind/perspective taking, interpreting verbal/nonverbal actions, adjusting behavior to social cues, etc.) and emphasized the importance of teaching the why behind social awareness and skills. The authors hypothesized that teaching social cognitive tasks would result in a change in expected behaviors (e.g., making appropriate comments, initiating conversations appropriately, looking at speaker when spoken to, etc.) and unexpected behaviors (e.g., making rude or off topic comments, perseverating on a particular topic, atypical body movements, etc.) which reflects the common language of the methodology. Participants in the Crooke et al. (2007) study included six male children, aged 9-11 years old. The children were selected based on the criteria of having a current diagnosis of either High-Functioning Autism or Asperger syndrome. In addition to the presence of a diagnosis, the participants were required to have a Verbal IQ within the average range, have hearing within normal limits, and not have any prior or current enrollment in a social intervention program. This study (Crooke et al., 2007) utilized data from only pre- and post-treatment comparisons with the Social Thinking curriculum as the experimental intervention. Data

33 24 collection occurred in the form of behavioral observation with researchers video-taping and coding student interactions before and after the interventions. Dependent variables included five behaviors (three verbal and two nonverbal). Two of these five were coined unexpected (i.e., verbal output that is inappropriate or offensive, and atypical body movements that distracted from the social exchange or activity) and three were described as expected, (i.e., verbal output that is on-topic and involves a comment or question in response to another person, initiations which served to engage another in a novel social exchange, and looking at speaker when spoken to or looking at object or person being discussed). These five behaviors (dependent variables) were further broken down into specific descriptions, which if observed during the intervention and generalization sessions would be noted. Results from this study found significant changes from pre- to post-training measures on both expected and unexpected verbal and nonverbal behaviors. All six participants demonstrated increases in all subcategories of expected behavior, which is generally considered socially appropriate behavior. Four out of six participants demonstrated decreased unexpected behavior. Another study by Koenig, De Los Reyes, Cicchetti, Scahill, and Klin (2009) sought to evaluate social skills interventions for students, as well as the methods used to assess their efficacy. Koenig et al. (2009) specifically examined the use of a social skills training in schools and communities as effective interventions for children with socialcognitive challenges, specifically students with Pervasive Developmental Disorder-Not Otherwise Specified (PDD-NOS). Koenig et al. (2009) found that in general, group interventions intended to promote social reciprocity lacked robust evidence to support

34 25 their efficacy. The authors described a disconnect in how the construct of impaired social reciprocity in children with PDDs (i.e., ASD) is conceptualized, how change, development, and progress associated with intervention is measured, and whether and which changes are observed. Koenig et al. (2009) highlighted important issues in the evaluation of a group-delivered social skills training, as well as interpretation of any outcomes. This review reported several components as effective teaching tools, including utilizing a cognitive-behavioral approach, utilizing role-play and practice opportunities, the use of peer models, and the use of structured methods for teaching and reinforcement. However, inconsistencies in application, as well as inconsistencies in measurement tools, resulted in widely varied outcomes. Koenig et al. (2009) discussed the complex challenges in measuring progress and efficacy among intervention participants, as the construct of social reciprocity as a whole is incredibly complex, and the degree to which children with autism vary in their deficits and strengths is vast. The authors concluded by offering a detailed chart of guidelines for researchers to follow when designing studies to examine interventions, which considers the vast number of individual skills required to develop social reciprocity (Koenig et al., 2009). Cognitive behavioral therapy is another intervention strategy common among students with autism with typical cognitive abilities. Teaching children to use metacognitive skills (i.e., to think about thinking) can be effective in improving social cognitive skills. Koning, Magill-Evans, Volden, and Dick (2008) examined a different cognitive behavior therapy-based (CBT) social skills intervention that utilized components of Winner s Social Thinking curriculum and found similar results to

35 26 Crooke et al. (2007). Koning et al. (2008) implemented the 15-week CBT-based intervention with adolescent boys with ASD, and randomly assigned seven children to a control group and eight to the intervention group. Over the 15 weeks, students in the intervention group attended weekly two-hour group sessions, which addresses a variety of social skills from the CBT approach, including self-monitoring skills, social perception and affective knowledge, conversation skills, social problem-solving skills, and friendship management skills. Koning et al. (2008) used standardized and observational measures to collect data throughout the intervention. Despite the small sample size, results were encouraging. Researchers found that when compared to their peers in the control group, the boys who received the CBT-based intervention significantly improved their ability to infer emotions, use nonverbal cues effectively, interact successfully with a peer, and accurately answer questions about how to respond to a variety of social situations. Pre- and posttest standardized measures found significant improvement in social perception, peer interaction, and social knowledge. Koning et al. (2008) also asserted that the CBT model of intervention worked particularly well for their specific population (i.e., children with HFA) because it addressed generalizability, used a problem-solving, cognitive-based approach that reflected how children learn to make social responses, and because it took advantage of the average or above-average IQs of the study participants. Tse, Strulovitch, Tagalakis, Meng, and Fombonne (2007) conducted another study that addressed social skills among adolescents who were described as having High- Functioning Autism (HFA). Students participated in weekly interventions sessions for 12

36 27 weeks at a psychiatry center, focusing on topics such as awareness and expression of feelings, conversational skills, negotiation, and politeness. Parents filled out rating questionnaires about their children and students themselves also used Likert-style rating systems regarding their own understanding of what they had learned. Results were compared pre- and post-intervention. Tse et al. (2007) found that among the 46 participants significant improvement was found on total scores for social competence, calculated from a series of social responsiveness scales and child behavior scales. Tse et al. (2007) also reported a decrease in problem behaviors, such as hyperactivity, irritability, and other overly sensitive subscales. In addition, researchers found significant improvement in parents assessment of their children s social skills, which suggests generalizability of the lessons. Lack of Generalization: Clinic-Based Social Skills Trainings & Other Challenges Generalization of learned skills should be a critical outcome of interventions. However, as a number of the studies previously described in this chapter indicate, interventions for students with autism that are conducted in a short-term, clinic-based environment may not result in long-term benefits (McConnell, 2002). Barry et al. (2003) examined the efficacy of an out-patient, clinic-based social skills training for children described as having high-functioning autism (HFA). The authors discussed the challenges that individuals with autism face in forming positive peer relationships, and cited cognitive problems including executive dysfunction, impairments in theory of mind, weak central coherence, and challenges in disengaging and shifting attention. The authors determined that out-patient mental health clinics were often utilized as social

37 28 skills intervention services for children with autism and they sought to measure the efficacy of such treatment strategies. Four elementary-aged children with HFA participated and received direct instruction in discrete social skills including greetings, conversational skills, and play skills in an out-patient clinic. Measures included a preand posttest questionnaire for parents, as well as behavioral observations while the students engaged with a typically-developing peer. Behaviors were coded on a 43-item rating system including the categories of greeting, conversation, and play. In addition, the parents were interviewed on a weekly basis across the eight sessions to order to assess whether students were using the targeted skills outside of the clinical setting. Results from the pre- and post-intervention social skills rating system, in combination with results from observation-based behavioral coding, indicated that specific social skills (i.e., greeting, initiation/responding, and play skills) showed significant improvement following the clinic-based social skills training intervention Barry et al. (2003). Conversation skills showed a trend toward improvement, but this was not significant. Per the social skills rating system, parents reported an improvement in greeting skills in nonclinical settings (e.g. the home, in the community), but no significant improvements in other measures. Results from this study indicated that outpatient clinic-based social skills groups did improve several critical social behaviors in the four students with HFA when specific behaviors introduced during the intervention were measured. However, the majority of skills learned did not generalize outside of the clinical setting (Barry et al., 2003).

38 29 Another study by Bauminger (2007) found similar progress with limited generalizability in regards to social skills training programs, even when implemented outside of the out-patient clinical setting. Bauminger (2007) found that a two-year social skills training cognitive-behavioral-ecological (CB-E) intervention for 26 pre-adolescent children described as having HFA resulted in direct and indirect treatment effects regarding the students social interaction capabilities. The intervention utilized weekly small-group lessons with both neurotypical students and the students with HFA, which were conducted within each child s school. Each lesson consisted of a teaching component and a practice which was modeled after cognitive behavioral therapy (CBT) and focused on how a person thinks about social situations as well as how he or she behaves. In this study, a special education teacher led the students through 50 different lessons regarding group dynamics, emotional understanding in relation to verbal/nonverbal emotional behavior, group conversation skills, group cooperative skills, and recognition of nuanced communication, such as sarcasm or irony. Bauminger (2007) measured: (a) the children s change in overt cooperative skills within and outside the groups; and (b) direct (e.g., emotional understanding and problem solving) and indirect (e.g., theory of mind) treatment effects on the students social cognition. Cooperative skills were measured by behavioral observation, and behaviors were coded under five categories including: (a) mutual planning; (b) cooperative behavior; (c) negotiations; (d) eye-contact; and (e) sharing. Direct and indirect treatment effects were measured by a problem-solving measure (Lochman & Lampron, 1986) that included 12 stories to which the students responded, and a theory of mind measure

39 30 (Happe, 1994) that included five strange stories which required students to respond to questions and offer explanations regarding their perception of another person s motivation and thought process. All measures were used for all participants pre- and post-intervention. Bauminger (2007) found that overall, most participants improved significantly in their cooperative skills within the intervention groups, but found no significant improvement in these skills within generalized settings (i.e., lunchroom and playground). Pre- and post-test measurements of problem solving found a significant improvement in initiation on the part of the students, but no other components. Results also indicated significant improvement in students theory of mind scores. Overall, findings regarding the children s social cognition demonstrated a considerable improvement among most variables measured, and that this intervention can be considered effective in promoting social perception and problem solving, both critical components of social cognition. Less encouraging was the lack of generalized progress in spontaneous, nonstructured social interactions among the participants, as noted through behavioral observations. However, the robust improvement during structured group tasks within the intervention groups suggests that perhaps with more intentional opportunities for guided practice (e.g., recess buddies), there might be more generalized improvement in peer interactions (Bauminger, 2007). In short, the provision of on-going interventions that intentionally teach for generalization are critical. Moreover, these interventions need to utilize measures of student performance across time and settings.

40 31 Measuring Student Progress Within Interventions The methods for measuring outcomes within studies are also critical components in regards to demonstrating validity. McMahon, Vismara, and Solomon (2013) argued that the vast majority of previous studies of interventions for children with autism who have typical cognitive abilities demonstrated that while various programs can be used to improve child awareness of social skills, they may be limited by their use of measurement strategies. Child progress in both comprehending social skills and performing social skills were often based entirely on questionnaires completed by teachers or caregivers. McMahon et al. (2013) justified their research with the argument that such forms of measurement are subject to a variety of biases and that considerable research has shown that observing and coding child behavior is far more objective and produces data with greater validity. Therefore, they asserted that their study, which used strict observational practices and coding techniques, would provide consumers and stakeholders with a less subjective source of data. Utilizing this form of data collection, McMahon et al. (2013) examined the use of a social skills training program in children with autism spectrum disorder with typical cognitive abilities and measured the effect that the program had on social behavior. McMahon et al. (2013) found that participation in a social skills training was associated with positive changes in social behavior, including increased vocalizations and interactions directed toward peers. Participants in this study (n = 14; 9 males and 5 females) were selected based on their family s interest in the study, presence of an ASD diagnosis, and scores on several language/communication scales. Participants were

41 32 grouped into two elementary school-aged groups (ages 10-13) and one adolescent group (ages 13-16) and received the intervention in a clinical setting. In the study, children s social behaviors were observed weekly during a 1.5-hour social skills training activity over the course of 19 weeks. Social skills training sessions included a structured lesson, unstructured playground time, unstructured game time (e.g., board games or card games), and joke telling time. It was during the unstructured game time that researchers observed and coded participants behavior. Nineteen data points were used to measure changes in behavior, which was broken down into verbal behavior and interactions with others. Vocalizations were coded as initiating, responding, or other (e.g, self-talk). Interactions were coded in regards to the interactive partner (e.g., peer, leader, group of peers, or self), or as time spent by self. Results from the McMahon et al. (2013) study indicated that participation in the social skills training program resulted in participants making fewer initiating and other (e.g., self-talk) vocalizations, and more responding vocalizations. Upon completion of the social skills training, researchers noted that participants spent more time interacting with a single peer, more time interacting with a group of peers, and spent less time interacting with a leader. Data showed that greater attendance rates resulted in steeper increases in peer- and group-interaction. Furthermore, the researchers reported that reliability was excellent for most data (i.e., the behavioral coding system which measured interacting behavior including interacting with a peer, with a group, with a leader, and by self). McMahon et al. (2013) found that overall, behavioral coding was a reliable and useful strategy for measuring changes in social behavior of the participants over the

42 33 course of the social skills training. A meta-analysis by White, Keonig, & Scahill (2006) explored the need for social skills interventions for children with ASD and discussed the critical components of successful interventions, as well as effective measurement tools. White et al. (2006) reviewed 14 different studies and examined the direct and indirect consequences of social impairments in adolescents with ASD, and discussed the diverse areas affected by these impairments, including speech, linguistic conventions, and interpersonal interactions. Challenges in social pragmatics (e.g., turn-taking, taking the communication partner s perspective), speech prosody (e.g., inflection and tone of voice), perseveration on certain topics, understanding and expressing emotions, and interpreting nonliteral language such as metaphors, irony, humor, and sarcasm all contribute to difficulties navigating social situations for individuals with ASD. The authors stated that such a varied and complex set of difficulties makes creating a one size fits all type of intervention and corresponding measures of progress nearly impossible, especially when the range of abilities and areas of need in a given group of individuals with ASD and other social learning challenges is considered. White et al. (2006) found that effective intervention programs will include many of the same strategies outlined previously in this chapter (Matson et al., 2007), including fostering self-awareness and self-esteem, making social rules clear and concrete, teaching social response scripts and using modeling/role play, and providing opportunities to practice skills in a variety of settings. In addition, measurement tools of each study were evaluated. Many social skills training programs reviewed in the meta-analysis use a

43 34 standardized scale known as the social skills rating system (SSRS), but White et al. (2006) concluded that studies that utilized this system found no progress among participants on this measure. White et al. (2006) argued that perhaps this measure is not the most appropriate for measuring such nuanced behaviors and social reciprocity that is the true target of many of these interventions. Using behavioral observations, with a check-list-style coding system, might be more effective. Anagnostou et al. (2015) examined a variety of different measures used to chart social communication skills in individuals with autism. Specifically, the authors utilized a panel of experts to conduct a meta-analysis of social communication measures, and selected 37 measures to evaluate. The panel met via conference calls and two face-toface meetings over 14 months, and classified measures as (a) appropriate; (b) appropriate with conditions; (c) potentially appropriate; (d) unproven; or (e) not appropriate. The panel determined that out of the 37 measures reviewed, while none were determined to be appropriate without conditions, six were deemed appropriate with conditions. Anagnostou et al. (2015) found that among a variety of data collection methods (e.g., interviews, questionnaires, and observation) and data collection participants (e.g., clinician-completed, informant-completed, or self-assessed), observation-based systems were evaluated to be among the most appropriate in measuring social communication, especially for young children with ASD. Summary Autism spectrum disorder is increasingly prevalent in our communities and the individuals who navigate our subtle, social world with this neurological difference need

44 35 instruction that is effective and enduring. A wide variety of interventions exist for our students, and one size does not fit all. Students with autism who have significant cognitive challenges and who require more adaptive support may benefit from instruction on specific, concrete social behaviors such as greetings and conversation. Meanwhile, students with ASD and higher cognitive skills may benefit from instruction on more nuanced topics, such as interpreting others beliefs and adjusting their own behavior accordingly (Jones & Carr, 2004). Studies have produced a solid body of research indicating that effective interventions target social skills and social awareness, such as using communication skills (e.g., initiation), interpreting nonverbal communication, interpreting abstract/inferential language and communication (e.g., sarcasm), and taking the perspective of others (e.g., theory of mind) (Bauminger, 2007; Barry et al., 2003; Crooke et al., 2007; Gevers et al., 2006; Kimhi et al., 2014; Koenig et al., 2009; Koning & Magill-Evans, 2001; Koning et al., 2008; Kransy et al., 2003; Matson et al., 2007; McConnell, 2002). Similarly, certain components have found to be most effective in the structure and implementation of interventions. These components include making the abstract concrete, using a cognitive-behavioral approach, providing structure and predictability, providing scaffolded language support, providing multiple and varied learning opportunities, including other -focused activities (e.g., pair work), fostering self-awareness and selfesteem, selecting relevant goals, organizing the program in a logical and sequential order, and providing opportunities for programmed generalization and on-going practice (Kransy et al., 2003; Matson et al., 2007; Winner, 2007). Winner s Social Thinking

45 36 teaching methodology is based in these components, and the related curriculum The Incredible Flexible You, the focus of the remainder of this project, combines these components, best practices, theories, and frameworks (Hendrix et al., 2013). While many researchers utilize pre- and posttest standardized measures to examine participant progress in social skills interventions, this strategy is somewhat flawed in that measuring the nuances of social awareness is not easily assessed through questionnaires (White et al., 2006). For students with ASD who have typical intellectual skills and who require less adaptive support, who are participating in social skills and social thinking training, measuring efficacy can be challenging (Koenig et al., 2009). Observation-based strategies were found to be among the most reliable for measuring outcomes (Anagnostou et al., 2015; White et al., 2006). The Incredible Flexible You offers a series of rubrics for educators to evaluate student progress (Crooke et al., 2007; Winner, 2007). While these rubrics are helpful in that they are designed to describe and measure expectations for different components of lessons, they lack the ability to concretely track behavioral observations, which the research described in this chapter has shown to be critical (White et al., 2006). Moving forward, in utilizing a curriculum that focuses on teaching the why of social interactions, such as The Incredible Flexible You curriculum, creating a system of on-going data collection tools for educators will be essential in measuring student progress. This data could then be applied to the rubrics provided in the curriculum, which will allow educators to measure student progress in a more objective, data-driven manner.

46 37 Chapter 3 METHODOLOGY This chapter covers the methodology and development of the product created for this project. First, the process involved in analyzing the curriculum to determine the concepts and behaviors to be measured by the data collection tools is explained. A brief description of each of the five concepts in The Incredible Flexible You (Vol. 1) is also provided. Next, the methodology of the development of the data collection tools themselves are described. Finally, the development of the progress monitoring system is explained, followed by a description of the development of the tips for goal writing. Analysis of Key Concepts and Identification of Observable Behaviors The Incredible Flexible You (Vol. 1) curriculum centers around five storybooks for early learners, which take five major concepts from Winner s Social Thinking methodology and present them in a preschool-friendly format. The five books feature four characters Evan, Ellie, Jesse, and Molly whose adventures at school, at the farm, in outer space, under the sea, and at the zoo provide a backdrop for instruction and discussion regarding these five major concepts (Hendrix et al., 2013). The first step in the development of this project was the analysis of the five concepts in volume one of The Incredible Flexible You (Hendrix et al., 2013). Specifically, these include: 1. Thinking Thoughts and Feeling Feelings 2. The Group Plan 3. Thinking With Your Eyes 4. Body in the Group

47 38 5. Whole Body Listening. Each of the five concepts is described in greater detail below. The goal for the data collection tools and corresponding progress monitoring system was to have them broken down into five specific units based upon these five concepts which are tied to the curriculum and individual lessons. Each concept was analyzed for the creation of a specific data collection tool that could be used based on observation which has been shown to be an accurate and less subjective method for measuring participant progress and evaluating how effective a particular intervention may be for children with autism (McMahon et al., 2013; Crooke et al., 2007; Anagnostou et al., 2015). Each of the five concepts described below is featured in a picture book for students that accompanies a corresponding section within the educators manual for The Incredible Flexible You (Hendrix et al., 2013). In particular, each concept addressed within the educators manual concludes with a section of Take-Away Points or main concepts and ideas which students should be able to demonstrate after concluding the lesson (the timeline of which will vary depending on the population). These Take-Away Points, in conjunction with the stories themselves, were analyzed to identify key themes and specific behaviors that represent understanding of the specific skills. For example, for Thinking with Your Eyes, the concept that addresses eye gaze, eye contact, and the why behind this particular social skill, analysis of the story book and the corresponding section in the educator manual resulted in identifying the following observable behaviors: (a) reference looking and eye-gaze to a peer or adult; (b) reference looking and thinking to a peer or adult; and (c) reference concept that looking at a person indicates you are

48 39 thinking about them. This analysis process was repeated for each of the five concepts in The Incredible Flexible You (Vol. 1). Thinking Thoughts and Feeling Feelings The main concept taught within this story and associated activities/lessons is the definition of a thought and a feeling. Children are taught that their brain is their thought-maker, that their heart is their feelings keeper, that they have thoughts and feelings about other people, and that others have thoughts and feelings about them. This concept is the foundation upon which all other Social Thinking lessons are taught. In short, the ability to think about others, play collaboratively, and develop friendships involves understanding our own thoughts and feelings and understanding how our behavior affects the thoughts and feelings of others (Hendrix et al., 2013). For this concept, the following observable behaviors were identified as the target skills: (a) recognize and identify feelings in self and others, (b) identify why others may be thinking and feeling a particular way, and (c) recognize that others have thoughts and feelings which are different than our own. The Group Plan The concept taught within this story and associated activities/lessons is the definition of the group plan, which involves everyone thinking about and doing the same thing. Students are taught to identify and follow the group plan in contrast to following their own plan. This is an essential building block in helping children understand what is expected of them when in a group. Following the group plan means thinking about others who are in the group with you, and adjusting your own behavior accordingly.

49 40 Students are taught that following the group plan makes everyone feel calm and comfortable and that following their own plan can make others have uncomfortable thoughts and feelings about them (Hendrix et al., 2013). For this concept, the following observable behaviors were identified as the targeted skills: (a) recognize when he/she is part of a group, (b) recognize that all groups have a plan, (c) identify when someone is following his or her own plan, as opposed to following the group plan, and (d) identify how that might make others feel. Thinking with Your Eyes The concept taught within this story and related lessons is the idea that we use our eyes to gather information, and we also use our eyes to show others what we are thinking about, including when we are thinking about them. Instruction moves beyond the typical Look at me directive often used with children with social learning challenges such as autism. Instead, the term thinking with your eyes is used to support the underlying thinking behind this behavior (Hendrix et al., 2013). Children are taught that the purpose for observing others and the environment is to help them understand what is occurring in their environment, what others might be thinking and feeling, and subsequently, how to respond. Children are also taught that when we look at someone, they know we are thinking about them; in other words, looking equals thinking. For this concept, the following observable behaviors were identified as the targeted skills: (a) follow eye-gaze/joint-attention, (b) identify what someone might be thinking about based on their eye-gaze, and (c) use eye-gaze appropriately (e.g., looking at a communication partner when speaking or being spoken to).

50 41 Body in the Group The concept and skills taught within this story and lesson are all related to maintaining a comfortable physical presence around others (i.e., not too close, not too far away) (Hendrix et al., 2013). Students are taught that when their body is in the group, it sends a message to others that they are interested in the activity and following the group plan. If their body is out of the group, it sends the message that they are not interested and are following their own plan. Students learn that this behavior affects how others think and feel about them. For this concept, the following observable behaviors were identified as the targeted skills: (a) recognize when they are part of a group, (b) recognize that all groups have a plan, (c) identify what it looks like to be part of a group, (d) identify what it looks like to be out of the group. The ability to respond to cues such as Put your body in the group, is essential. Whole Body Listening The final concept of targeted in this curriculum was originally developed by Susanne Poulette Truesdale in the 1990 s, and was adapted for this curriculum (Hendrix et al., 2013). Students are taught that whole body listening is when your eyes, ears, mouth, hands, arms, legs, and feet are calm and quiet. When your whole body is calm and quiet, it helps you pay attention to what others are saying and shows that you care and are thinking about them. Students develop an awareness of how we listen as part of an active process and also begin to understand how the nonverbal messages that we send when we are part of a group might make others think and feel.

51 42 For this concept, the following observable behaviors were identified as the targeted skills: (a) respond to cues such as Listen with your hands, or Listen with your mouth, (b) articulate an understanding of how using whole body listening might make others think and feel, and (c) recognize when it is expected to use whole body listening. Development of Data Collection Tools The second step was to create a data collection tool for each of the five concepts. Specifically, the five data collection tools were developed to measure the observable skills identified as a result of the analysis described in the previous section. Each tool has the skill listed with examples, followed by a table where student behavior is recorded. The skills are divided into verbal behavior and nonverbal behavior as some students may demonstrate understanding of a concept in multiple ways. As noted in Chapter Two, researchers such as Crooke et al. (2007) and McMahon (2013) utilized behavioral coding which discriminates between the two types of behaviors. Student behavior, either verbal or nonverbal, is recorded as independent (denoted by a + symbol), prompted (either gestural, physical, or verbal prompting, denoted by a p ), or absent/incorrect (denoted by a - symbol). This system was selected as it closely matches the data collection format that many special education teachers who work with students with autism may be accustomed to using. On each form, there are 10 boxes where student behavior for a specific skill can be recorded; each box represents an opportunity or occurrence within the classroom. Educators can then record the percentage of opportunities in which the student demonstrated the target skill or behavior over a particular day. This percentage can be used to inform the corresponding rubric and to measure the individual student s

52 43 progress on their mastery sheet (see Appendix A for examples provided within the actual project). Initial drafts of data sheets were piloted in the current author s classroom over a period of two weeks. Data sheets were placed on clipboards and the author, as well as support staff in the classroom, collected data on individual students during non-structured play periods, both inside and outside. The original versions tried to fit too much information on one page (both verbal and nonverbal behavior) and the spaces provided for writing were too small to be functional. The original data sheets also provided too many boxes for individual opportunities which were found to be unnecessary and crowded the space; it was determined that ten boxes were sufficient. Future iterations of the data collection sheets split the two behavior types onto different pages, which were copied onto two-sided paper for ease of use. In other words, data collection of verbal behavior is on one side and data collection of nonverbal behavior is on the other side. The space for notes was also made larger, which made jotting down pertinent information much easier for staff. While behavioral coding of videotapes of participants for data collection is the format used in many studies examining particular interventions, this strategy is impractical for classroom use. As such, the data collection tools developed for this project are designed to give educators an easy system to track student progress within The Incredible Flexible You (Vol. 1) curriculum, throughout their day and within opportunities for generalization.

53 44 Progress Monitoring System The data sheets created for this project each correspond to the rubrics provided within The Incredible Flexible You (Vol. 1) curriculum. Within the teacher s manual, each concept comes with a rubric intended for measuring individual student progress which asks the teacher to measure the child s knowledge of the concept on a 0-5 scale, with 0 indicating No understanding of the concept. They are not using the vocabulary or demonstrating any of the requisite associated behavior, and a 5 indicating Solid understanding of the concept and can demonstrate its use with minimal cues (Hendrix et al., 2013, p. 62). The rubrics are identical for each of the five concepts. For this project, the rubrics were modified slightly to create a progress monitoring system. Specifically, the data collected from the individual student s performance in the classroom utilizing the newly developed data collection forms was used to calculate a specific level of mastery for the concept (i.e., percent accuracy of performance). The rubric forms were modified with the addition of a column in which a teacher can indicate where a child might be rated in terms of accuracy of performance for a particular time period. The percent accuracy of performance were assigned to the rubric numbers based on the description provided in the manual. A child with a current level of mastery of 0% accuracy corresponds to a rubric level of 0, which indicates that he/she has No understanding of this concept. They are not using the vocabulary or demonstrating any of the requisite associated behaviors. A rubric level of 0 prompts the teacher to consider the appropriateness of this curriculum for the student (Hendrix et al., 2013, p. 39). A child with a current level of mastery of 1-20% accuracy corresponds to a rubric

54 45 level of 1, which indicates that he/she has Emerging awareness of the concept. May be able to point out or give examples of the use or misuse on others but are not demonstrating its use even with maximum support and cueing. A child with a current level of mastery of 20-30% accuracy corresponds to a rubric level of 2, which indicates that he/she has Emerging awareness of the concept. May be able to point out or give examples of its use or misuse on others and are demonstrating its use with support and cueing. Both levels 1 and 2 prompt the teacher to consider spending more time teaching this concept before moving on to the next lesson; explore it in different ways and across settings (Hendrix et al., 2013, p. 39). A child with a current mastery level of 30-60% accuracy corresponds to a rubric level of 3, which indicates that he/she has a Solid understanding of the concept and can demonstrate its use with moderate cues. A child with a current mastery level of 60-70% corresponds to a rubric level of 4, which indicates that he/she has a solid understanding of the concept and may be starting to use it with minimal cueing. Finally, a child with a current mastery level of % accuracy corresponds to a rubric level of 5, which indicates that he/she has a Solid understanding of the concept and can demonstrate its use with minimal cues. Rubric levels of 3-5 prompt the teacher to consider continuing to use the concept in context and teachable moments as you move forward in teaching the next lesson or deepening knowledge on this one (Hendrix et al., 2013, p. 39). As part of the progress monitoring system, a mastery sheet was also developed to provide educators with an easy at-a-glance method to record and track student progress over time. This sheet was developed by listing the five concepts across the top, in the

55 46 order they are intended to be taught (i.e., Thinking Thoughts and Feeling Feelings, The Group Plan, Thinking With Your Eyes, Body in the Group, and Whole Body Listening). Down the left-hand side is a column in which to record the date. The corresponding row holds five boxes, one for each concept in which the educator will record the particular student s level of mastery at the time. For example, data gathered on a new student might translate into about 20% accuracy for the concept of Whole Body Listening which corresponds to a 2 on the rubric or Emerging awareness of the concept. The educator would record this percentage of accuracy on the mastery sheet. The student might have similar levels of mastery/percentage of accuracy for the remaining four concepts. Two months later, analysis of new data on this particular student might indicate about 65% accuracy for the concept of Whole Body Listening, or a 4 on the rubric or Solid understanding of the concept and may be beginning to use it with minimal cueing. A student s progress over time can be recorded on this mastery sheet which will in turn help educators in IEP goal and objective development. For example, a special education preschool teacher seeking to create a goal on Whole Body Listening for a particular student who seems to struggle with attending during small group activities might refer to the mastery sheet to inform his or her baseline data. A child who is currently demonstrating a mastery level of about 6% accuracy (level 1) in September could have a goal that he or she will Respond to cues to listen with your hands or other body part with 75% accuracy by April. Specific guidance on how to use the mastery sheet in combination with the data collection sheets and progress monitoring rubrics is included with examples in the product itself.

56 47 Goal- and Objective-Writing Tips The authors of The Incredible Flexible You recognize that social thinking and social learning are complex, nuanced, and multifaceted, and therefore, may not lend themselves perfectly to the performance-based data collection model used in discrete trial training or other trial-based instruction to develop specific goals for students (Hendrix et al., 2013). Instead, they recommend using the rubrics provided in the educator s manual to inform goal writing. The Ideas for Goal Writing section of the curriculum s educator manual provides teachers with sample goals for each of the concepts, with several examples for different target behaviors, and each based on a different level of understanding drawn from the rubric (Hendrix et al., 2013, p. 183). This guide for goal and objective writing is a useful tool, albeit difficult to use with the rubrics alone. The current project, however, attempts to provide educators with one method to gather data objectively as well as to determine and evaluate progress in a more data-driven manner, both to inform them about their students current level of performance and to support the development and implementation of IEP goals and objectives. As such, when educators are utilizing the Ideas for Goal Writing section of The Incredible Flexible You (Vol. 1) curriculum to create goals and objectives for their students, it is the intent of this project that they will actually use the data collected on specific students from the tools provided to create goals and objectives that are more easily and objectively measured and are uniquely tailored to the individual students. Specific guidance on how to use the tools provided to write goals is also included with examples in the product itself.

57 48 Chapter 4 DESCRIPTION OF PROJECT, DISCUSSION, AND RECOMMENDATIONS Interventions for children with autism spectrum disorder (ASD) or other social learning challenges are developed to address a wide range of student needs, including differences in communication skills, adaptive behavior, sensory processing, socialcognitive skills, social pragmatics, theory of mind, executive functioning, and selfregulation (Jones & Carr, 2004). For children with ASD who have significant support needs, interventions that focus on fundamental skills such as functional communication, imitation, and basic social skills (e.g., greetings and routines) have been found to be effective (White, Keonig, & Scahill, 2006). For children with ASD or other social learning challenges who have fewer support needs and typical cognitive skills, instruction and teaching on the more nuanced, complex facets of social communication is often necessary. The Social Thinking methodology, a curriculum and teaching framework developed by Michelle Garcia Winner, helps bridge the gap between teaching students behaviorally-based, memorized rote social skills and teaching them to read and interpret social situations and the why behind them (Crooke, Hendrix, & Rachman, 2007; Winner, 2007). The Incredible Flexible You (Vol. 1) curriculum is part of the Social Thinking methodology developed for preschoolers and early elementary students with social learning challenges (Hendrix et al., 2013). This curriculum includes storybooks and an educator s guide for implementing lessons and has rubrics embedded to track student progress. However, the rubrics are generic and subjective making accurate measurement and evaluation of student growth a challenge. This project attempts to

58 49 address this need for observation-based, objective methods for monitoring student progress within this nuanced, complex curriculum. Description of the Project This project includes the development of a set of data collection tools and a progress monitoring system for educators to use in the classrooms of young children with autism spectrum disorder and other social learning challenges. Specifically, this project provides teachers with an objective system to measure progress by students in demonstrating specific skills and concepts taught within a social learning curriculum, The Incredible Flexible You (Vol. 1) curriculum, a component of Winner s Social Thinking methodology for preschoolers and early elementary school students. The data collection component involves a series of observation-based tools for classroom staff to use during structured or unstructured play periods, in order to gather information regarding students mastery and generalization of skills taught during small group lessons. These tools inform the use of a progress monitoring system, which includes a modified version of the student ability rubrics that are included in the educator manual which accompanies The Incredible Flexible You. A mastery sheet for tracking student progress over time is also part of the progress monitoring system. The progress monitoring system was developed for this project to provide educators implementing the curriculum with a straightforward set of tools to record students levels of mastery/percent accuracy of performance for specific concepts and skills and to transfer this data into long-term mastery sheet to assist in data-based decision making and goal development. Tips and strategies for using the information from the data collection and

59 50 progress monitoring system in conjunction with the Individualized Education Plan (IEP) goal and objectives writing section of The Incredible Flexible You educators manual are also included. Discussion This data collection and progress monitoring system was field-tested in two special education preschool classrooms serving young children with social learning challenges, which are part of a large, urban and suburban school district. Staff (i.e., special educators and paraeducators) using the data collection system were previously trained on how to take observation-based data and were familiar with the general format of a data collection system. The two preschool classrooms utilizing The Incredible Flexible You curriculum had been using this methodology with students for over two years and were familiar with the concepts and materials. Other educators utilizing this data collection and progress monitoring system in conjunction with The Incredible Flexible You might face more challenges in application if untrained in the curriculum, in data collection, or in progress monitoring in general. Ultimately, The Incredible Flexible You will be a two-part curriculum. This project addresses only volume one which covers the first five concepts: Thinking Thoughts and Feeling Feelings, The Group Plan, Thinking With Your Eyes, Body in the Group, and Whole Body Listening. Volume two, which covers the next five concepts: Hidden Rules and Expected/Unexpected Behaviors, Smart Guess, Flexible and Stuck Thinking, Size of the Problem, and Sharing an Imagination is currently in development but not yet available for educators. As such, additional data collection tools and an

60 51 updated progress monitoring system will need to be developed for Volume two of the curriculum. Another limitation of this project is that this data collection and progress monitoring system was developed for one specific curriculum, The Incredible Flexible You. While the skills measured are foundational and could be applicable within other social learning curricula, educators utilizing other interventions/curricula for young children with social learning challenges may need to modify these tools to accommodate these other interventions/curricula. Finally, the data collection tools and progress monitoring system are entirely observation-based and intended for educators to use within the school setting. While the benefits of observation-based data collection are clear in regards to reporting on progress toward specific goals, relying only on this performance-based method of monitoring progress omits other important sources of information, such as parent input. Therefore, these tools should be part of a comprehensive data collection and progress monitoring system. In addition, the type of flexible, situation-based thinking and responding this curriculum seeks to teach students is not easily measured in such concrete narrow parameters. Quantifying behavior as a means of measuring generalization of a nuanced thinking and understanding has its limitations, and should be considered as one part of a greater picture of student ability, growth, and learning.

61 52 Recommendations for Future Practice and Research This project has implications for future research regarding not only Social Thinking methodology and related curricula such as The Incredible Flexible You, but also any other interventions targeting students with social learning challenges. Information gathered regarding participant growth and progress could be utilized in future studies, and the data collection tools and progress monitoring system developed in this project could be modified for use in studies which seek to examine the efficacy of certain interventions, such as The Incredible Flexible You. However, in order to utilize these tools to evaluate the efficacy of a program or curriculum, the data collection tools and progress monitoring system developed for this project need greater pilot testing in the field, both at the current author s school site and other schools throughout the school district and beyond, in order to determine their validity and reliability. The need arises for more research overall. In regards to The Incredible Flexible You and its use in the school setting, future research projects might focus on the efficacy of the curriculum when implemented with fidelity. Other research studies might examine the use of different styles of data collection and progress monitoring to evaluate fidelity and reliability of specific measurement tools. This work could result in being included in future iterations of the curriculum for all educators to use. Those interested in long-term impact of social skills interventions could design longitudinal studies that examine various outcomes of children with social learning differences who received The Incredible Flexible You during their early years as compared to children who did not,

62 53 or research the overall social outcomes of students who participated in the curriculum in preschool. Finally, a recommendation for future practice is additional support to educators and support staff in the form of training. While the project is written to be clear and easy to use, an initial orientation for all staff utilizing the data collection tools and progress monitoring system will ensure that the tools are being applied in a consistent manner. Subsequent refreshers should also be offered periodically, especially in the first part of the school year, as staff will likely have questions after the initial use of the tools in the classroom.

63 54 APPENDIX A Tools for Teachers: A Progress Monitoring System to be Utilized by Educators Implementing The Incredible Flexible You (Vol. 1)

64 55 Tools for Teachers: A Progress Monitoring System to be Utilized by Educators Implementing The Incredible Flexible You (Vol. 1) By Hayley Kercher Sacramento, CA 2016