Model Sites Project with the National Professional Development Center on Autism Spectrum Disorders: Final Report and Recommendations History and Details The Texas Education Agency approached the (TSLAT) with an opportunity to consider application for inclusion in a national project for the school year (SY) 2009-10. The TSLAT agreed that the proposal would support autism training in Texas and applied for the required funding for the project from the Texas Education Agency and also applied to the National Professional Development Center on Autism Spectrum Disorders (NPDC) for inclusion in the second of four cohorts. Some of the required elements in the application included a plan of action to implement the project and a commitment for funding for the project. The NPDC agreed to provide some onsite support and training, but the bulk of the funding was to come from within the target state. This program became part of the plan for the TSLAT. The application was not accepted by the NPDC and the TSLAT was left with a plan that would not be supported by the NPDC. Initial planning proceeded with the development of a process to choose the model sites, assembly of an Interagency Planning Group (IAPG) and development of a two-year plan designed to create model sites for Evidence-Based Practices in classrooms in Texas. In addition, it was determined that the TSLAT would submit a second application to be included in the third cohort that would begin in SY 2019-11. The IAPG was assembled in order to provide input to the TSLAT for development of the plan, application, and project. The IAPG was to monitor the project, provide input as needed, and review the final report prior to publication. The plan was developed and application was submitted. Applications were published in August 2010 and the plan called for the development of 9 model sites across Texas. Just prior to the application deadline, NPDC contacted the TSLAT and announced that Texas would be one of three states included in cohort 3 and that our direct partner in the project would come from the Waisman Center at the University of Wisconsin, Madison. At the close of the application process, there were only seven applicants and all were accepted into the Texas Model Sites for Evidence Based Practices (EBP). At the same time, resources previously available only to the state partners were published by the NPDC for public consumption. August 2012-1
Some important elements of the Texas project included: Teams of six individuals who would meet regularly to coordinate each model site. The team consisted of a parent, the special education teacher of record, the local Education Service Center (ESC) representative, a campus administrator capable of providing local resources, a supervisor/autism specialist or other educator involved in the classroom, and an agency representative or educator from another district campus. This last position was to be staffed as follows: o If the target classroom was Preschool Program for Children with Disabilities (PPCD) a local Early Childhood Intervention (ECI) caseworker that supervised students 0-3 years old in the community must be on the team to facilitate transition from ECI to PPCD. o If the target classroom was a secondary campus, a local adult services caseworker was required to sit on the team. o If the target classroom was an elementary classroom, an educator from a campus either sending students to the target classroom or receiving students from the target classroom would be the sixth member of the team. ESC autism contacts would support model site classrooms within their region. If a site was designated as NPDC classroom, direct support from NPDC would be provided. If the site was not designated as an official site, the local ESC would provide the direct supervision. It was anticipated that each model site would need the services of an expert in autism spectrum disorders (ASD) to assure that fidelity of implementation was a strong factor in the site. Supplemental funding would be provided to offset any costs associated with training and implementation at the district level. This supplemental funding would be provided by the Texas Education Agency. Data points for the project would include scores from a classroom assessment, the Autism Program Evaluation Rating Scale (APERS) for a pre- and post-assessment, scores from IEPs for the targeted students on a 5-point scale developed by the NPDC for comparing IEP goals and progress in the implementation of steps for up to three evidence-based practices that the team chose to implement in the targeted classroom. Each team member was required to access an on line course at the University of North Carolina, Chapel Hill prior to the summer training. Access to this course was through the university only. Each ESC autism contact was to be trained to conduct the APERS assessment and would provide that service at no charge to the model site. August 2012-2
Each site would submit a proposal to the Texas State Autism Conference (TSAC) for consideration to present at the 2010 and 2011 conferences. Each team was to receive two days of assessment and orientation and five days of training in the summer of 2010 related to EBPs and the project expectations. Each team in Texas was to provide video examples of EBPs as implemented in the classroom along with examples of tools, processes, or procedures that were implemented as a result of the project. These will be posted at the end of the project on the TSLAT website. Each team was expected to provide a classroom for two years and in the second year expand the project into one other classroom in the region. The project would not publish the district or campus information for the model site. These sites would be designated by ESC only. For example, a Model Site Classroom in Austin High School in Region 13 would be designated a secondary classroom in Region 13. No reference would be made in reports that would designate a district or campus information. This was in response to the concern that the designation might cause families to consider transfer or moving. This project was designed to show that every classroom can become a model classroom. Preference would be given to smaller districts and areas of Texas outside the larger urban areas. Region 6, Region 13, and Region 19 were selected to be NPDC model sites with support from the NPDC staff from the Waisman Center at the University of Wisconsin, Madison. Dr. Linda Tuchman-Ginsberg, Ellen Fanzone, and Katherine Szidon were assigned to the Texas model sites and worked with TSLAT for three years including training for two summer training sessions, APERS training for all autism contacts, and multiple technical assistance visits and phone calls. Information from the three ESC programs was used to support the national program. The first 7 applicants were from Region 1 in Edinburg, Region 6 in Huntsville, Region 9 in Wichita Falls, Region 13 in Austin, Region 16 in Amarillo, Region 17 in Lubbock, and Region 19 in El Paso. All were approved as model sites. The National Professional Development Center on Autism chose Region 6, Region 13, and Region 19 as the official Model Sites and created a timeline for visits, assessments, trainings, and support. Plans were made for support for the other regions. Seven teams met in Austin in November 2009 for two days of training. NPDC staff attended to meet teams and begin the process. Teams were given skills assessments and devised an action plan for training for the team and individual team members. Resources were shared from the NPDC and other sources. In addition, each team was provided with a set of materials designed August 2012-3
to support the 24 EBPs identified by the National Professional Development Center on ASD. These resources can be found at http://autismpdc.fpg.unc.edu/ The Project Year 1 As a result of the meeting and project publication, other ESCs expressed an interest in the project and it was determined that the TSLAT would reopen the application process to offer other regions to participate. Three additional ESCs applied and it was determined that with some budget modifications, TSLAT could support all three new teams. This provided half of the state with a designated model site for the implementation of evidence-based practices within their region. The additional regions included Region 3 in Victoria, Region 5 in Beaumont, and Region 12 in Waco. NPDC staff scheduled APERS assessments in Region 6, Region 13, and Region 19. Other ESCs were offered the opportunity to attend the training in one of the sites. All 10 ESC autism contacts attended the training and were given permission to conduct APERS assessments in their region for the project. YEAR 2 The first year of actual implementation was SY 2010-11. Teams were asked conduct an APERS assessment, create Goal Attainment Scaling (GAS) (see attachment) for up to three students in each classroom, determine targeted EBPs (see checklists at NPDC website) and schedule monthly updates for each targeted EBP, assign a coach for regular classroom visits ( see attached Coaching Log), plan any training indicated by the action plan that was not already provided, create a proposal for presentation at the TSAC, create and submit a budget for additional district/campus costs associated with the project, schedule team meetings, and plan for video examples and collection of tools, processes and/or procedures that would be provided to TSLAT for examples. During the year, a coach visited with documentation at least once every two months. Another individual would drop in and document steps being implemented in each of the targeted EBPs monthly, update GAS for each grading period, provide support directly to the teacher, document contact by the ESC autism contact or other supervisors visiting in relation to the project, collect and maintain data on IEPs targeted for the GAS, secure and attend any training needed, develop presentation for the TSAC, and attend the TSAC conference. In the spring, this team would designate an expansion site and create an expansion team with the same guidelines applied to the original team with these exceptions: 1. No two day orientation was provided to these expansion sites August 2012-4
2. The expansion sites would work with existing funding (no budget from TEA and no additional supplies from the project), 3. The mandatory 5 days of training was provided in this way. Expansion teams attended 2.5 days of training in Austin in the summer. One day of specific training was provided specifically for these teams during the first day of the Texas Conference on Autism. The remainder was provided through their attendance at session during the TSAC. In the spring of SY 2010-11, the team would schedule and conduct APERS post-assessment for the initial team and APERS pre-assessment for the expansion team. This information along with coaching logs, GAS updates, and EBP documentation has been requested by TSLAT staff. YEAR 3 In SY 2011-12, each ESC was asked to run two model sites. During the year, the expectation was to complete the necessary elements detailed for the Year 2 teams for both the original and expansion classroom. Both teams were required to attend the TSAC in Houston and the expansion teams would attend a one-day training provided by TSLAT staff and NPDC staff on the first day of the conference. The next day and a half was spent securing training in the various EBP identified by NPDC. Initial teams were told they may present at the conference highlighting what was learned in the Model Site classroom project. In the spring, each ESC was charged with providing documentation of data points, video examples, and tools; processes; or procedures. In addition, all ESCs were expected to create a plan to begin to provide some type of Evidence- Based Practice Project in their region. Current participating ESCs were asked to document how they plan to expand, new ESCs were asked to provide plans for their EBP projects. All ESC contacts were provided the opportunity to secure APERS training in Texas during the spring of SY 2011-12. ESCs also have a copy of the APDAR, the Autism Rubric developed by Region 20, and the Quality Indicators for Life skills Teachers (QuILT) developed by Dr. Amanda Boutot for classroom assessment. ESCs should be using some type of coaching support, the EBP materials from NPDC, and some kind of IEP documentation such as the GAS. ESCs will report efforts in the Network Implementation Plan provided to TEA each summer. August 2012-5
Results Significant growth in the model sites programs was noted. Parents reported the best part of the program was participating in the GAS process during summer training. The opportunity to sit in on the IEP development process was specifically useful from the parent perspective. Educators all seemed to appreciate the tools provided by the NPDC and plan to incorporate them into programs after the end of the process. APERS Classroom assessment: For the Texas project, we saw an average increase in the first year 2010-11 of +.83. The second year, 2011-12, for this group the averages were higher, +1.19, with all scores reported as above average by the end of the year. The first year of the expansion saw an average increase of +.94. All reporting classrooms showed significant increases in the number of steps for identified checklists over the course of the year. Not all classrooms attained 80% fidelity of implementation during the first year of the project, but all made significant improvements. The average percent of implemented steps over all reporting classes was 86% with scores ranging from 69-100%. 75% of all classes met the 80% fidelity point. GAS scores were higher on average, but some students still failed to meet identified goals using the GAS measures. There is no feedback regarding actions taken to deal with those students. The average for the GAS scores for reporting classes was.36 with a score of 0 being on target. The scores ranged the full possible scale of 0-5. Public school is not a stable educational environment and not necessarily the best place to implement rigorous experimental investigation with strict guidelines. There is no way to control all variables needed to conduct research at the highest levels. Some initial classrooms changed teachers three times during the school year. Some classrooms disbanded in the second year due to students moving to another district or students moving to a different campus during the year. One region chose not to participate in the second year expansion process. Since this was a NPDC model site, Region 13 provided an additional classroom for the expansion data for the NPDC project. Coaches were lead teachers, autism specialists with districts, and ESC autism contacts; they seemed to be an important element in the success of the Texas project. While there were no data points associated with the coaches, it would be nice to be able to quantify the impact on the implementation of EBP in the classroom. It is suggested that a self-evaluation of personal skills associated with targeted EBP be provided as a baseline before coaching/training during the year. In addition, the educator s skills in the same EBP could be collected at the end of the year and a comparison made between the two to estimate increase in skills. We do have the changes in the EBPs documented but that is just one of many elements that determine individual skills. August 2012-6
Recommendations and Comments First, the resources provided at no charge allow a public school to prove fidelity. Creating a task analysis of the process as written in the literature was a great idea and the individuals working on the project at the NPDC should be commended for their efforts. Many of the resources depend upon the development of a good Functional Behavior Analysis (FBA). This one pivotal element should be a mandatory element for every program serving students with ASD. While sensory processing is not specifically addressed in the identified EBPs through the NPDC, may be considered an antecedent-based intervention by some. If an FBA team includes an occupational therapist (OT), sensory needs may be identified as one factor in the behavior intervention plan (BIP). The coaching element is also a vital part of the program yet there are no data points associated with the service. All ESC contacts have been offered coaching training and many ESCs offer coaching training in a comprehensive package available to all educators. Using the content knowledge of an autism specialist in combination with developing coaching skills should be an important part of training for all district autism specialists. It is recommended that there should be consideration regarding a process that could standardize the coaching element in future projects. This could allow evaluation of the value of this part of the project. Texas Statewide Leadership in Autism Training may want to consider this as a piece of the statewide plan in the future. NPDC is offering an Autism 101 Course (a requirement for all team members) to each state in a cohort. This course can be posted anywhere. Since the TSLAT already has a course with the same title, it is recommended that the course be given to a university program in the state of Texas with the requirement that it be available to the general public at no cost. I also recommend that the 20 autism contacts in the ESCs determine the university program that will receive the course. Each ESC has been given four classroom assessment options. Each ESC has been offered training to allow the autism contact the option to conduct APERS assessments in their region. It is recommended that the State Leadership for Autism Training continue to collaborate with the author of the APERS, Dr. Sam Odom at the University of North Carolina Chapel Hill, to supply information about the use of APERS in Texas. This tool is not in general release; Texas has been given permission to use the APERS after the administering professional has been provided training. This position in some ESCs tends to have a rather significant turnover. The TSLAT should consider the option of continuing to provide APERS training to new ESC contacts. While it is recommended that future teams utilize the APERS, it is important to note that other tools are available to evaluate classrooms in relation to best practices. If a team chooses to use a different assessment tool, it is recommended that they tie the results to the EBP August 2012-7
resources provided by the NPDC. This is not an option for any of the other assessments, but could be easily created. As a part of the GAS training, new teams built around the process set up by the NPDC should consider taking the Standards Based IEP Development training offered at Region 20 ESC through the statewide project for Access to the General Curriculum. Information about this training can be accessed at www.esc20.net/agcnetwork. One other element that was gained from the NPDC resulted from attendance at the annual OCALI exposition held in November of each year. We were invited to participate in the National Autism Leadership Summit set up by this group in Ohio. In 2011, the group consisted of individuals from 28 states and 2 Canadian Provinces. This group determined to continue work in four areas by meeting virtually over the year. This organization could prove to be a very effective tool to foster collaboration and resource sharing at the state and national level. It is recommended that this effort continue as a statewide project with information shared with the Texas Education Agency and thereby to the Texas Council on Autism and Other Developmental Disabilities. August 2012-8
Appendix 1: Goal Attainment Scaling: From the National Professional Development Center on ASD, June 8, 2009 PROCEDURE AND ILLUSTRATIONS Goal attainment scaling (GAS) is a system for assessing progress children and youth with Autism Spectrum Disorders (ASD) make on individual goals across a specified time period. These goals are drawn from Individualized Education Programs (IEPs) or Individualized Family Service Plans (IFSPs) and established by teachers, related services professionals, family members, and in some cases the individual with ASD. In this process, staff from the National Professional Development Center on ASD (NPDC-ASD) collaborate with teachers/practitioners to identify three IEP/IFSP goals that have the highest priority for the individual with ASD and establish a five-point scale to measure child/student progress. An illustration of the five-point scale is found in the table below. Table 1. Example of Goal Attainment Scale Level Goal 1: Of Uses questions Attainment Much less than expected -2 Somewhat less than expected -1 Expected level of outcome 0 When given a model and prompted, will ask questions of adults occasionally. When prompted, will ask questions of adults during 50% of the opportunities presented. Independently asks questions to obtain information from adult in classroom during 80% of opportunities presented. Goal 2: Completes work assigned independently Requires teacher prompts on at least 75% of the tasks assigned for independent work time. Completes 50% of work assigned during specified work time with appropriate supports. Independently completes work assigned during specified independent work time with appropriate supports for 80% of work sessions. Goal 3: Engages in playful social interaction with peers during play period Primarily is alone, unengaged, and inattentive to peers during recess. Plays alone in close proximity to peers and watches their play for 50% of play period during recess. During recess, plays with at least one peer for 30% of the play period. August 2012-9
Somewhat more than expected +1 Much more than expected +2 Comments From: Cardillo & Choate, 1994 Independently uses questions with at least two different adults in classroom during 80% of opportunities presented. Independently asks questions of adults in classroom and in at least one other context during 100% of opportunities. Independently completes work and puts away materials after work task is completed for 80% of work sessions. Independently completes tasks, puts materials away, and tells teacher he/she is done for 100% of work sessions. During recess, routinely plays with at least one peer for at least 50% of the play session. During recess, routinely plays with one peer for at least 70% of the play session. Procedure for Developing a Goal Attainment Scale for Children and Youth with ASD 1. Select three IEP/IFSP goals that have high priority for the child/student with ASD. These goals should be scaleable, in that a continuum of outcomes is identifiable. An example of a scaleable goal is: Participant uses a question to obtain information in different contexts. Dichotomous goals which are answered as either yes or no should not be used. An example of a dichotomous goal is: Learner makes an appointment with a vocational rehabilitation counselor. 2. Assign each goal an abbreviated title and in accompanying documentation, cross reference the actual goal from the IEP or IFSP. For example, a longer behavioral objective such as, In the classroom, the learner will independently ask questions to obtain information 8 out of 10 opportunities, could be modified to the title, Uses questions. 3. Specify the levels of attainment according to the numbering on the scale, which ranges from -2 to +2 with 0 being the expected outcome of the goal. The GAS will be completed in the fall and spring of the academic year. Specific information related to the scoring of each goal is described below. a. The goal from the IEP or IFSP is the expected outcome for the objective which appears at the middle or 0 point on the continuum of outcomes. In the example above, the 0 on the continuum is that the learner asks questions 80% of the time or 8 out of 10 opportunities. The baseline or initial functioning level of the learner for a particular goal could be the -2 designation (much less than expected) on the continuum of Goal Attainment Guide. b. Progress that is slightly below or slightly above the expected outcomes should be specified as -1 (somewhat less than expected) or +1 (somewhat more than August 2012-10
expected). In the example above, the -1 on the continuum might be that the learner asks questions only 6 out of 10 opportunities. The + 1 on the continuum might be that the learner asks questions 8 out of 10 opportunities. c. Progress that is much less or much more than expected should be designated for -2 (much less than expected) or +2 (much more than expected). In our example, the +2 on the continuum might be that the learner asks questions every time the opportunity arises or 10 out of 10 times. 4. Each of these scaling steps should be completed for the three IFSP/IEP goals. To complete this process, NPDC-ASD staff will meet with teachers/practitioners to prioritize goals and explain the need to predict a continuum of outcomes for individual children and youth with ASD. 5. Goal Attainment Scaling will be completed by NPDC staff and teachers/practitioners in the fall and again in the spring to assess student/child progress on expected outcomes for the three selected goals. References Cardillo, J. E., & Choate, R. O. (1994). Illustrations of goal setting. In T. Kiresuk, A. Smith, & J. Cardillo, (Eds.). Goal attainment scaling: Applications, theory, and measurement (pp. 15-37). Hilldale, NJ: Lawrence Erlbaum Associates. August 2012-11
Goal Attainment Worksheet School: State: Date: Student Initials: Student Date of Birth: Level Of Attainment Goal 1: Much less than expected -2 Somewhat less than expected. -1 Expected level of outcome 0 Somewhat more than expected +1 Much more than expected +2 Comments From National Professional Development center on Autism Spectrum Disorders August 2012-12
Appendix 2: Coaching Log Example August 2012-13
August 2012-14