Using GAINS to Overcome Problems in ABA Delivery

Transcription

1 Using GAINS to Overcome Problems in ABA Delivery John T. Nosek, Ph.D Abstract ABA is the gold standard in treating autism spectrum disorder (ASD); however there are many problems in delivering ABA, from unavailability in many places, to skill gaps and variability in instructor experience which leads to therapy inconsistencies and loss of program fidelity. With support from the National Science Foundation (NSF), a software platform named GAINS (Guidance, Assessment and Information Systems) has been developed that uniquely incorporates artificial intelligence to overcome problems in delivering ABA. The heart of GAINS is expert agent software that incorporates knowledge of ABA practice and curricula customized for the individual with developmental challenges to provide real-time process and decision support. For instructors and family members who need it, GAINS provides detailed, step-by-step guidance that adapts to student responses on the fly. In this way, expert guiding agents reduce a priori training requirements in ABA while providing real-time apprenticeship coaching like a training therapist to overcome variability in instructor experience and improve program fidelity. After two years of intensive field testing it has been demonstrated that the technology works, children learn, and instructors find it valuable. GAINS reduces upfront training needs while providing training as a by-product of using it, improves workflow, and redistributes work, time and resources away from non-instructional to instructional effort. Value of Applied Behavior Analysis (ABA) Applied Behavior Analysis (ABA) is the gold standard in treating autism spectrum disorder (ASD). 1,2 Early Intensive Behavioral Intervention (EIBI) remains the most effective and scientifically-validated approach to remediate the deficits caused by ASD; 3,4,5 for example, up to 50 percent of young children with ASD who receive early, quality intensive ABA (40+hours/week) improve their IQs and mainstream into general education classrooms with limited need for supports. 6,7,8 Effective ABA instruction can be demanding for both instructors and students. Student readiness for instruction and instructor availability do not always coincide. 9 Ideally, a team of instructors consisting of behavior analysts, less-skilled therapists/instructors, educational assistants, and family members provides coordinated, consistent ABA instruction throughout the day, every day, in formal and natural settings. ABA therapy is extraordinarily data intensive, and consistent, well-coordinated instruction improves student outcomes. 10 Program fidelity is a critical component of EIBI. 11 Reed and colleagues have shown parametric declines in student progress associated with decreasing fidelity; 12 and Strain and Bovey found that program fidelity correlates significantly with better outcomes (r =.67 to.86). 13 However, ASD service providers struggle to maintain program fidelity in ABA therapy across instructors with varying experience. Problems in ABA Delivery There are problems in delivering ABA: effective treatment is unavailable in many places; instructor turnover is high; new instructors and family members experience skill gaps; complexity of ABA and variability in instructor experience leads to therapy inconsistencies and loss of program fidelity, which negatively affect treatment outcomes; student information is kept in binders or apps that mimic binders, making it hard for therapists to know what skills to work on; onerous data collection steals attention from the child; and creating required reports and charts for clinical evaluation, parents, and funding agencies, which include insurance companies, school systems, and government agencies, is a time sink that steals time away from instructional activities.

2 Transforming ABA GAINS (Guidance, Assessment and Information System) goes beyond recording data and generating reports. Beyond behavior recording, the heart of GAINS is expert agent software that incorporates knowledge of ABA to provide real-time process and decision support, which is updated on the fly based on student performance during instruction. A software agent is a persistent, goal-oriented computer program that reacts to its environment and acts for a user to decide which, if any, action is appropriate. GAINS software agents are expert guiding agents because they incorporate knowledge of ABA and use this in deciding actions to take on behalf of the instructor during a therapy session, which includes guidance on what the instructor should say and do next. In GAINS the expert guiding agents enable simplified data collection and automated reporting and charting and interact with the instructional state component to guide the instructor in what to do next. Currently GAINS expert guiding agents support two Discrete Trial Training (DTT) protocols, three Task Analysis (TA) protocols and one incidental teaching protocol. [See Figure 1]. The human ABA Program Developer translates behavioral theory and practice into available GAINS therapy programs and guidance. Outside of GAINS, the supervisor assesses the student s developmental delays and identifies target ABA therapy programs and behaviors. Using available therapy programs in GAINS, a path of instructional steps, called the Path to Mastery, is created for the student. Using the Path to Mastery, the GAINS instructor s assistant app: 1) provides guidance in selecting the correct therapy program; 2) provides guidance during a therapy session that is updated in real-time based on student performance; and 3) captures detailed and accurate data about student performance and instructor/student interaction. ABA Program Developer (ABA Programs in GAINS are Dynamic, Modifiable) Supervisory Work Assessment Program Selection Guidance GAINS automatically provides correct active Programs for selection Available Programs instructor s assistant app constructing instructional path Assigned Programs Session Guidance GAINS provides real-time guidance updated based on student performance Individual Path to Mastery Data Analysis (includes data mining) Data Collection Figure 1. GAINS Functionality Overview to Support Roles in ABA Currently, without GAINS, instructors need to manually check the student s training binder for supervisory requirements and look for previous trials completed by whom, at what locations and on what days to determine what to target during a session. Instead, GAINS eliminates the burden, a priori training requirements, and selection errors, while saving valuable instructional time by providing what should be targeted and when was the last time a student worked on a behavior. GAINS does this by using a) supervisory requirements for student mastery of a behavior in the Path to Mastery that identifies the required number of different instructors, days and locations, b) student performance data, and c) the current instructor, day and location. The figure below is a screen shot from the GAINS App. It is showing the target steps at the correct prompt level within the programs the instructor should work on next. The order of the targets is not apparent in the figure, but it is prioritized based on the background expert system.

3 During an ABA session, the instructor must be aware of many requirements to provide quality, consistent, efficient ABA therapy. Although too much to fully describe here, examples of requirements an instructor must consider during a session are: a) whether the current trials in a session are baseline, prompted, independent, or task analysis (chaining), b) what determines success (4/5; 8/10), whether success can be achieved this trial set, when the goal has been achieved and when the maximum attempts have been reached, c) whether a discriminative stimulus (SD) (the cue to the student to initiate the target behavior) should be given and what it should be, d) should prompting be given and what level of prompting should it be (e.g., fully assisted help, point at something, verbal assist), and e) should reinforcement be given and what should it be (e.g., good job, a hug, a treat). GAINS efficiently uses valuable instructional time, eliminates instructor burdens, and maintains program fidelity across instructors, while eliminating errors in therapy and data collection. An example from putting on glasses is provided in the series of panels. It is displaying detailed, step-by-step guidance, which is also provided via Bluetooth bone conduction headphones, that a new instructor or family member may need. Setup is provided; then the SD is provided only for Step 1, the first step in the chain which should be done at the Independent prompt level because this step was previously mastered; Step 1 was done correctly and Step 2, the target step, is displayed at the Model Prompt Level (note the SD is not given because completion of Step 1 is both the reinforcement for completing Step 1 and the SD to begin Step 2). When done correctly, (see panel to the right) the instructor is guided to reinforce because the first two steps were done correctly and provide errorless learning for Step 3 by using a full physical prompt. When Step 2 is done incorrectly, (see panel below), the instructor is guided to give a stronger prompt. When Step 2 is done correctly with the stronger prompt, the instructor is guided for Step 3. However reinforcement is not given because not all the steps in the chain prior to and including the target step were done correctly. Intensive Trials Demonstrate Value and Technology Acceptance Field tests of increasing complexity and length were conducted from August 2015 through June 2017 at a large regional ASD service provider for children with severe developmental challenges due to ASD. Evaluation of data from two expert agents supporting two DTT protocols from May-June 2017 prove the value of GAINS. Data were collected from 33 instructors and 15 students (5 female and 10 males, 5 to 10 years old, mean of 8 years). On average, these students had been at the school 3.33 years, had experienced sessions in which the technology was used by their instructor. Over this period students responded to trials and experienced from 4 to 9 different instructors with a mean of 5.9. The student data showed clearly that these students acquired the skills taught by the instructors using GAINS.

4 Responses on the instructor surveys were transformed to construct a breadth score of the number of things the instructor liked about using the technology. These scores ranged from 7 to 44 points with a mean of These scores were weighted by the percentage of time the student spent with each instructor and then summed across all the instructors a student experienced, yielding a measure that weights the use of the technology in a session by percent of time. It is expected that a greater performance by an instructor who likes the technology, should result in better student results. Fifteen students were divided into two groups. One group was students whose experience with the technology was in the bottom half of the distribution and students in the other group were those whose experience with the technology was in the top half. The dependent variable was the number of trials for which a student scored at least 50% correct. The scores for those in the bottom half of the distribution ranged from 7 to 18 with a mean of 10.2, those in the upper half scored from 6 to 27 with a mean of Students with an overall better experience with the technology did in fact perform better (t (14) =1.91, p =.07, d= 1.02) as evidenced by a marginally acceptable level of significance of.07 given the very small sample size with a large effect size. The data show, on average, a 50% increase in correct responses. Instructors completed baseline and final surveys on a 5-point Likert scale concerning the perceived usability and usefulness of the technology [See Figure 2]. Overall perceived ease of use and usefulness of the technology are very positive, which predicts acceptance of the changes in work caused by the technology. 14 The technology works, children learn and instructors find it acceptable. Figure 2. Instructor Perceived Ease of Use and Usefulness Summary ABA is the gold standard in treating autism spectrum disorder (ASD); however there are many problems in delivering ABA, from unavailability in many places to skill gaps and variability in instructor experience, which leads to therapy inconsistencies and loss of program fidelity. The problems in delivering ABA can be mitigated using GAINS (Guidance, Assessment and Information Systems). GAINS incorporates knowledge of ABA practice and curricula into expert guiding agents to provide instructors and family members real-time process and decision support that adapts to student responses on the fly. Two years of intensive trials demonstrate that GAINS reduces upfront training needs while providing training as a by-product of using it, improves workflow and redistributes work, time and resources away from non-instructional to instructional effort. For more information:

5 References 1. Centers for Disease Control and Prevention (2014). Counting Autism. Author. 2. Yale Child Study Center (2009). Autism. Yale School of Medicine Flanagan, H. E., Perry, A., & Freeman, N. L. (2012). Effectiveness of large-scale communitybased intensive behavioral intervention: A waitlist comparison study exploring outcomes and predictors. Research in Autism Spectrum Disorders, 6 (2), National Autism Center (2015). National Autism Standards Report: Phase II. Randolph, MA: Author. 5. Dionne, C., Joly, J., Paquet, A., Rousseau, M., & Rivard, M. (2017). Organizations choices when implementing an Early Intensive Behavioral Intervention program (EIBI). Education Sciences & Society-Open Access Journal, 7, Cohen, H., Amerine-Dickens, M., & Smith, T. (2006). Early Intensive Behavioral Treatment: Replication of the UCLA Model in a Community Setting. Journal of Developmental & Behavioral Pediatrics. 27, Lovaas I. (1987). Behavioral treatment and normal educational and intellectual functioning in young autistic children. Journal of Consulting and Clinical Psychology. 55, Sallows, G., & Graupner, T. (2005). Intensive behavioral treatment for children with autism: Four-year outcome and predictors. American Journal on Mental Retardation. 110, Stahmer, A. C., Rieth, S., Lee, E., et al. (2015). Training teachers to use evidence-based practices for autism: Examining procedural implementation fidelity. Psychology in the Schools, 52, Strauss, K., Benvenuto, A., Battan, B., et al. (2015). Promoting shared decision making to strengthen outcome of young children with autism spectrum disorders: The role of staff competence. Research in Developmental Disabilities, 38, Jenkins, S., Hirst, J., Reed, F. (2015). The effects of discrete-trial training commission errors on learner outcomes: An extension. Journal of Behavioral Education, 24, doi: /s Reed, F., Reed, D., Baez, C., Maguire, H. (2011). A parametric analysis of errors of commission during discrete-trial training. Journal of Applied Behavior Analysis, 44(3): doi: /jaba Strain, P. S., & Bovey, E. H. (2011). Randomized, controlled trial of the LEAP Model of early intervention for young children with autism spectrum disorders. Topics in Early Childhood Special Education, 31 (3), Davis, F. (1989). "Perceived usefulness, perceived ease of use, and user acceptance of information technology", MIS Quarterly, 13 (3): , doi: /