There s a Bug in Your Ear!: Using Technology to Increase the Accuracy of DTT Implementation

Transcription

1 Education and Training in Autism and Developmental Disabilities, 2014, 49(4), Division on Autism and Developmental Disabilities There s a Bug in Your Ear!: Using Technology to Increase the Accuracy of DTT Implementation Tracy McKinney Georgia State University Eleazar Vasquez III University of Central Florida Abstract: Many professionals have successfully implemented discrete trial teaching in the past. However, there have not been extensive studies examining the accuracy of discrete trial teaching implementation. This study investigated the use of Bug in Ear feedback on the accuracy of discrete trial teaching implementation among two pre-service teachers majoring in elementary education and one pre-service teacher majoring in exceptional education. An adult confederate was used to receive discrete trial teaching. Implementing a multiple baseline across participants design, this study examined whether there was a functional relationship between receiving Bug in Ear feedback and the accuracy of discrete trial teaching implementation. The discrete trial teaching evaluation form was utilized to measure the accuracy of discrete trial teaching implementation. The findings demonstrated an increase in the discrete trial teaching implementation accuracy after Bug in Ear feedback was introduced. Participants agreed that using a self-instruction manual combined with receiving Bug in Ear feedback was beneficial in learning to implement discrete trial teaching. Individuals with autism typically struggle to learn information through incidental or informal teaching (Smith, 2001). This difficulty acquiring educational information haphazardly from the environment can lead to irritation and undesirable behaviors. Typically, individuals with autism function better with a more direct learning approach. Applied Behavior Analysis (ABA) is one such approach. ABA is an evidenced based methodology (Green, 1996) with strategies that can be used to teach individuals with Autism Spectrum Disorders (ASD) (Arnal et al., 2007; Salem et al., 2009; Smith, 2001; Steege, Mace, Perry, & Longenecker, 2007; Sturmey, 2008; Thiessen et al., 2009). The State of New York Health Department (1999) and The United States Surgeon General support the use of ABA for students with autism. Discrete trial teaching (DTT) is a teaching Correspondence concerning this article should be addressed to Tracy McKinney, Department of Educational Psychology, Special Education and Communication Disorders (ESC), College of Education, Georgia State University, Atlanta, GA TMckinney@GSU.edu strategy that uses principles from ABA (Smith, 2001) and has been shown to be helpful in teaching skills to individuals with autism (LeBlanc, Ricciardi, & Luiselli, 2005; Lerman, Vorndran, Addison, & Contrucci Kuhn, 2004; Lovaas, 1987; Smith, 2001). DTT consists of skills that are broken down into minute steps and is typically taught in a one-on-one environment using a highly organized method (Smith, 2001; Tews, 2007). The one-on-one format also contributes to the students increased learning as it allows instruction to be individually designed for each student. DTT is made up of many short teaching cycles, which means the information can be presented frequently and learning opportunities are increased. The teaching cycle is very predictable to the child because it follows the same basic format (Smith, 2001) and consists of several parts. The use of this supported instruction is important as the prevalence of autism continues to rise. Currently, one out of every 88 children is diagnosed with autism according to the Autism and Developmental Disabilities Monitoring Network (2012). As parents become more interested in DTT for their children, it is important the individ- 594 / Education and Training in Autism and Developmental Disabilities-December 2014

2 uals administering DTT have proper training (Ingersoll & Smith, 2003). Most teachers do not graduate with extensive experience in DTT implementation (Downs, Downs, & Rau, 2008) and some teachers feel as though they have been thrown in the classroom without enough support. Several researchers have examined the effectiveness of preparing individuals to implement DTT using various treatment packages. Koegel (1977) combined video clips, practice, and feedback to teach in-service teachers to accurately implement DTT. There has been extensive research at the University of Manitoba by various teams examining the use of a self-instruction manual to teach students, enrolled in psychology classes, to implement DTT. Results of these studies have been positive. Severtson (2011), used video instruction and modeling in conjunction with feedback to instruct paraprofessionals and in-home therapists. Bug in Ear (BIE) technology has been used effectively in the past to instruct various individuals in numerous fields. Korner and Brown (1952) were the originators as it relates to providing feedback to individuals via BIE technology. Their feedback was in the area of psychology. In the field of education, Bowles and Nelson (1976), Scheeler and Lee (2002), Rock, Gregg, Gable, and Zigmond (2009), Scheeler, Congdon, and Stansbery (2010), are all notable researchers. Bowles and Nelson used BIE feedback to increase fidelity of behavior modification techniques. Scheeler and Lee used BIE feedback to increase the use of three term contingencies. Rock et al. increased the mobility of BIE feedback by introducing a Skype component to the technology package. Scheeler et al. expanded BIE feedback to include co-teachers. While there have been seminal studies in both DTT implementation and feedback using BIE technology there have not been any studies combining the two important topics. This article will share the findings of this innovative merger. The following are the three research questions: 1. How does Bug In Ear feedback impact or affect implementation of discrete trial teaching procedures as measured by the DTTEF for three undergraduate pre-service teachers in education? 2. How does participants percent correct implementation of DTT procedures change from pre to post Bug In Ear feedback? 3. How does fidelity impact participants rating of the acceptability of the goals, procedures, and outcomes as socially valid as measured by a social validity questionnaire? Method Participants, Setting and Materials Three undergraduate students participated in the study. Two were elementary education majors and one was an exceptional education major. The study took place in a room with a two way mirror on the campus of a large university in the South Eastern United States. The study utilized a built-in video recording system to record the DTT sessions. This recording system also allowed the researchers to watch during the sessions and provide real-time comments on the DTT implementation. Additionally, a table top video camera was utilized to record the BIE feedback delivered to the participants by the researcher. Self-instruction manuals, BIE devices, data sheets, reinforcers, and pictures for the DTT tasks were also an integral part of the study. Experimental Design and Variables The dependent variable was the accuracy of DTT implementation as measured by the Discrete Trial Teaching Evaluation Form (DTTEF). The independent variable was the combination of the 1-page self instruction manual and BIE feedback. A validated data collection form (DTTEF) that has been used in previous studies was used during this study. The DTTEF examines 20 components of DTT implementation. This study implemented a multiple baseline across participants design, non experimental pre- post-tests, and a social validity questionnaire to answer the research questions. Each participant moved through three phases (baseline, treatment, and, maintenance) according to the following pre-determined phase change rules. First, a participant would move from baseline into treatment Using Technology to Increase the Accuracy of DTT / 595

3 once 80% of the participant s scores were on or within the stability envelope. Second, the intervention phase was terminated once a participant implemented DTT procedures with 90% accuracy 3 out of 4 consecutive sessions or if the participant had received a total of 10 sessions (the intervention needed to have a completion criteria in case a participant became stagnated at 90% accuracy) of intervention. During the baseline phase the participants delivered three different tasks or objectives using DTT procedures to a confederate playing the role of an individual with autism. The baseline phase consisted of participating pre-service teachers reading an abbreviated one-page self instruction manual on DTT procedures. Each of the three tasks was administered in a systematic rotation to ensure variety. However, each of the participants delivered the same DTT tasks during specific sessions of the study. For example, during baseline 3, all participants implementing the matching task. Prior to the treatment sessions, the participants were fitted with BIE devices. The BIE devices were paired to their phones and tested to make certain they worked properly. During the treatment phase the participants implemented DTT with a confederate while receiving feedback from the researcher via BIE technology. The researcher provided very brief intermittent encouraging feedback when the participants correctly implemented DTT and concise instructional feedback when the participants were implementing DTT incorrectly. The feedback the researcher provided was scripted to guarantee each participant received the same type of feedback, which helped to maintain control. During the maintenance phase, the participants implemented DTT with a confederate without any instruction or BIE feedback. Additionally, each participant took a pre-test prior to beginning the study and took a post-test once the study was over. During the pre- and post-tests the participants implemented DTT without any instruction or feedback. All of the feedback provided by the researcher was coded on the DTTEF as EF for encouraging feedback and IF for instructional feedback. The accuracy of DTT implementation was thought to have improved if the amount to instructional feedback decreased throughout the sessions. Reliability, Fidelity of Implementation, and Social Validity Two independent observers were responsible for primary and secondary data collection. A point-by-point method was used to analyze interobserver agreement (IOA) over 23% of all sessions. The average IOA was 87% during the multiple baseline phase and 90% during nonexperimental pre- post tests. To address treatment fidelity a confederate was utilized throughout the study phases. The confederate followed a script to ensure each participant received balanced responses and to increase procedural integrity. A procedural integrity checklist was also used across 20% of all sessions in the study to ensure the study was carried out with high levels of fidelity. The mean percentage of procedural integrity was 90%. A social validity questionnaire, similar to one used in previous studies, was delivered to ascertain the importance of the study. The questionnaire contained eight questions on a Likert scale. The questions related to the goals, procedures and effects of the study. The procedural integrity for the social validity checklist was 100%. Results Both visual analysis of the multiple baseline and descriptive analysis were conducted to evaluate the results of the treatment package. Visual analysis included an investigation of level, trend, and variability; predictability of the baseline data points; immediacy of change between phases; and percent of nonoverlapping data points (see Figure 1). Participant one demonstrated a noticeable change in level between the baseline and treatment phases. There was also a large amount of nonoverlapping data points present. Participant one had a baseline mean 55%, a treatment mean of 81% and a mean of 93% correct DTT implementation during the maintenance phase. Visual analysis of participant two demonstrated a noticeable change between the level and trend between the baseline and treatment phases. Similar to participant one, participant two displayed a large amount of nonoverlapping data points. Participant two displayed a mean of 64% correct DTT implementation during baseline, 85% 596 / Education and Training in Autism and Developmental Disabilities-December 2014

4 Figure 1. Multiple baseline results of percent correct DTT procedures. Using Technology to Increase the Accuracy of DTT / 597

5 TABLE 1 Difference scores between pre-tests and post-tests during the treatment phase and, 91% during the maintenance phase. While participant three displayed comparable levels between baseline and treatment phases, there was a marked change in trend lines between baseline and treatment phases. There was some overlap of data points. The baseline mean was 75% correct DTT implementation, 90% during treatment, and 94% during maintenance. Each participant increased at least 45% from the pre-test to the post-test (see Table 1). Participants completed the social validity questionnaires and indicated they felt the goals of the study were important. The participants also shared they learned an important skill that could be used when working with students with autism. The participants reported mixed feelings regarding the effectiveness of the self-instruction manual alone, but they all agreed that it was effective when combine with the BIE device. Discussion Pre-Test Post-Test Difference Annette 65% 98% 51% Mary 47% 90% 91% Phoebe 62% 90% 45% There was a 31% mean increase between the baseline and treatment phases among the three participants suggesting a functional relationship between BIE feedback and accurate DTT implementation. The primary difference between the current study and previous studies are the length of the self-instruction manuals. The current study used a one-page selfinstruction manual, while previous studies utilized a lengthier self-instruction manual (21 pages). Additionally, this study implemented BIE technology to provide real-time feedback to the participants. Even with these discrepancies, the findings of this study are similar to previous studies in that each of the participants made gains in the accurate implementation of DTT. Important to the study was the need to address treatment fidelity. Two experts in DTT implementation viewed an initial baseline session video and a video of the final treatment session to determine if DTT implementation accuracy increased from the baseline phase to the treatment phase. Both experts used the Discrete Trial Teaching Evaluation Form (DTTEF) to score the accuracy of DTT implementation and agreed that the implementation during the treatment session was more accurate than the implementation during the initial baseline session (20% increase from baseline to treatment). This finding lends support to the functional relationship between accurate DTT implementation and BIE feedback. This finding expands past research by supporting the successes of self-instruction manuals as well as the effectiveness of providing immediate feedback via BIE (Arnal et al., 2007; Fazzio, Martin, Arnal, & Yu, 2009; Rock et al., 2009; Scheeler, Congdon, & Stansbery, 2010). To help preserve treatment fidelity a confederate was used to receive DTT instruction. In this study and in previous studies, the use of the confederate also was helpful in maintaining experimental control to help ensure each of the participants received balanced responses from the confederate (Arnal et al., 2007; Fazzio et al., 2009; Salem et al., 2009; Thiessen et al., 2009). These balanced responses allowed the participants to have very similar opportunities to exhibit specific components of DTT (i.e., providing reinforcers and blocking) versus a model where Participant One only experiences opportunities to provide reinforcers and Participant Two only experiences opportunities to block attempts to respond incorrectly. Limitations Limitations of this study include possible researcher bias as the researcher served as the BIE coach and provided feedback vie BIE technology to the participants. The use of a confederate rather than an individual with autism can also be interpreted as a limitation of the current study. Additionally, the setting could also be seen as a limitation as it was a clinical setting rather than a setting that would be considered natural. It is foreseeable that researchers could replicate and extend this 598 / Education and Training in Autism and Developmental Disabilities-December 2014

6 line of research with these procedures to teach classroom teachers, paraprofessionals, and private therapists to accurately implement DTT. Implications for Practice The outcome of this study has several implications related to agencies and trainers who are preparing individuals to implement DTT through feedback using BIE. The procedures in the current study can be replicated to prepare individuals to accurately implement DTT in a relatively quick timeframe and for minimal funding (approximately $30 for a BIE device) which is an added benefit in today s economy. With an added component of Skype, travel time and costs for supervision could be decreased. A final implication of this study is related to individuals with ASD. Researchers have discovered that students with ASD are more successful when instruction is more direct and less incidental (Smith, 2001). DTT is an intervention that fits that description. Following these procedures to learn how to accurately implement DTT could mean that more individuals could be educated in correct DTT implementation and in turn, more students with ASD would receive proper DTT instruction. This implementation is important as the Autism and Developmental Disabilities Monitoring Network recently reported the rate of prevalence for children born with Autism has risen to 1 out of every 88 children (2012). Therefore an effective treatment package to prepare individuals to accurately implement DTT would be of great benefit. When examining the results of this study, one can assume that pre-service teachers with no experience implementing DTT can likely be trained to implement DTT in an average of eight sessions with each session lasting approximately five minutes as this was the case in the current study. Given the outcomes of this study, participants of future replications are likely to maintain the acquired skills without feedback. Additionally, these findings support future treatment packages that combine selfinstruction manuals and BIE feedback to train individuals to implement DTT. The mobility of BIE technology is an advantage of this study and should help generalize this study to other natural settings. A component analysis would be helpful in future research as it will provide insight on whether there are specific aspects of the DTT process that individuals are struggling with implementing. Two guidelines for practice that can be used to make certain implementation fidelity is maintained when delivering DTT are 1) ensure individuals are accurately trained to implement DTT and 2) use an implementation fidelity checklist to measure fidelity on a regular basis. References Arnal, L., Fazzio, D., Martin, G., Yu, C. T., Keilback, L., & Starke, M. (2007). Instructing university students to conduct discrete-trials teaching with confederates stimulating children with autism. Developmental Disabilities Bulletin, 35(1 2), Bowles, P. E., Jr., & Nelson, R. O. (1976). Training teachers as mediators: Efficacy of a workshop versus the bug-in-the-ear technique. Journal of School Psychology, 14, Centers for Disease Control and Prevention, Autism and Developmental Disabilities Monitoring Network. (2008). Prevalence of the autism spectrum disorders. Retrieved from gov/pubmed/ Downs, A., Downs, R. C., & Rau, K. (2008). Effects of training and feedback on Discrete Trial Teaching skills and student performance. Research in Developmental Disabilities: A Multidisciplinary Journal, 29(3), Fazzio, D., Martin, G. L., Arnal, L., & Yu, C. T. (2009). Instructing university students to conduct discrete-trials teaching with children with autism. Research in Autism Spectrum Disorders, 3, doi: /j.rasd Green, G. (1996). Early behavioral intervention for autism. In C. Maurice, G. Green, & S. C. Luce (Eds.), Behavioral interventions for young children with autism (pp ). Austin, TX: Pro ed. Ingersoll, R. M., & Smith, T. M. (2003). The wrong solution to the teacher shortage. Educational Leadership, 60(8), Koegel, R. L., Russo, D. C., & Rincover, A. (1977). Assessing and training teachers in the generalized use of behavior modification with autistic children. Journal of Applied Behavior Analysis, 10, Korner, I. N., & Brown, W. H., (1952). The mechanical third ear. Journal of Counseling Psychology, 16, LeBlanc, M-P, Ricciardi, J. N., & Luiselli, J. K. (2005). Improving discrete trial instruction by paraprofessional staff through an abbreviated Using Technology to Increase the Accuracy of DTT / 599

7 performance feedback intervention. Education and Treatment of Children, 28, Lerman, D. C., Vorndran, C. M., Addison, L., & Contrucci Kuhn, S. (2004). Preparing teachers in evidence-based practices for young children with autism. School Psychology Review, 33, Lovaas, O. I. (1987). Behavioral treatment and normal educational and intellectual functioning in young autistic children. Journal of Consulting and Clinical Psychology, 55, 3 9. New York State Department of Health Early Intervention Program. (1999). Clinical practice guideline: Report of the recommendations, autism/pervasive developmental disorders, assessment, and intervention for young children. Publication# Heath Education Services. Retrieved from health.state.ny.us/nysdoh/eip/menu.htm Rock, M. L., Gregg, M., Gable, R. A., & Zigmond, N. P. (2009). Virtual coaching for novice teachers. Phi Delta Kappan, 91(2), Salem, S., Fazzio, D., Arnal, L., Fregeau, P., Thomson, K., Martin, G. L., & Yu, C. T. (2009). A self-instruction package for teaching university students to conduct discrete-trials teaching with children with autism. Journal on Developmental Disabilities, 15, Scheeler, M. C., Congdon, M., Stansbery, S. (2010). Providing Immediate Feedback to Co-teachers through bug-in-ear technology: An effective method of peer coaching in inclusion classrooms. Teacher Education and Special Education, 33, doi: / Scheeler, M. C., & Lee, D. L. (2002). Using technology to deliver immediate corrective feedback to preservice teachers. Journal of Behavioral Education, 11, doi: /02/ /0 Severtson, (2011). A sequential analysis of staff training procedures to efficiently teach novice instructors to implement errorless discrete-trial teaching procedures. (Doctoral dissertation, Western Michigan University). Retrieved from com/dissertations/docview/ /preview PDF/13BECB95EEE3FDCF0EE/2?accountid Smith, T. (2001). Discrete trial training in the treatment of autism. Focus on Autism and Other Developmental Disabilities, 16, doi: / Steege, M. W., Mace, F. C., Perry, L., & Longenecker, H., (2007). Applied behavior analysis: Beyond discrete trial training. Psychology in the Schools, 44, doi: /pits Sturmey, P. (2008). Best practice methods in staff training. In J. Luiselli, D. Russo, W. Christian, & S. Wilczynski (Eds.), Effective practices for children with autism. Educational and behavioral support interventions that work (pp ). New York: Oxford University Press. Tews, L. (2007). Early intervention for children with autism: Methodologies critique. Developmental Disabilities Bulletin, 35(1 & 2), Retrieved from login.aspx?direct true&db eric&an EJ &site ehost-live Thiessen, C., Fazzio, D., Arnal, L., Martin, G. L., Yu, C. T., & Keilback, L. (2009). Evaluation of a selfinstruction manual for conducting discrete-trials teaching with children with autism. Behavior Modification, 33, doi: / U.S. Department of Health and Human Services. (1999). Mental Health: A Report of the Surgeon General. U.S. Department of Health and Human Services: Rockville, MD. Received: 12 April 2013 Initial Acceptance: 15 June 2013 Final Acceptance: 28 August / Education and Training in Autism and Developmental Disabilities-December 2014