POST-HOC ANALYSIS OF CHALLENGING BEHAVIOR BY FUNCTION: A COMPARISON OF MULTIPLE-RESPONDENT ANECDOTAL ASSESSMENTS, FUNCTIONAL ANALYSES, AND TREATMENTS

POST-HOC ANALYSIS OF CHALLENGING BEHAVIOR BY FUNCTION: A COMPARISON OF MULTIPLE-RESPONDENT ANECDOTAL ASSESSMENTS, FUNCTIONAL ANALYSES, AND TREATMENTS Kathleen Dignan, B.S. Thesis Prepared for the Degree of MASTER OF SCIENCE UNIVERSITY OF NORTH TEXAS August 2016 APPROVED: Richard Smith, Major Professor Karen Toussaint, Committee Member Jesús Rosales-Ruiz, Committee Member and Chair of the Department of Behavior Analysis Tom Evenson, Dean of the College of Public Affairs and Community Service Victor Prybutok, Vice Provost of the Toulouse Graduate School

Dignan, Kathleen. Post-Hoc Analysis of Challenging Behavior by Function: A Comparison of Multiple-Respondent Anecdotal Assessments, Functional Analyses, and Treatments. Master of Science (Behavior Analysis), August 2016, 63 pp., 10 tables, 17 figures, references, 23 titles. The current study examines anecdotal assessment, functional analysis, and treatment outcomes from 44 participants. Agreement across Motivation Assessment Scale (MAS), Questions About Behavioral Function (QABF), and Functional Analysis Screening Tool (FAST) assessments, agreement between those anecdotal assessments and functional analyses, and agreement between those anecdotal assessments and treatment outcomes were analyzed across maintaining variables and topography categories of challenging behaviors. Overall, the QABF had the highest agreement results with functional analyses and treatment with 70% and 92% of cases respectively. Patterns in the distribution of maintaining variables was examined across behavior topography categories.

Copyright 2016 by Kathleen Dignan ii

ACKNOWLEDGEMENTS I would like to thank my advisor, Dr. Rick Smith, for his guidance and friendship throughout my education at the University of North Texas. He has been a wonderful role model of a pragmatic, disciplined, and caring behavior analyst. I have learned many skills while working with him, and all of the BCBAs and students at the Behavior Analysis Resource Center. Without the dedication and skills of these students and BCBAs, this project would not be possible. I would also like to thank my committee members, Dr. Jesus Rosales-Ruiz and Dr. Karen Toussaint, for their input on this project and their contributions to my skills as a behavior analyst. I am very grateful to the entire faculty and staff of the Department of Behavior Analysis. I could not have succeeded without their dedication to my education and support throughout the years. I would like to thank my parents and family for their support, both emotionally and financially, throughout my years at UNT. I cannot express enough my gratitude for everything they have done and all the love they have shown me. They have taught me to always help those who cannot help themselves, and I know that I can achieve that through my work in behavior analysis. iii

TABLE OF CONTENTS Page ACKNOWLEDGMENTS LIST OF TABLES LIST OF ILLUSTRATIONS iii v vi Chapter 1. INTRODUCTION 1 2. METHOD 5 3. RESULTS: ANECDOTAL ASSESSMENT AGREEMENT...12 4. DISCUSSION: ANECDOTAL ASSESSMENT AGREEMENT 17 5. RESULTS: ANECDOTAL ASSESSMENT AND FA AGREEMENT 18 6. DISCUSSION: ANECDOTAL ASSESSMENT AND FA AGREEMENT 22 7. RESULTS: ANECDOTAL ASSESSMENT AND TREATMENT AGREEMENT 24 8. DISCUSSION: ANECDOTAL ASSESSMENT AND TREATMENT AGREEMENT 27 9. RESULTS: PATTERNS ACROSS BEHAVIOR TOPOGRAPHIES 29 10. DISCUSSION: PATTERNS ACROSS BEHAVIOR TOPOGRAPHIES 36 11. GENERAL DISCUSSION AND LIMITATIONS 38 REFERENCES...61 iv

LIST OF TABLES Page 1. Participant Demographics 41 2. General Topographies, Included Behaviors, and Participant Distribution 41 3. Inter-Observer Agreement (IOA) for Functional Analyses 42 4. Inter-Observer Agreement (IOA) for Treatments 43 5. Treatment Categories 44 6. Overall Agreement for Anecdotal Assessments 44 7. Ties on Inter-Rater Agreement for Anecdotal Assessments 44 8. Ties on Inter-Rater Agreement for MAS by Maintaining Variables 45 9. Ties on Inter-Rater Agreement for QABF by Maintaining Variables 45 10. Ties on Inter-Rater Agreement for FAST by Maintaining Variables 45 11. Anecdotal Assessment Results and FA Outcomes by Topography Category 46 v

LIST OF ILLUSTRATIONS Page 1. MAS and QABF Results 48 2. MAS and FAST Results 49 3. QABF and FAST Results 50 4. MAS Results and FA Outcomes 51 5. QABF Results and FA Outcomes 52 6. FAST Results and FA Outcomes 53 7. MAS Results and Treatment Categories 54 8. QABF Results and Treatment Categories 55 9. FAST Results and Treatment Categories 56 10. MAS Results by Topography Category 57 11. QABF Results by Topography Category 57 12. FAST Results by Topography Category 58 13. Anecdotal Assessment Agreement by Topography Category 58 14. FA Outcomes by Topography Category 59 15. Agreement of Anecdotal Assessment Results & FA Outcomes by Topography Category 59 16. Treatment Category by Topography Category 60 17. Agreement of Anecdotal Assessment Results & Treatment Categories by Topography Category 60 vi

CHAPTER 1 INTRODUCTION Behavior analysts working with individuals with intellectual disabilities are often tasked with decreasing challenging behavior. Challenging behaviors are those that trouble the client, or others, enough for a solution to be deemed warranted by the client, or others, and the therapist (Baer, Wolf, & Risley, 1987). Topographies of challenging behavior vary across individuals, and can include aggression, self-injury, property destruction, and stereotypic behaviors, among others (Matson & Williams, 2014). Applied behavior analysts typically attempt to identify the environmental variables maintaining challenging behavior in order to provide effective treatments. Behavior analysts have developed three general assessment processes to aid in the identification of these maintaining variables. One way to identify the maintaining variables of challenging behavior is through direct observation of the behavior when it occurs in the natural environment, or descriptive assessment (Bijou, Peterson, & Ault, 1968). By recording behavior and contextual events, clinicians can detect patterns in the environmental variables surrounding the instances of a challenging behavior that may suggest possible maintaining variables. Several studies have shown that descriptive assessments do not often accurately identify the environmental variables responsible for the occurrence and maintenance of challenging behaviors (Hall, 2005; Thompson & Iwata, 2007; Neef & Peterson, 2007). Anecdotal assessment (e.g. Durand & Crimmins, 1988) is another method to identify the maintaining variables of challenging behavior. Caregivers are asked to provide information about the challenging behavior and the situations under which it is more or less likely to occur. Anecdotal assessments are typically presented in rating scales (e.g., Durand & Crimmins). The 1

suggested maintaining variable is inferred from responses identifying particular antecedents and consequences that are highly associated with the behavior. Three commonly used anecdotal assessments are the Motivation Assessment Scale (MAS, Durand & Crimmins), the Questions about Behavioral Function (QABF, Paclawskyj et al., 2000) and the Functional Assessment Screening Tool (FAST, Iwata et al., 2013). These assessments are simple and quick to administer, making them appealing to clinicians; however, their reliability and validity should be determined before treatments are based exclusively on their results. The utility of anecdotal assessments to identify the function of different topographies of challenging behavior, including aggression, self-injury, stereotypy, disruptive behavior, destructive behavior, and feeding problems has been investigated. For example, studies have analyzed the degree to which respondents to anecdotal assessments agree with each other on the suggested maintaining variable for a given case. Paclawskyj et al. (2001) compared QABF results to MAS results and found that there was agreement on response function in 61.5% of cases. Koristas & Iacono (2013) administered the MAS and QABF to disability support workers for individuals with intellectual disabilities and found that respondents agreed on behavioral function in 39.1% of cases. Other studies have produced similarly mixed results on agreement across respondents and assessments, as well as on the validity of anecdotal assessment outcomes (Sigafoos et al., 1994; Duker & Sigafoos, 1998; Matson et al., 1999; Matson et al., 2005). Experimental functional analysis (FA) directly manipulates environmental variables to test the conditions under which behavior does and does not occur. Using FA procedures, it is possible to demonstrate the effects (or lack thereof) of potentially relevant antecedent and subsequent conditions on challenging behavior. Iwata et al. (1982/1994) described a procedure in which participants were repeatedly exposed to test conditions that isolated the effects of specific 2

contingencies on challenging behavior. The maintaining variable was determined from the condition, or conditions, that were associated with the highest measures of challenging behavior. For example, if aggression occurred with more frequency in the attention condition, in which the therapist delivered attention contingent on challenging behavior, relative to a control condition in which attention was frequently presented on a time-based schedule, it was determined that the patient s aggression was most likely maintained by social positive reinforcement in the form of attention. Iwata et al. (1994) reported FA outcomes for 152 individuals and evaluated the efficacy of treatments based on these FA outcomes. Results indicated that treatments based on the FA outcomes were successful in 84.9% of cases. Although FA has been established as a highly valid means of identifying the maintaining variables for challenging behavior, training and resource requirements may limit the feasibility of its use for many practitioners (Matson & Williams, 2014). Some previous studies have evaluated correspondence between anecdotal assessment results and FA outcomes. Hall (2005) compared QABF results and FA results for 4 individuals with intellectual disabilities and found that agreement occurred in 75% of cases. Smith et al. (2012) evaluated the agreement among MAS, QABF, and FA outcomes for 7 adults with intellectual disabilities. Agreement with FA outcomes occurred in 57.1% of cases for MAS and 85.7% cases for QABF. Paclawskyj et al. (2001) also compared QABF and MAS results to FA outcomes for 13 participants and found that QABF and FA agreed in 56.3% and MAS and FA agreed in 43.8% of cases. Although agreement between anecdotal assessments and FA has been used to assess the validity of anecdotal assessments, some argue that a better measure of validity would be successful treatments based on assessment outcomes. Wolf (2007) evaluated the effectiveness of 3

treatment corresponding to behavioral function suggested by MAS and FAST results with 2 individuals with intellectual disability. Treatment based on the anecdotal assessments was successful for 1 participant and unsuccessful for the other. Staff (2012) expanded on this research by administering MAS and QABF assessments to caregivers of 4 adults with intellectual disabilities. Based on agreement of respondents on the anecdotal assessments, testcontrol analyses of the challenging behavior were conducted. In the test-control analysis, baseline sessions, in which the contingency was placed on challenging behavior, alternated with treatment sessions, in which the contingency was placed on a replacement behavior. For all participants, the treatments based on the MAS and QABF outcomes resulted in a decrease of challenging behavior and increase of replacement behavior in treatment conditions. The distribution of maintaining variables determined by FA across behavior topographies has been examined in several studies (Hanley, Iwata, & McCord, 2003; Beavers, Iwata, & Lerman, 2013). Beavers et al. found the most common problem behaviors for which an FA was conducted were aggression and self-injury, according to a search of publications. Social negative reinforcement was found to be the most frequently determined maintaining variable for both behavior topographies. To date, there have been no studies which examine the distribution of maintaining variables determined by anecdotal assessments across behavior topographies. The purpose of the current study is to examine anecdotal assessment results, FA results, and treatment outcomes from multiple cases to identify patterns across behavioral function and topography. The current study assessed agreement between the MAS, QABF, and FAST, between each anecdotal assessment and FA results, and between each anecdotal assessment and treatment outcomes. Relations between behavior topography and the outcomes of assessments were also examined. 4

CHAPTER 2 METHOD Participants and Setting Participants. All participants resided in a large, state-operated residential facility for adults with intellectual disabilities. The data were collected from participants enrolled in a behavior disorder clinic over a 10-year period (2006-2015). The participants were referred by their interdisciplinary team to the behavior disorder clinic for assessment and treatment of challenging behavior. Based on the assessments and treatments completed during their enrollment in the behavior disorder clinic, 44 participants were included in this study. To qualify for inclusion, anecdotal assessments must have been conducted with at least 5 respondents (an exception was made for Participant 3, for whom data were obtained from only 4 respondents), FA must have been completed, and/or treatment must have successfully reduced challenging behavior by 75% and must have been considered complete by the clinicians who conducted it. Participant demographics are displayed in Table 1. Some participants data have previously been reported. Data from participants 14, 16, 17, 18, 19, and 20 were previously published with the names Asa, Annie, Greg, Karen, Vern, and Jolinda respectively in Smith et al. (2012). Data from participant 44 was previously published with the name Peter in Dracobly & Smith (2012) and Smith et al. (2012). Behavior topographies. Target behaviors were grouped into 6 categories according to their physical and operational properties: aggression toward property (AGP), physical aggression towards others (PAO), physically disruptive behavior (PDB), self-injurious behavior (SIB), stealing (STL), and verbally disruptive behavior (VDB). For example, physical aggression against others includes any behavior that could cause physical injury or tissue damage to others 5

(e.g., hitting, punching, biting, etc.). Specific topographies that were included in these categories are displayed in Table 2. Setting. Anecdotal assessments were administered in the participants homes and vocational settings. Functional analyses were conducted in the behavior disorder clinic located on the state-operated residential facility campus. Treatment sessions were conducted in the behavior disorder clinic, participants homes, and/or participants vocational settings. Sessions in the behavior disorder clinic were conducted in a therapy room (3.7 m by 3.7 m) containing all the necessary materials for each session. Data Collection Trained graduate and undergraduate students collected data for all anecdotal assessments, FA sessions, and treatment sessions. For all sessions conducted in the behavior disorder clinic, observers collected data through a one-way window on the wall of the therapy room. For all sessions conducted in the participants homes and vocational settings, observers collected data in the vicinity of the session being conducted. For all sessions, observers did not interact with the participant or the therapist, other than to signal to the therapist that session time had elapsed. Observers collected data on paper forms, hand-held electronic devices, or laptop computers equipped with data collection software. Interobserver Agreement (IOA) Functional analysis. For Participants 25, 26, 29, 30, 33, 36-39, and 41, IOA was calculated by dividing the session into 10-s intervals. For each interval, the lower number of responses (or amount of time) recorded by an observer was divided by the higher number of responses recorded by the other observer. If both observers recorded a 0, then the interval was scored as 1. The outcome for all 10-s intervals was summed and the total was divided by the 6

number of intervals in the session. The result was then multiplied by 100 to give a percentage. For Participants 14-20, and 34, IOA was calculated the same as previously described, but sessions were divided into 1-s intervals. For participant 44, IOA was calculated the same way as previously described, but the session was divided into 1-s intervals and the observers were given a 2-s tolerance window (e.g. if one observer recorded an event at time x, agreement was scored if the other observer recorded the same event at time x-1 s, time x or time x+1 s). Individual IOA results are listed in Table 3. Treatment. For Participants 29, 36, and 40, IOA was calculated by dividing the session into 10-s intervals. For each interval, the lower number of responses (or amount of time) recorded by an observer was divided by the higher number of responses recorded by the other observer. If both observers recorded a 0, then the interval was scored as 1. The outcome for all 10-s intervals was summed and the total was divided by the number of intervals in the session. The result was then multiplied by 100 to give a percentage. For Participants 1, 2, 15, 17, 19, 20, 21, 24, and 29, IOA was calculated the same as previously described, but sessions were divided into 1-s intervals. For Participant 44, IOA was calculated the same way as previously described, but the session was divided into 1-s intervals and the observers were given a 1-s tolerance window (e.g. if one observer recorded an event at time x, agreement was scored if the other observer recorded the same event at time x-1 s, time x or time x+1 s). Individual IOA results are listed in Table 4. General Procedures Anecdotal assessments. Anecdotal assessments were administered to 5 respondents per participant, per target behavior (with exceptions of 4 respondents for participant 3 and 6 respondents for participant 17). Respondents were caregivers that, at the time of the assessment, worked at the residential facility. Respondents must have worked directly with the participant for 7

at least 6 months, and must have seen the challenging behavior occur. For all anecdotal assessments, the interviewer read the questions aloud to the rater. The rater responded aloud and the rater s answers were recorded by the interviewer. The interviewer scored the anecdotal assessment out of sight of the rater. The interviewer did not provide clarification for any questions on the assessment, nor did the interview give feedback on any answers provided by the rater. Raters completed the multiple types of anecdotal assessments for a target behavior within the same interview, if applicable. Once 5 respondents had been interviewed, inter-rater agreement was calculated for each assessment (per behavior). The category that ranked highest among the most raters was the determined to be the suggested contingency maintaining the target behavior. Motivation Assessment Scale (MAS). The MAS (Durand & Crimmins, 1988b) is a 16- question assessment with the goal of identifying the maintaining variables of a given behavior. Respondents rate how often the behavior occurs under different situations by completing a 7- point Likert-type scale. This scale includes 0 (never), 1 (almost never), 2 (seldom), 3 (half the time), 4 (usually), 5 (almost always), 6 (always) for each question. Four questions correspond to 4 categories of behavioral function (attention, sensory, escape, and tangible). The ratings on the 4 questions of each category are totaled, and the categories are ranked with the highest total ranked first, and the lowest total ranked fourth. Categories with equal totals are given a shared ranking. Questions About Behavioral Function (QABF). The QABF (Matson & Vollmer, 1995) is a 25-question assessment with the goal of identifying the maintaining variables of a given behavior. Respondents rate how often the behavior occurs under different situations by completing a 4-point Likert-type scale. This scale includes 0 (never), 1 (rarely), 2 (some), and 3 (often) for each question. The rating scale also has the option of answering Does not apply 8

which results in a score of 0 for that question. Five questions correspond to 5 categories of maintaining consequences (attention, non-social, physical, escape, and tangible). The ratings on the 5 questions within each category are totaled, and the categories are ranked with the highest total placed first, and the lowest total placed fifth. Categories with equal totals are given a shared ranking. Functional Analysis Screening Tool (FAST). The FAST (Iwata, DeLeon, & Roscoe, 2013) is a 16-question assessment with the goal of identifying the maintaining consequences of a given behavior. Respondents rate whether the behavior occurs in different situations with either Yes, No, or N/A. Four questions correspond to 4 categories of behavioral function (social positive, social negative, automatic positive, and automatic negative). The number of Yes answers on the 4 questions within each category is totaled. The category with the most number of Yes answers is determined to be the maintaining variable of the challenging behavior. Categories with equal totals are given a shared ranking. The FAST includes a section that asks open-ended and multiple-choice questions that provide more information about the challenging behavior, including general frequency, severity, and environment. This portion of the assessment was not included in the evaluation of inter-rater agreement. Functional analysis (FA). Functional analysis procedures varied slightly between participants due to the topography of the target behavior, the results of anecdotal assessments, and the participant s communication and motor skills. Generally, FA procedures were similar to those described in Iwata et al. (1982/1994). Participants were exposed to a repeating series of 4 conditions (alone, attention, play/control, and escape) which created an environment to test for possible maintaining functions of the target behavior. A no-interaction condition replaced the alone condition for some participants if the severity or topography of the behavior was such that 9

the participant could not be left alone. A tangible condition was included for some participants if anecdotal assessments indicated that the target behavior might be maintained by social positive reinforcement in the form of access to tangible items (Rooker, Iwata, Harper, Fahmie, & Camp, 2011). Trained graduate and undergraduate students served as therapists. Each session was 10 minutes in length and consisted of one condition. Between 1 and 6 sessions were conducted each day for 3-5 days per week. Treatment. Treatment procedures varied considerably across participants based on the participant s functional analysis and/or anecdotal assessment results, the desired result of the treatment, the participant s current skills, and the setting in which the target behavior occurred. It was not the case that an FA preceded all treatments included in this study. For the purposes of this study, treatments were categorized by the maintaining consequence of the behavior that they were designed to address. For example, if the treatment adjusted the antecedent conditions such that the motivation for the behavior s automatically occurring consequence was lowered, it was categorized as automatic. For a list of treatment categories and types of treatment that were included in each category, see Table 5. Comparison Calculations In order to accurately compare anecdotal assessment, FA, and treatment outcomes, the following rules were created. The sensory category of the MAS, the non-social and physical categories of the QABF, and the automatic positive and automatic negative categories of the FAST were scored as agreement with each other and with automatic outcomes of FA and treatment. For ease of comparison, the previously listed categories will all be referred to as automatic. The social positive category of the FAST was scored as agreement with tangible and attention categories of the MAS, QABF, FA, and treatments. The social 10

negative category of the FAST was scored as agreement with the escape category of the MAS, QABF, FA, and treatment. MAS, QABF, and FAST. Agreement across anecdotal assessments was calculated and categorized by suggested maintaining consequence and behavior topography. Comparisons were scored as an agreement if the maintaining consequence determined by the inter-rater agreement for both assessments matched. If inter-rater agreement for an assessment resulted in multiple categories being suggested as the maintaining consequence, that assessment was included in all relevant categories for calculations. Anecdotal assessments and FA. Agreement between anecdotal assessments and functional analysis was calculated and categorized by maintaining consequence and behavior topography. Comparisons were scored as agreement if the maintaining consequence determined by the inter-rater agreement for the anecdotal assessment and the results of FA matched. Anecdotal assessments and treatment. Agreement between anecdotal assessments and treatment was calculated and categorized by function and behavior topography. Comparisons were scored as agreement if the function determined by the inter-rater agreement for the anecdotal assessment and the chosen treatment category matched. 11

CHAPTER 3 RESULTS: ANECDOTAL ASSESSMENT AGREEMENT MAS and QABF Agreement Figure 1 shows a comparison of maintaining variables identified by the MAS and QABF. MAS categories are distributed vertically by category and QABF categories are distributed horizontally by category. The upper left corner of each cell in the display contains the ratio of the frequency of cases in which the QABF identified the category corresponding to that cell, given the MAS results corresponding to that cell. The bottom right corner of each cell contains the percentage of cases in which the QABF identified the category corresponding to that cell, given the MAS results corresponding to that cell. The shaded cells indicate areas of agreement between assessments. Figures 1 through 9 follow this general structure to compare different anecdotal assessment results, FA outcomes, and treatment categories. The top panel shows the results of the QABF, given specific MAS results. Of the 15 target behaviors included in the calculations for which the MAS suggested attention as the maintaining consequence, QABF results suggested attention for 5 (33%) behaviors. The QABF outcomes suggested automatic reinforcement for 5 (33%) behaviors, escape for 5 (33%) behaviors, and a tangible for 1 (7%) behavior. Note that the total number of QABF is greater than the total number of MAS due to a tied score between the escape and automatic categories for one behavior. Of the 35 target behaviors included in the calculations for which MAS suggested automatic reinforcement, QABF results suggested automatic reinforcement for 22 (63%) behaviors, tangible for 8 (23%) behaviors, attention for 5 (14%) behaviors, and escape for 3 (9%) behaviors. Of the 12 target behaviors included in the calculations for which MAS suggested escape as the maintaining function, QABF results suggested escape for all 12 (100%) 12

behaviors. Of the 36 target behaviors included in the calculations for which MAS suggested tangible reinforcement, QABF results suggested tangible for 21 (53%) behaviors, escape for 13 (36%) behaviors, automatic for 5 (14%) behaviors, and attention for 3 (8%) behaviors. The bottom panel shows the results of the MAS, given specific QABF results. Of the 13 target behaviors included in the calculations for which the QABF suggested attention as the maintaining consequence, MAS results suggested attention for 5 (38%) behaviors, suggested automatic reinforcement for 5 (38%) behaviors, tangible for 3 (23%) behaviors. Of the 30 target behaviors included in the calculations for which QABF suggested automatic reinforcement, MAS results suggested automatic reinforcement for 23 (77%) behaviors, attention for 5 (17%), and tangible for 5 (17%) behaviors. Of the 27 target behaviors included in the calculations for which QABF suggested escape as the maintaining function, MAS results suggested escape for 12 (44%) behaviors, tangible for 13 (48%) behaviors, attention for 5 (19%) behaviors, and automatic for 3 (11%) behaviors Of the 30 target behaviors included in the calculations for which QABF suggested tangible reinforcement, MAS results suggested tangible for 21 (70%) behaviors, automatic for 9 (30%) behaviors, and attention for 1 (3%) behavior. Overall, MAS and QABF showed agreement for 60 out of 90 (66.7%) behaviors as shown in Table 6. MAS and FAST Agreement Figure 2 displays a comparison of maintaining variables identified by the MAS and FAST. The top panel shows the results of the FAST, given specific MAS results. Of the 2 target behaviors included in the calculations for which the MAS suggested attention as the maintaining consequence, FAST results suggested social positive for 1 (50%) behavior and social negative for 1 (50%) behavior. Of the 16 target behaviors included in the calculations for which MAS suggested automatic reinforcement, FAST results suggested automatic reinforcement for 9 (56%) 13

behaviors, social negative for 6 (38%), and social positive for 3 (19%) behaviors. Of the 7 target behaviors included in the calculations for which MAS suggested escape as the maintaining variable, FAST results suggested social negative for all 7 (100%) behaviors. Of the 13 target behaviors included in the calculations for which MAS suggested tangible reinforcement, FAST results suggested social positive for 11 (85%) behaviors, and social negative for 2 (15%) behaviors. Because the FAST combines attention and tangible maintaining variables into one category, those categories on the MAS have been combined for the final comparison. Of the 15 target behaviors included in the calculations for which MAS suggested either attention or tangible maintaining variables, FAST results suggested social positive for 12 (80%) behaviors, and social negative for 3 (20%) behaviors. The bottom panel shows the results of the MAS, given specific FAST results. Of the 15 target behaviors included in the calculations for which the FAST suggested social positive reinforcement, MAS results suggested attention or tangible for 12 (80%) behaviors and automatic for 3 (20%) behaviors. Of the 9 target behaviors included in the calculations for which FAST suggested automatic reinforcement, MAS results suggested automatic reinforcement for 9 (100%) behaviors. Of the 13 target behaviors included in the calculations for which FAST suggested social negative reinforcement, MAS results suggested escape for 7 (54%) behaviors, automatic for 6 (46%) behaviors, and attention/tangible for 3 (23%) behaviors. Overall, MAS and FAST showed agreement for 28 out of 35 (80%) behaviors as shown in Table 6. QABF and FAST Agreement Figure 3 shows a comparison of maintaining variables identified by the QABF and FAST. The top panel shows the results of the FAST, given specific QABF results. Of the 3 target behaviors included in the calculations for which the QABF suggested attention as the 14

maintaining consequence, FAST results suggested social positive for 1 (33%) behavior, automatic reinforcement for 1 (33%) behavior, and social negative for 2 (67%) behaviors. Of the 12 target behaviors included in the calculations for which QABF suggested automatic reinforcement, FAST results suggested automatic reinforcement for 8 (67%) behaviors, social positive for 4 (33%), and social negative for 1 (8%) behavior. Of the 10 target behaviors included in the calculations for which QABF suggested escape as the maintaining function, FAST results suggested social negative for 9 (90%) behaviors and automatic for 1 (10%) behavior. Of the 10 target behaviors included in the calculations for which QABF suggested tangible reinforcement, FAST results suggested social positive for 7 (70%) behaviors and social negative for 3 (30%) behaviors. Because the FAST does not differential between attention and tangible functions, those functions on the QABF have been combined for the final comparison. Of the 13 target behaviors included in the calculations for which the QABF suggested either an attention or tangible function, FAST suggested social positive for 8 (62%) behaviors, social negative for 5 (38%) behaviors, and automatic for 1 (8%) behavior. The bottom panel shows the results of the QABF, given specific FAST results. Of the 15 target behaviors included in the calculations for which the FAST suggested social positive reinforcement, QABF results suggested attention or tangible for 8 (80%) behaviors and automatic for 4 (40%) behaviors. Of the 9 target behaviors included in the calculations for which FAST suggested automatic reinforcement, QABF results suggested automatic reinforcement for 8 (89%) behaviors, attention for 1 (11%) behavior, and escape for 1 (11%) behavior. Of the 13 target behaviors included in the calculations for which FAST suggested social negative reinforcement, QABF results suggested escape for 9 (69%) behaviors, tangible for 3 (23%) 15

behaviors, attention for 2 (15%) behaviors, and automatic for 1 (8%) behavior. Overall, QABF and FAST showed agreement for 25 out of 30 (83.3%) behaviors as shown in Table 6. Tied Scores among Anecdotal Assessments Table 7 shows the total number of inter-rater agreements that resulted in two or more categories ranked first. For the MAS, 9 (9.09%) behaviors resulted in a tie. For the QABF, 10 (11.24%) behaviors resulted in a tie. For the FAST, 2 (5.71%) behaviors resulted in a tie. Tables 8 through 10 display the specific maintaining variables with ties for each assessment type. For the MAS, the escape/tangible categories had 33.3%, attention/automatic and automatic/escape had 22.2%, and attention/escape and 3-or-more categories had 11.1% of ties. For the QABF, attention/automatic, attention/tangible, automatic/escape, and automatic/tangible had 20% and escape/tangible and 3-or-more categories had 10% of ties. For the FAST, social positive/automatic and social positive/social negative each had 50% of ties. 16

CHAPTER 4 DISCUSSION: ANECDOTAL ASSESSMENT AGREEMENT The highest percentage of agreement between anecdotal assessment was 83.3%, between QABF and FAST. MAS and FAST showed similar results with 80% agreement. One factor which could make agreement higher when comparing the FAST to another assessment is that there are multiple ways that the FAST can produce agreement with the other assessments. For example, both the automatic positive and automatic negative categories were combined into an overarching automatic category for comparison against the other assessments. Results in that category were scored as agreement to sensory, non-social, and physical maintaining variables on the MAS and QABF. Similarly, the attention and tangible maintaining variables on the MAS and QABF were scored as agreement with the social positive category of the FAST. There were instances where each anecdotal assessment showed high agreement in specific categories of maintaining variables. However, when the comparison was reversed high levels of agreement were not always obtained. For example, 100% of behaviors for which MAS suggested escape as a maintaining variables showed correspondence with QABF and FAST results. Conversely, MAS results suggested escape for only 44% of behaviors for which QABF suggested escape and 54% of behaviors for which FAST suggested social negative reinforcement. For 100% of behaviors for which FAST suggested automatic reinforcment, MAS results also suggested automatic reinforcement. Conversely, FAST results suggested automatic for only 56% of behaviors for which MAS suggested automatic reinforcement. This pattern indicates that MAS underidentifies a maintaing variable of escape, or the QABF and FAST overidentifies this maintaining variable, or both. 17

CHAPTER 5 RESULTS: ANECDOTAL ASSESSMENT AND FA AGREEMENT MAS and FA Agreement Figure 4 shows a comparison of MAS results and FA outcomes. The top panel shows the results of the MAS, given specific FA outcomes. Of the 4 target behaviors included in the calculations for which the FA suggested attention as the maintaining consequence, MAS results suggested attention for 0 (0%) behaviors, automatic for 3 (75%) behaviors, and tangible for 1 (25%) behavior. Of the 5 target behaviors included in the calculations for which the FA suggested automatic reinforcement, MAS results suggested automatic reinforcement for 5 (100%) behaviors and attention for 1 (20%) behavior. Of the 6 target behaviors included in the calculations for which FA suggested escape as the maintaining function, MAS results suggested escape for 4 (67%) behaviors, tangible for 3 (50%) behaviors, and attention for 1 (17%) behavior. Of the 7 target behaviors included in the calculations for which FA suggested tangible reinforcement, MAS results suggested tangible for 7 (100%) behaviors. One FA did not show differentiation and was therefore classified as unclear. The MAS that corresponded to this FA suggested tangible reinforcement of the behavior. The bottom panel shows the outcomes of the FA, given specific MAS results. Of the 2 target behaviors included in the calculations for which the MAS suggested attention as the maintaining consequence, FA outcomes suggested attention for 0 (0%) behaviors, escape for 1 (50%) behavior and tangible for 1 (50%) behavior. Of the 8 target behaviors included in the calculations for which the MAS suggested automatic reinforcement, FA outcomes suggested automatic reinforcement for 5 (63%) behaviors and attention for 3 (37%) behaviors. Of the 4 target behaviors included in the calculations for which MAS suggested escape as the maintaining 18

function, FA outcomes suggested escape for 4 (100%) behaviors. Of the 12 target behaviors included in the calculations for which MAS suggested tangible reinforcement, FA outcomes suggested tangible for 7 (58%) behaviors, escape for 3 (25%) behaviors, attention for 1 (8%) behavior, and unclear results for 1 (8%) behavior. Overall, of the 23 behaviors for which an FA was conducted, MAS showed agreement for 16 (70%) behaviors. QABF and FA Agreement Figure 5 shows a comparison of QABF results and FA outcomes. The top panel shows the results of the QABF, given specific FA outcomes. Of the 4 target behaviors included in the calculations for which the FA suggested attention as the maintaining consequence, QABF results suggested attention for 2 (50%) behaviors, automatic for 1 (25%) behaviors, and tangible for 1 (25%) behavior. Of the 5 target behaviors included in the calculations for which the FA suggested automatic reinforcement, QABF results suggested automatic reinforcement for 4 (80%) behaviors, escape for 1 (20%) behavior, and tangible for 1 (20%) behavior. Of the 4 target behaviors included in the calculations for which FA suggested escape as the maintaining function, QABF results suggested escape for 4 (100%) behaviors. Of the 6 target behaviors included in the calculations for which FA suggested tangible reinforcement, QABF results suggested tangible for 4 (67%) behaviors, escape for 2 (33%) behaviors, and automatic for 1 (17%) behavior. One FA did not show differentiation and was therefore classified as unclear. The QABF that corresponded to this FA suggested escape as the maintaining consequence of the behavior. The bottom panel shows the outcomes of the FA, given specific QABF results. Of the 2 target behaviors included in the calculations for which the QABF suggested attention as the maintaining consequence, FA outcomes suggested attention for 2 (100%) behaviors. Of the 6 19

target behaviors included in the calculations for which the QABF suggested automatic reinforcement, FA outcomes suggested automatic reinforcement for 4 (67%) behaviors, attention for 1 (17%) behavior, and tangible for 1 (17%). Of the 8 target behaviors included in the calculations for which QABF suggested escape as the maintaining function, FA outcomes suggested escape for 4 (50%) behaviors, tangible for 2 (25%) behaviors, automatic for 1 (13%) behavior, and was unclear for 1 (13%) behavior. Of the 6 target behaviors included in the calculations for which QABF suggested tangible reinforcement, FA outcomes suggested tangible for 4 (67%) behaviors, attention for 1 (17%) behavior, and automatic for 1 (17%) behavior. Overall, of the 20 behaviors for which an FA was conducted, QABF showed agreement for 14 (70%) behaviors. FAST and FA Agreement Figure 5 shows a comparison of FAST results and FA outcomes. For the purposes of comparison, the attention and tangible outcomes of FA have been combined into one category, attention/tangible. The top panel shows the results of the FAST, given specific FA outcomes. Of the 6 target behaviors included in the calculations for which the FA suggested attention/tangible as the maintaining consequence, FAST results suggested attention for 3 (50%) behaviors, social negative for 2 (33%) behaviors, and automatic for 1 (25%) behavior. Of the 2 target behaviors included in the calculations for which the FA suggested automatic reinforcement, FAST results suggested automatic reinforcement for 2 (100%) behaviors. Of the 4 target behaviors included in the calculations for which FA suggested escape as the maintaining function, FAST results suggested social negative for 2 (50%) behaviors and social positive for 2 (50%) behaviors. The FAST that corresponded to this FA suggested social negative reinforcement. 20

The bottom panel shows the outcomes of the FA, given specific FAST results. Of the 5 target behaviors included in the calculations for which the FAST suggested social positive reinforcement as the maintaining consequence, FA outcomes suggested attention/tangible for 3 (60%) behaviors and escape for 2 (40%) behaviors. Of the 3 target behaviors included in the calculations for which the FAST suggested automatic reinforcement, FA outcomes suggested automatic reinforcement for 2 (67%) behaviors and attention/tangible for 1 (33%) behavior. Of the 5 target behaviors included in the calculations for which FAST suggested social negative reinforcement as the maintaining variable, FA outcomes suggested escape for 2 (40%) behaviors, attention/tangible for 2 (40%) behaviors, and was unclear for 1 (20%) behavior. Overall, of the 13 behaviors for which an FA was conducted, FAST showed agreement for 7 (54%) behaviors. 21

CHAPTER 6 DISCUSSION: ANECDOTAL ASSESSMENTS AND FA AGREEMENT Agreement between FA and other forms of functional assessment has been used in the literature as a method of validation of alternative assessments. If an alternative form of assessment demonstrates high agreement with FA, it is deemed to have high validity as a functional assessment tool. Although FA can demonstrate functional relationships between behavior and environmental contingencies, it is reasonable to argue that, ultimately, the validity of any assessment, including FA, can only be determined by its effectiveness to inform effective intervention. In the current study, all treatments that corresponded to FA outcomes were effective. Furthermore, a substantial literature exists that supports the validity of FA as a pretreatment assessment (Beavers et al., 2013; Hanley et al., 2003). Therefore, agreement between FA and anecdotal assessment results will be discussed in terms of the validity of the anecdotal assessments; however, these interpretations are somewhat tempered by potential limitations of the FA. In the current study, MAS and QABF showed agreement with FA for 70% of behaviors and FAST showed agreement for 54% of behaviors. Based on these results, the MAS and QABF assessments appear to be somewhat better than the FAST at accurately determining the maintaining variable of a target behavior. These findings are similar to those of Hall (2005) and Smith et al. (2012). There were specific maintaining variables for which the anecdotal assessment results showed differential patterns of agreement with FA outcomes. When the result of the anecdotal assessment suggested a specific maintaining variable, but the FA did not agree, the result was interpreted as a false positive for the anecdotal assessment. An example of a tendency toward 22

false positive results is the automatic category on the MAS. For 100% of behaviors for which FA indicated an automatic reinforcement account, MAS also suggested automatic reinforcement. However, FA suggested an automatic reinforcement account for only 63% of behaviors for which MAS suggested automatic reinforcement. Therefore, for 27% of behaviors, MAS appeared to produce false positive results suggesting automatic reinforcement as a maintaining variable. Similar patterns were observed between FA and anecdotal assessment results for the tangible category on MAS, the escape category on QABF, and the automatic reinforcement category on FAST. When the result of the FA suggested a specific maintaining variable but the anecdotal assessment did not agree, the result was interpreted as a false negative outcome of the anecdotal assessment. An example of a tendency toward false negative results is the escape category on the MAS. For 100% of behaviors for which MAS suggested escape as the maintaining variable, FA also suggested escape. However, MAS suggested escape for only 50% of behaviors for which FA suggested escape. Therefore 50% of MAS appeared to produce false negative results, not identifying the maintaining variable as escape when the FA results indicated an escape account. A similar pattern was observed between FA and anecdotal assessment results for the attention category of the QABF. While some categories of maintaining variables showed higher agreement between anecdotal and FA results than others, there were no categories that showed 100% agreement of FA outcome given anecdotal assessment result and 100% agreement of anecdotal assessment result given FA outcome. These results may support a recommendation that multiple types of functional assessments should be used when determining the maintaining variables of a target behavior. 23

CHAPTER 7 RESULTS: ANECDOTAL ASSESSMENT AND TREATMENT AGREEMENT MAS and Treatment Agreement Figure 7 shows a comparison of MAS results and treatment categories used to successfully reduce the target behavior. The top panel shows the results of the MAS, given specific treatment category. Of the 4 target behaviors included in the calculations for which the treatment was in the attention category, MAS results suggested attention for 2 (50%) behaviors, and automatic for 2 (50%) behaviors. Of the 3 target behaviors included in the calculations for which the treatment was in the automatic category, MAS results suggested automatic reinforcement for 3 (100%) behaviors and attention for 1 (33%) behavior. Of the 3 target behaviors included in the calculations for which the treatment was in the escape category, MAS results suggested escape for 1 (33%) behavior, tangible for 2 (67%) behaviors, and automatic for 1 (33%) behavior. Of the 8 target behaviors included in the calculations for which the treatment was in the tangible category, MAS results suggested tangible for 7 (88%) behaviors and attention for 1 (22%) behavior. The bottom panel shows the behaviors in each treatment category, given specific MAS outcomes. Of the 4 target behaviors included in the calculations for which MAS results suggested attention, treatments were categorized in attention for 2 (50%) behaviors, automatic for 1 (25%) behavior, and tangible for 1 (25%) behavior. Of the 6 target behaviors included in the calculations for which MAS results suggested automatic reinforcement, treatments were categorized in automatic for 3 (50%) behaviors, attention for 2 (33%) behaviors, and escape for 1 (17%) behavior. Of the 1 target behavior included in the calculations for which MAS results suggested escape, treatments were categorized in escape for 1 (100%) behavior. Of the 9 target 24

behaviors included in the calculations for which MAS results suggested tangible, treatments were categorized in tangible for 7 (78%) behaviors and escape for 2 (22%) behaviors. Overall, of the 18 behaviors for which a treatment was conducted, MAS showed agreement for 13 (72%) behaviors. QABF and Treatment Agreement Figure 8 shows a comparison of QABF results and treatment categories used to successfully reduce the target behavior. The top panel shows the results of the QABF, given specific treatment category. Of the 4 target behaviors included in the calculations for which the treatment was in the attention category, QABF results suggested attention for 4 (100%) behaviors. Of the 3 target behaviors included in the calculations for which the treatment was in the automatic category, QABF results suggested automatic reinforcement for 2 (67%) behaviors and tangible for 1 (33%) behavior. Of the 1 target behaviors included in the calculations for which the treatment was in the escape category, QABF results suggested escape for 1 (100%) behavior. Of the 5 target behaviors included in the calculations for which the treatment was in the tangible category, QABF results suggested tangible for 5 (100%) behaviors. The bottom panel shows the behaviors in each treatment category, given specific QABF outcomes. Of the 4 target behaviors included in the calculations for which QABF results suggested attention, treatments were categorized in attention for 4 (100%) behaviors. Of the 2 target behaviors included in the calculations for which QABF results suggested automatic reinforcement, treatments were categorized in automatic for 2 (100%) behaviors. Of the 1 target behavior included in the calculations for which QABF results suggested escape, treatments were categorized in escape for 1 (100%) behavior. Of the 6 target behaviors included in the calculations for which QABF results suggested tangible, treatments were categorized in tangible 25

for 5 (83%) behaviors and automatic for 1 (17%) behaviors. Overall, of the 13 behaviors for which a treatment was conducted, QABF showed agreement for 12 (92%) behaviors. FAST and Treatment Agreement Figure 9 shows a comparison of FAST results and treatment categories used to successfully reduce the target behavior. For the purposes of comparison, the attention and tangible treatment categories have been combined into one category, attention/tangible. The top panel shows the results of the FAST, given specific treatment category. Of the 6 target behaviors included in the calculations for which the treatment was in the attention/tangible category, FAST results suggested social positive for 5 (83%) behaviors and social negative for 1 (17%) behavior. There were no target behaviors for which a treatment in the automatic category also had FAST results indicating an automatic reinforcement function. Of the 3 target behaviors included in the calculations for which the treatment was in the escape category, FAST results suggested social negative for 1 (33%) behavior and social positive for 2 (67%) behaviors. The bottom panel shows the behaviors in each treatment category, given specific FAST outcomes. Of the 7 target behaviors included in the calculations for which FAST results suggested social positive reinforcement, treatments were categorized in attention for 5 (71%) behaviors and escape for 2 (29%) behaviors. There were no target behaviors for which a FAST outcome indicating maintenance by automatic reinforcement also had a treatment based on an automatic reinforcement function. Of the 2 target behaviors included in the calculations for which FAST results suggested social negative reinforcement, treatments were categorized in automatic for 1 (50%) behavior and attention/tangible for 1 (50%) behavior. Overall, of the 9 behaviors for which a treatment was conducted, FAST showed agreement for 6 (67%) behaviors. 26

CHAPTER 8 DISCUSSION: ANECDOTAL ASSESSMENT AND TREATMENT AGREEMENT As noted previously, perhaps the most definitive measure of the validity of any form of functional assessment is the extent to which the assessment outcomes can be used to create successful treatments. In the current study, anecdotal assessment outcomes were compared with successful treatments to determine the percentage of cases for which anecdotal assesment outcomes corresponded with successful treatment. Results showed that QABF results showed the highest correspondence with treatment, with 92% agreement. The MAS and the FAST showed 72% and 67% agreement with successful treatment categories, respectively. Based on these results, the QABF has the highest validity of the anecdotal assessments. There were specific maintaining variables for which the anecdotal assessment results showed differential patterns of agreement with treatment categories. For example, only 50% of behaviors for which MAS suggested automatic reinforcement corresponded with a treatment for automatically reinforced behavior that was successful. However, MAS results indicated automatic reinforcement for 100% of behaviors for which treatment for automatically reinforced behavior was successful. These outcomes strongly suggest that the MAS tends to produce falsepositive automatic reinforcement outcomes. A similar pattern was observed between anecdotal assessment outcomes and treatment categories for the tangible category of the QABF. Although 100% of behaviors for which the QABF suggested automatic reinforcement were successfully treated using interventions corresponding to an automatic reinforcement account, the QABF identified automatic reinforcement as a maintaining variable for only 67% of behaviors that were successfully treated using automatic reinforcement based interventions. 27

Thus, the QABF appeared to show a tendency toward false-negative outcomes relative to automatic reinforcement accounts. The QABF showed perfect agreement with the treatment in the attention and escape categories. The FAST and the MAS did not show perfect agreement with any treatment category. 28

CHAPTER 9 RESULTS: PATTERNS ACROSS BEHAVIOR TOPOGRAPHIES Patterns across Topography in Anecdotal Assessment and FA Results Figure 10 displays the MAS outcomes categorized by behavioral topography category. Topography categories are distributed vertically by category and maintaining variable categories are distributed horizontally. The upper left corner of each cell in the display contains the ratio of the frequency of cases in which the MAS identified the category corresponding to that cell. The bottom right corner of each cell contains the percentage of cases in which the MAS identified the category corresponding to that cell. Figures 10 through 17 follow this structure to display patterns across anecdotal assessments, FA, and treatment categories. Of the 3 target behaviors included in the AGP category, MAS results suggested tangible as the maintaining consequence for 2 (67%) behaviors and escape for 1 (33%) behavior. Of the 27 target behaviors included in the PAO category, MAS results suggested tangible as the maintaining consequence for 19 (70%) behaviors, attention for 4 (15%) behaviors, escape for 3 (11%) behaviors, and automatic for 2 (7%) behaviors. Of the 12 target behaviors included in the PDB category, MAS results suggested automatic for 9 (75%) behaviors, attention for 3 (25%) behaviors, and 2 (17%) behaviors. Of the 39 target behaviors included in the SIB category, MAS results suggested escape for 20 (51%) behaviors, tangible for 13 (33%) behaviors, attention for 6 (15%) behaviors, and automatic for 5 (13%) behaviors. Of the 4 target behaviors included in the STL category, MAS results suggested automatic for 4 (100%) behaviors. Of the 14 target behaviors included in the VDB category, MAS results suggested tangible for 7 (50%) behaviors, attention for 3 (21%) behaviors, automatic for 3 (21%) behaviors, and escape for 3 (21%) behaviors. 29

Figure 11 displays the QABF outcomes categorized by behavioral topography category. Of the 2 target behaviors included in the AGP category, QABF results suggested escape as the maintaining consequence for 1 (50%) behaviors and tangible for 1 (50%) behavior. Of the 24 target behaviors included in the PAO category, QABF results suggested tangible as the maintaining consequence for 12 (50%) behaviors, automatic for 8 (33%) behaviors, and attention for 5 (21%) behaviors. Of the 11 target behaviors included in the PDB category, QABF results suggested automatic for 6 (56%) behaviors, escape for 3 (27%) behaviors, attention for 2 (18%) behaviors, and tangible for 1 (9%) behavior. Of the 35 target behaviors included in the SIB category, QABF results suggested automatic for 20 (57%) behaviors, tangible for 9 (26%) behaviors, escape for 7 (21%), and attention for 5 (14%) behaviors. Of the 4 target behaviors included in the STL category, QABF results suggested tangible for 4 (100%) behaviors. Of the 14 target behaviors included in the VDB category, QABF results suggested escape for 8 (60%) behaviors, automatic for 5 (36%) behaviors, tangible for 2 (13%) behaviors, and attention for 1 (7%) behavior. Figure 12 displays the FAST outcomes categorized by behavioral topography category. Of the 1 target behavior included in the AGP category, FAST results suggested social positive reinforcement for 1 (100%) behaviors. Of the 9 target behaviors included in the PAO category, FAST results suggested social positive reinforcement for 5 (56%) behaviors, social negative for 3 (33%) behaviors, and automatic for 1 (11%) behavior. Of the 4 target behaviors included in the PDB category, FAST results suggested automatic reinforcement for 3 (75%) behaviors and social negative for 1 (25%) behavior. Of the 16 target behaviors included in the SIB category, FAST results suggested social negative reinforcement for 8 (50%) behaviors, social positive for 6 (38%) behaviors, and automatic for 4 (25%) behaviors. Of the 1 target behavior included in 30

the STL category, FAST results suggested social positive reinforcement for 1 (100%) behavior. Of the 4 target behaviors included in the VDB category, FAST results suggested social positive reinforcement for 2 (50%) behaviors, automatic for 1 (25%) behavior, and social negative for 1 (25%) behavior. Figure 13 displays the agreement across anecdotal assessments categorized by behavior topography. Of the target behaviors included in the AGP category, MAS and QABF agreed for 2 of 2 (100%) behaviors and MAS and FAST agreed for 1 of 1 (100%) behavior. Of the target behaviors included in the PAO category, MAS and QABF agreed for 17 of 24 (71%) behaviors, MAS and FAST agreed for 8 of 9 (89%) behaviors, QABF and FAST agreed for 7 of 7 (100%) behaviors, and all 3 anecdotal assessments agreed for 5 of 7 (71%) behaviors. Of the target behaviors included in the PDB category, MAS and QABF agreed for 7 of 11 (64%) behaviors, MAS and FAST agreed for 3 of 4 (75%) behaviors, QABF and FAST agreed for 3 of 4 (75%) behaviors, and all 3 anecdotal assessments agreed for 3 of 4 (75%) behaviors. Of the target behaviors included in the SIB category, MAS and QABF agreed for 27 of 35 (77%) behaviors, MAS and FAST agreed for 12 of 16 (75%) behaviors, QABF and FAST agreed for 12 of 14 (86%) behaviors, and all 3 anecdotal assessments agreed for 10 of 14 (71%) behaviors. Of the target behaviors included in the STL category, QABF and FAST agreed for 1 of 1 (100%) behavior. Of the target behaviors included in the VDB category, MAS and QABF agreed for 8 of 14 (57%) behaviors, MAS and FAST agreed for 4 of 4 (100%) behaviors, QABF and FAST agreed for 3 of 4 (75%) behaviors, and all 3 anecdotal assessments agreed for 3 of 4 (75%) behaviors. Figure 14 displays the FA outcomes categorized by behavioral topography. Of the 1 target behavior included in the AGP category, FA results suggested escape as the maintaining 31

consequence for 1 (100%) behavior. Of the 7 target behaviors included in the PAO category, FA results suggested tangible as the maintaining consequence for 3 (43%) behaviors, escape for 2 (29%) behaviors, attention for 1 (14%) behavior, and results were unclear for 1 (14%) behavior. Of the 3 target behaviors included in the PDB category, FA results suggested automatic for 2 (67%) behaviors and attention for 1 (33%) behavior. Of the 8 target behaviors included in the SIB category, FA results suggested automatic for 3 (38%) behaviors, tangible for 3 (38%) behaviors, attention for 1 (12%) behavior, and escape for 1 (12%) behavior. No FA was conducted for any behaviors in the STL category. Of the 4 target behaviors included in the VDB category, FA results suggested escape for 2 (50%) behaviors, attention for 1 (25%) behaviors, and tangible for 1 (25%) behavior. Figure 15 displays the agreement between anecdotal assessment outcomes and FA results categorized by behavior topography. Of the 1 target behavior included in the AGP category, there were no behaviors for which anecdotal assessment results agreed with the FA outcome. Of the target behaviors included in the PAO category, FA agreed with MAS for 4 of 7 (57%) behaviors, QABF for 3 of 6 (50%) behaviors, FAST for 3 of 5 (60%) behaviors, and all 3 anecdotal assessments for 2 of 4 (50%) behaviors. Of the target behaviors included in the PDB category, FA agreed with MAS for 2 of 3 (67%) behaviors, QABF for 3 of 3 (100%) behaviors, FAST for 2 of 3 (67%) behaviors, and all 3 anecdotal assessments for 2 of 3 (67%) behaviors. Of the target behaviors included in the SIB category, FA agreed with MAS for 7 of 8 (88%) behaviors, QABF for 6 of 7 (86%) behaviors, FAST for 0 of 1 (0%) behaviors, and all 3 anecdotal assessments for 0 of 1 (0%) behaviors. No FA was conducted for any behaviors in the STL category. Of the target behaviors included in the VDB category, FA agreed with MAS for 3 32

of 4 (75%) behaviors, QABF for 2 of 3 (67%) behaviors, FAST for 2 of 3 (67%) behaviors, and all 3 anecdotal assessments for 1 of 3 (33%) behaviors. Table 10 shows visual representations of the percentage of behaviors in each category of maintaining variables for anecdotal assessment results and FA outcomes across each topography category. Assessment types are distributed vertically and topography categories are distributed horizontally. The legend for the circle graphs is located at the bottom of the table. Solid gray shading indicates behaviors in the attention category, a dotted pattern indicates the escape category, black shading indicates the automatic category, and a striped pattern indicates the tangible category. For the FAST assessment, the attention and tangible categories have been combined and are indicated with a striped pattern with gray background. This display enables an overview of the distributions of hypothesized maintaining variables across topography categories, as well as agreement between anecdotal assessments and FA results as categorized by topography category. Results of assessments of AGP indicated either tangible or escape accounts across all assessments. Results for PAO were more variable; however, the distributions of maintaining variables for PAO seem most similar between the QABF and FA. Results across assessments suggest that PAO is most frequently maintained by social contingencies, with only the MAS producing outcomes suggesting automatic reinforcement accounts. Results for PDB showed substantial agreement that automatic reinforcement was identified as the likely maintaining variable in a large proportion of cases across anecdotal assessments and FAs. SIB showed perhaps the highest amount of variability in maintaining variables, both within and across assessments. SIB was the only category for which all tested contingencies were implicated within and across each type of assessment. That is, all assessment outcomes identified cases of 33

positive reinforcement (attention and/or tangible), escape, and automatic reinforcement. However, there also appeared to be substantial disagreement across assessments on maintaining variables, as evidenced by differentiation across the distributions of contingency categories. Results for STL are unique in that an FA was not conducted for any STL. The distribution of maintaining variables showed a highly idiosyncratic outcome, with 100% of MAS suggesting an automatic reinforcement account, 100% of QABF suggesting tangible reinforcement, and 100% of FAST suggesting social positive reinforcement. VDB outcomes showed general agreement across assessments that most cases of VDB were maintained by social contingencies of reinforcement. Although variability was observed across the specific type of reinforcement (positive or negative), no assessment identified automatic reinforcement as the most frequently implicated contingency. Patterns across Topography in Anecdotal Assessment Results and Treatment Categories Figure 16 displays the treatment categories used to successfully reduce challenging behavior categorized by behavioral topography category. Of the 1 target behavior included in the AGP category, the treatment category used was escape for 1 (100%) behavior. Of the 6 target behaviors included in the PAO category, the treatment category used was tangible for 4 (66%) behaviors, attention for 1 (17%) behaviors, and escape for 1 (17%) behavior. Of the 2 target behaviors included in the PDB category, the treatment category used was attention for 2 (100%) behaviors. Of the 9 target behaviors included in the SIB category, the treatment category used was tangible for 4 (44%) behaviors, automatic for 3 (33%) behaviors, attention for 1 (11%) behavior, and escape for 1 (11%) behavior. No treatments were conducted for any behaviors in the STL and the VDB categories. 34

Figure 17 displays the agreement between anecdotal assessment outcomes and treatment categories categorized by behavior topography. Of the 1 target behavior included in the AGP category, there were no behaviors for which anecdotal assessment results were consistent with the treatment category. Of the target behaviors included in the PAO category, FA agreed with MAS for 5 of 6 (83%) behaviors, QABF for 4 of 4 (100%) behaviors, FAST for 3 of 4 (75%) behaviors, and all 3 anecdotal assessments for 2 of 2 (100%) behaviors. Of the target behaviors included in the PDB category, FA agreed with MAS for 1 of 2 (50%) behaviors, QABF for 2 of 2 (100%) behaviors, FAST for 0 of 1 (0%) behaviors, and all 3 anecdotal assessments for 0 of 1 (0%) behaviors. Of the target behaviors included in the SIB category, FA agreed with MAS for 7 of 9 (78%) behaviors, QABF for 6 of 7 (86%) behaviors, FAST for 3 of 3 (100%) behaviors, and all 3 anecdotal assessments for 1 of 1 (100%) behaviors. No treatment was conducted for any behaviors in the STL and VDB categories. 35

CHAPTER 10 DISCUSSION: PATTERNS ACROSS BEHAVIOR TOPOGRAPHIES Though behavioral treatments are based on the maintaining variables of the target behavior, not on the topography of the behavior, analyzing anecdotal assessment results by topography may provide information on how maintaining variables are distributed across topographies (and vice-versa) as well as how well assessments determine the maintaining variables of various target behaviors. STL appears to provide an illustrative example of how anecdotal assessments can produce different outcomes. Whereas the MAS suggested that all tested STL was maintained by automatic reinforcement, QABF suggested tangible accounts, and FAST suggested social positive maintaining variable. Thus, although high percentages of agreement were observed within assessments, no agreement was observed across assessments. The reasons for such differences are unclear, but may be worthy of further study. While all the assessments include questions corresponding to a fairly common set of maintaining variables (social positive and negative reinforcement and automatic positive and negative reinforcement), the questions are very different across assessments. For example, whereas the QABF asks the respondent to rate engages in the behavior as a form of self-stimulation, the MAS asks the respondent to rate does it appear to you that this person enjoys performing the behavior? (e.g., it feels, tastes, looks, smells, and/or sounds pleasing) (Matson & Vollmer, 1995; Durand & Crimmins, 1988). Such differences in wording could lead respondents to rate the same behavior, such as stealing, differently on different anecdotal assessments. The topography category with the highest percent of agreement across anecdotal assessment results, FA outcomes, and treatment categories was PAO with 84% agreement. The reason for this outcome is not entirely clear; however, these results do support the notion that 36

PAO is almost always maintained by social reinforcement, with social positive reinforcement contingencies appearing to be responsible for many cases. The second highest percent of agreement across anecdotal assessment results, FA outcomes, and treatment categories was SIB with 81% agreement. Like PAO, the reason for this outcome is unclear. Only one case of AGP was included in the current assessment, so results for AGP should be interpreted with a great deal of caution. Results showed a complete absence of correspondence between either anecdotal assessment result and treatment. This occurred because the treatment was based on the FA outcome, which did not match either anecdotal assessment result. These outcomes suggest that the anecdotal assessments did not produce accurate or valid outcomes for AGP; however, it is unclear if this result would be shown across additional cases. Unfortunately, treatment data were not available for behaviors in STL and VDB categories. FA data was not available for behaviors in the STL category. Future research should examine complete sets of assessments and treatment for these topography categories so that a more informed comparison can be made. 37

CHAPTER 11 GENERAL DISCUSSION AND LIMITATIONS The purpose of the current study was to examine anecdotal assessment, FA, and treatment outcomes from multiple cases to identify patterns across behavioral function and topography. In general, the QABF demonstrated the highest agreement with FA outcomes and treatment categories used to successfully reduce the target behavior, with 70% and 92% of cases, respectively, showing correspondence. These results suggest that the QABF might be a viable alternative to an experimental functional assessment when conditions do not allow for one to be conducted. Results suggest that anecdotal assessments show some interesting distributions of maintaining variables across topographies, relative to those derived from FA and treatments. For example, if the current results for STL are shown to have generality, anecdotal assessment would be contraindicated for those cases. On the other hand, some categories of challenging behavior, such as SIB and PDB, showed greater correspondence between assessment and FA distributions, suggesting that anecdotal assessments may be useful in such cases. Further research examining the use of anecdotal assessments with behaviors that have not been extensively evaluated such as hoarding, suicidal gestures, and elopement, might provide additional information for the types of behavior for which anecdotal assessment is more or less helpful. While the results of the current study lend further support to the literature on anecdotal assessments, some limitations must be identified. One limitation is there were very few behaviors for which all 3 anecdotal assessments, FA, and treatment were conducted. Future studies should include more behaviors that have been exposed to all components of the study to 38

provide fuller and more valid comparison of agreements across assessment results and treatment categories. Another limitation is the data used in this study spanned 10 years and, therefore, some differences in FA protocols occurred over time. The primary difference in protocols was that a tangible condition was only included in the FA if the anecdotal assessments suggested a tangible maintaining variable (Rooker et al. 2011). Thus all FA protocols were not able to detect false negative results of anecdotal assessments for a tangible maintaining variable. Another limitation is that there is a large difference in the number of behaviors in each topography category. For example, 3 behaviors were included in AGP and 39 behaviors were included in SIB. A similar number of behaviors in each topography category would allow for a more accurate comparison of agreements across anecdotal assessments, FA outcomes, and treatment categories. Also, a similar number of behaviors should have all components of the study conducted so that a more informed comparison can be made. Lastly, none of the behaviors included in this study were determined to have multiple maintaining variables by FA or treatments; therefore, the reliability of anecdotal assessments to suggest multiple maintaining variables cannot be examined. For the purpose of this study, only ties on inter-rater agreement on anecdotal assessment results were evaluated. However, it may be that intra-rater ties in anecdotal assessment are associated with cases of multiple control. For one FA included in this study, the outcome was unclear because the target behavior did not occur in the assessment; furthermore, treatment based on the anecdotal assessments was not conducted. The inclusion of analyses for multiply maintained and undetermined FA results for which treatment is provided could help evaluate the usefulness of anecdotal assessments. 39

This study extends the current literature on anecdotal assessments through the inclusion of many data sets across types of assessments, permitting a more robust comparison of anecdotal assessment results, FA outcomes, and treatment categories than has been previously conducted. This study also examined assessment and treatment agreement across topography categories, which enabled a description of some apparent patterns of how maintaining variables may be distributed across types of behavior (e.g., SIB, STL, PAO) as well a description of differential outcomes across assessments. Finally, this study included a preliminary analysis of relationships between anecdotal assessment results and successful treatments of challenging behaviors (Staff, 2012 & Wolf, 2007). Future replications and extensions of this study could help produce a clearer understanding of how well various types of assessments perform with different functional and descriptive categories of behavior, and how operant functions are distributed across those descriptive categories of challenging behavior. 40

Table 1 Participant Demographics Year Admitted Age Sex 2006 2 Below-29 6 Male 27 2007 3 30-39 4 Female 17 2008 0 40-49 14 2009 25 50-59 27 2010 5 60-Above 5 2011 7 2012 1 2013 2 2014 5 2015 7 Note. Participants were included in all relevant categories when admitted into the clinic. Table 2 General Topographies, Included Behaviors, and Participant Distribution Topography Category Included Behaviors # participants % Aggression Towards Property (AGP) Includes hitting walls, destroying objects, overturning furniture, displacing materials 3 3.0% Physical Aggression Towards Others (PAO) Physically Disruptive (PDB) Self-injurious (SIB) Stealing (STL) Includes hitting, kicking, pinching, pushing, pulling, grabbing, scratching, biting, choking, spitting at, throwing items at others Includes stripping, repetitive movement, stomping, slamming doors, dropping to floor, bucking wheelchair, twirling shirt, standing within 6 inches of another person Includes hitting self, biting self, head banging, scratching self, picking at skin, PICA, rumination, hand mouthing, excessive water drinking, pulling G-tube, meal refusal, medication refusal, rectal/vaginal digging, hitting body against hard surface, head up (precursor behavior to hitting self) Includes taking food and/or objects not belonging to the person, without permission 27 27.3% 12 12.1% 39 39.4% 4 4.0% Verbally Disruptive (VDB) Includes yelling, whining, cursing, threatening statements, repetitive vocalizations 14 14.1% Note. Participants that displayed more than one behavior are included in all relevant categories. 41

Table 3 Inter-Observer Agreement (IOA) for Functional Analyses Participant Behavior(s) Evaluated Overall IOA % Sessions with IOA IOA Range 14* Self-Injurious Behavior 98% 63% 97-100 15* Aggression towards Property 95% 43% 92-100 15* Physical Aggression towards Others 95% 43% 92-100 16* Physical Aggression towards Others 99% 31% 86-100 16* Self-Injurious Behavior 99% 31% 86-100 17* Self-Injurious Behavior 99% 85% 99-100 18* Verbally Disruptive Behavior 98% 64% 89-100 19* Self-Injurious Behavior 99% 65% 97-100 20* Self-Injurious Behavior (biting self) 99% 43% 82-100 20* Self-Injurious Behavior (skin picking) 99% 43% 82-100 25 Physical Aggression towards Others 99% 30% 99-100 26 Verbally Disruptive Behavior 99% 83% 98-100 29 Physically Disruptive Behavior 99% 89% 73-100 30 Physical Aggression towards Others 99% 89% 99-100 30 Verbally Disruptive Behavior 99% 89% 99-100 33 Physically Disruptive Behavior 99% 33% 88-100 34* Physical Aggression towards Others 99% 44% 96-100 36 Physical Aggression towards Others 99% 63% 84-100 37 Physically Disruptive Behavior 96% 100% 86-100 38 Verbally Disruptive Behavior 96% 95% 94-99 39 Self-Injurious Behavior 99% 63% 91-100 41 Physical Aggression towards Others 100% 100% 100 44** Self-Injurious Behavior 99% 38% 85-100 Note. IOA calculated by dividing session into 10-s intervals. *IOA calculated by dividing session into 1-s intervals. **IOA calculated by dividing session into 1-s intervals and agreement was given 1-s tolerance window. 42

Table 4 Inter-Observer Agreement (IOA) for Treatments Participant Behavior(s) Treated Overall IOA # Sessions IOA Range with IOA 1* Physical Aggression towards Others 99% 43% 99-100 1* Self-Injurious Behavior 99% 43% 99-100 2* Self-Injurious Behavior 98% 45% 90-100 15* Aggression towards Property 98% 57% 97-99 15* Physical Aggression towards Others 98% 57% 97-99 17* Self-Injurious Behavior 100% 77% 99-100 19* Self-Injurious Behavior 97% 26% 83-100 20* Self-Injurious Behavior (biting self) 97% 22% 85-100 20* Self-Injurious Behavior (skin picking) 97% 22% 85-100 21* Physical Aggression towards Others 99% 31% 83-100 24* Physical Aggression towards Others 99% 75% 83-100 24* Self-Injurious Behavior 99% 75% 83-100 29* Physical Aggression towards Others 99% 79% 97-100 29* Physically Disruptive Behavior (2011) 99% 79% 97-100 29 Physically Disruptive Behavior (2015) 84% 23% 64-97 36 Physical Aggression towards Others 96% 30% 53-100 40 Self-Injurious Behavior 92% 25% 46-100 44** Self-Injurious Behavior 99% 38% 85-100 Note. IOA calculated by dividing session into 10-s intervals. *IOA calculated by dividing session into 1-s intervals. **IOA calculated by dividing session into 1-s intervals and agreement was given 1-s tolerance window. 43

Table 5 Treatment Categories Treatment Category Attention Automatic Escape Tangible Included Treatments Teaching an alternative behavior to gain attention Providing attention only in the absence of the target behavior Antecedent, non-contingent presentation of edible items as incompatible behavior to PICA Antecedent, non-contingent presentation of malleable item as replacement for skin picking Teaching an alternative behavior to gain a break from task Decreasing aversive qualities of environment Non-contingent access to tangible items Teaching an alternative behavior to gain tangible items Providing tangible items only in the absence of the target behavior Table 6 Overall Agreement for Anecdotal Assessments Anecdotal Assessments Ratio % MAS & QABF 60/90 66.7 MAS & FAST 28/35 80.0 QABF & FAST 25/30 83.3 Table 7 Ties on Inter-rater Agreement for Anecdotal Assessments Anecdotal Assessment Overall ratio Overall % MAS 9/99 9.09% QABF 10/89 11.24% FAST 2/35 5.71% 44

Table 8 Ties on Inter-rater Agreement for MAS by Maintaining Variables Tied Categories Ratio of Ties % of Ties Attention & Automatic 2/9 22.2% Attention & Escape 1/9 11.1% Attention & Tangible 0/9 0% Automatic & Escape 2/9 22.2% Automatic & Tangible 0/9 0% Escape & Tangible 3/9 33.3% 3+ Categories 1/9 11.1% Table 9 Ties on Inter-rater Agreement for QABF by Maintaining Variables Tied Categories Ratio of Ties % of Ties Attention & Automatic 2/10 20% Attention & Escape 0/10 0% Attention & Tangible 2/10 20% Automatic & Escape 2/10 20% Automatic & Tangible 2/10 20% Escape & Tangible 1/10 10% 3+ Categories 1/10 10% Table 10 Ties on Inter-rater Agreement for FAST by Maintaining Variables Tied Categories Ratio of Ties % of Ties Social Positive & Automatic 1/2 50% Social Positive & Social Negative 1/2 50% Social Negative & Automatic 0/2 0% 45

SIB Topography Category PDB PAO AGP Table 11 Anecdotal Assessment Results and FA Outcomes by Topography Category MAS Results QABF Results FAST Results FA Outcomes (table continues) 46

Topography Category VDB STL Table 10 (continued) MAS Results QABF Results FAST Results FA Outcomes N/A 47