Cognitive training for children with ADHD: Individual differences in training and transfer gains van der Donk, M.L.A.

Transcription

1 UvA-DARE (Digital Academic Repository) Cognitive training for children with ADHD: Individual differences in training and transfer gains van der Donk, M.L.A. Link to publication Citation for published version (APA): van der Donk, M. L. A. (2016). Cognitive training for children with ADHD: Individual differences in training and transfer gains. General rights It is not permitted to download or to forward/distribute the text or part of it without the consent of the author(s) and/or copyright holder(s), other than for strictly personal, individual use, unless the work is under an open content license (like Creative Commons). Disclaimer/Complaints regulations If you believe that digital publication of certain material infringes any of your rights or (privacy) interests, please let the Library know, stating your reasons. In case of a legitimate complaint, the Library will make the material inaccessible and/or remove it from the website. Please Ask the Library: or a letter to: Library of the University of Amsterdam, Secretariat, Singel 425, 1012 WP Amsterdam, The Netherlands. You will be contacted as soon as possible. UvA-DARE is a service provided by the library of the University of Amsterdam ( Download date: 18 Apr 2019

2

3 COGNITIVE TRAINING FOR CHILDREN WITH ADHD Individual differences in training and transfer gains Marthe van der Donk

4 The studies described in this thesis are the result of a collaboration between the Department of Child and Adolescent Psychiatry of the Academic Medical Center, University of Amsterdam, and De Bascule, academic center for child- and adolescent psychiatry. This thesis was financially supported by a grant provided by the Ministry of Education, Culture and Science according to the program Onderwijs Bewijs, project number ODP ISBN Cover design: Layout: Printed by: Ferdinand van Nispen, Citroenvlinder-dtp.nl, my-thesis.nl, The Netherlands Marthe van der Donk GVO drukkers & vormgevers B.V. Ponsen & Looijen, Ede, The Netherlands M. L. A. van der Donk All rights reserved. No part of this publication may be reproduced, in any form or by any means, without the prior permission of the author.

5 COGNITIVE TRAINING FOR CHILDREN WITH ADHD Individual differences in training and transfer gains ACADEMISCH PROEFSCHRIFT ter verkrijging van de graad van doctor aan de Universiteit van Amsterdam op gezag van de Rector Magnificus prof.dr. D.C. van den Boom ten overstaan van een door het College voor Promotie ingestelde commissie, in het openbaar de te verdedigen in de Agnietenkapel op woensdag 22 juni 2016, te uur door Marthe Léonie Ariane van der Donk Geboren te Oss

6 Promotiecommissie Promotoren Prof. dr. D.A.V. van der Leij Prof. dr. R.J.L. Lindauer Universiteit van Amsterdam Universiteit Antwerpen Co-promotoren Dr. A. Hiemstra-Beernink Dr. A.C. Tjeenk-Kalff De Bascule De Bascule Overige leden: Prof. dr. P.J.M. Prins Prof. dr. R.W.H.J. Wiers Prof. dr. A. Popma Prof. dr. A.C. Krabbendam Dr. D.I.E. Slaats-Willemse Universiteit van Amsterdam Universiteit van Amsterdam Universiteit Leiden Vrije Universiteit Karakter Faculteit der Maatschappij- en Gedragswetenschappen

7 Contents Chapter 1 General introduction 7 Chapter 2 Cognitive training for children with ADHD: A randomized controlled trial of Cogmed Working Memory Training and Paying Attention in Class 23 Chapter 3 Predictors and moderators of treatment outcome in cognitive training for children with Attention-Deficit/ Hyperactivity Disorder 51 Chapter 4 The influence of individual differences on treatment outcomes of cognitive training in a sample of children with Attention-Deficit/Hyperactivity Disorder 81 Chapter 5 Individual differences in training gains and transfer measures: An investigation of training curves in children with Attention-Deficit/Hyperactivity Disorder 109 Chapter 6 Summary and general discussion 139 Nederlandse samenvatting References Dankwoord Curricilum vitae Publications

8

9 Chapter 1 General introduction

10 General introduction Attention-Deficit/Hyperactivity Disorder (ADHD) is a developmental psychiatric disorder that has its onset in early childhood and is characterized by inattention, impulsivity and/or hyperactivity (American Psychiatric Association, 2013). The world-wide prevalence for children and adolescents is approximately 3.4% (Polanczyk, Salum, Sugaya, Caye & Rhode, 2015). Comorbidity rates are high; about % of the children with ADHD meet criteria for one or more other psychiatric diagnoses (Gillberg et al., 2004). Although the course of symptoms usually changes throughout adolescence (e.g. less hyperactivity), the disorder persists into adulthood in about two third of the cases (Turgay et al., 2012). Children with ADHD are at risk for multi-faceted problems including persistent academic underachievement and poor educational outcomes (Loe & Feldman, 2007), increased proneness to injuries and accidents (Merrill, Lyon, Baker & Gren, 2009; Pastor & Reuben, 2006), peer relational difficulties (Van der Oord et al., 2005), sleep problems (Weiss & Salpekar, 2010) and poor selfesteem (Mazzone et al., 2013). ADHD entails extensive costs to health, social care and justice systems in society (Le et al., 2014; Pelham, Foster & Robb, 2007) but if left untreated, individuals with ADHD will suffer from poorer longterm outcomes (Shaw et al., 2012). Given the persistent and broad functional impairments caused by ADHD, early effective treatment is necessary to prevent adverse outcomes in everyday life. In the last decade, there has been a tremendous shift in treatment modalities for children with ADHD. Worldwide recommended multimodal treatment approaches (Landelijke Stuurgroep Multidisciplinaire Richtlijnontwikkeling in de GGZ, 2005; Taylor et al., 2004; Wolraich et al., 2011), usually a combination of medication and behavioral treatment, are effective in reducing ADHD core symptoms (MTA Cooperative Group, 1999a; 1999b). However, these effects cannot be sustained beyond 24 months (Jensen et al., 2007) and no improvements are found in key areas of functioning in everyday life such as academic performance (Raggi & Chronis, 2006; Van der Oord, Prins, Oosterlaan & Emmelkamp, 2008a). Additionally, growing concerns regarding the use of medication such as serious side effects (Graham & Coghill, 2008) and 8

11 Chapter 1 the unknown long term effects (Berger, Dor, Nevo, Goldzweig, 2008) led to the need and search for alternative non-pharmacological treatments for ADHD. Available non-pharmacological interventions for treating ADHD includes behavioral, neurofeedback, dietary and neurocognitive interventions. Most attention has been paid to this last mentioned intervention, cognitive training, as it directly addresses the executive functions (EF) deficits associated with the causal pathways of ADHD and therefore potentially would lead to greater transfer and generalization to functioning in everyday life. However, to date the evidence of beneficial effects of cognitive training has been mixed, especially in terms of improvements in functional impairments. On the other hand though, it was also suggested that there would still be a future for cognitive training in children with ADHD if theoretical and methodological caveats in previous intervention and study designs were addressed. Therefore, next to determining the efficacy of cognitive training in children with ADHD (aim 1), the current thesis also investigates whether transfer in the academic setting (classroom behavior as well as academic performance) can be improved with an innovative classroom embedded approach (aim 2). At the start of this thesis in 2011, evidence-based and standardized interventions implemented within a school context that could both support teachers and train executive function skills of children were scarce. Moreover, the current thesis also moves beyond the simple question whether cognitive training is effective for children with ADHD and is aimed at obtaining a more finer-grained knowledge of cognitive training effects (aim 3) by addressing factors that might influence the efficacy of training such as underlying mechanisms, individual differences and training features. Figure 1, which is based on the review article of Von Bastian and Oberauer (2013), depicts an oversight of these factors and in which chapters they are addressed throughout the thesis. 9

12 General introduction Figure 1. Factors possibly influencing cognitive training outcomes General discussion Intervention specific features CHAPTER 5 Progress trained task(s) CHAPTER 3 & 4 Individual differences Specific working mechanisms General discussion Non-specific working mechanisms CHAPTER 2 Effects cognitive training Methodological confounders 10

13 Chapter 1 Cognitive training in ADHD The rationale It has been suggested that the inability of multimodal treatment approaches to improve ADHD core symptoms on the long-term and key areas of functional impairments is unsurprising as these treatments were not based on a theoretical framework of the disorder (Rapport, Orban, Kofler & Friedman, 2013). Within the quest and need for novel interventions, the focus shifted towards the potential to improve neurocognitive (or executive functions) deficits seen in ADHD as it is assumed that these deficits give rise to the behavioral symptom expression of ADHD and that these neurocognitive processes are related to key areas of functional impairment. Deficits in EFs such as response inhibition, planning and working memory have often been implicated in children with ADHD (Castellanos, Sonuga-Barke, Milham & Tannock, 2006; Wilcutt, Doyle, Nigg, Faraone & Pennington, 2005) and are thought to mediate the causal pathway of the disorder to a great extent (Barkley, 1997; Castellanos & Tannock, 2002; Sonuga-Barke, 2002). Especially working memory impairments have been associated with ADHD, with large impairments in the visuospatial domain (Martinussen, Hayden, Hogg-Johnson & Tannock, 2005) and domain-general central executive (CE) component (Kasper, Alderson & Hudec, 2012). In turn, these impairments in the CE component are associated with inattentiveness (Kofler, Rapport, Bolden, Sarver & Raiker, 2010), hyperactivity (Rapport et al., 2008), impulsivity (Raiker, Rapport, Kofler & Sarver, 2012) and social problems (Kofler et al., 2011) in children with ADHD. For a long time it was thought that working memory capacity was a fixed trait. However, building on evidence of brain plasticity from rehabilitation science, research showed that working memory capacity could be improved in children with ADHD (Klingberg, Forssberg & Westerberg, 2002). Working memory refers to the ability to actively hold in mind and manipulate information, relevant for a goal, for brief periods of time. It is a necessary mechanism for many other complex tasks such as learning, comprehension 11

14 General introduction and reasoning (Baddeley, 2007). Throughout the years many distinct theoretical working memory models have emerged. However, Baddeley s and Hitch (1974) multicomponent model has been most widely accepted and applied. According to the most recent model (Baddeley, 2000), working memory is a multimodal and hierarchical model in which the central executive governs three components. There are two domain-specific slave short term memory systems; the phonological loop and the visuospatial sketchpad. The phonological loop contains the phonological store, which can hold memory traces for a few seconds before they fade, and the articulatory rehearsal process that is analogous to subvocal speech. The visuospatial sketchpad also contains a temporary and passive storage system and is assumed to hold visual (e.g. color and shape of object) and spatial (e.g. movement of object) information. The third component, the episodic buffer, is assumed to be capable of storing information in a multi-dimensional code; thus providing a temporary interface between the two slave systems and long term memory. Working memory deficiencies are associated with poor educational progress and negative behavior in the classroom. It is estimated that about 10% of the children at school suffer from working memory deficits (Alloway, Gathercole, Kirkwood & Elliot, 2009). There are two theoretical accounts that explain why training working memory capacity should lead to improvements in academic performance (Söderqvist & Bergman-Nutley, 2015). The first account, the learning route, refers to the possibility that increases in working memory capacity (and thereby reducing cognitive load) could help children to pay more attention in the classroom and stay more focused on a task, thereby aiding to the learning process. The second account is based on the fact that working memory is directly involved in many academic skills (see Titz & Karbach, 2014 for an overview) and, if working memory deficiencies have been acting as a bottleneck, improvements in working memory capacity could influence the performance or application of already learned skills. Types of cognitive training When cognitive training is defined as the process of improving cognitive functioning by means of practice and/or intentional instruction, two types of cognitive interventions can be distinguished (e.g. Jolles & Crone, 2012; 12

15 Chapter 1 Morrison & Chein, 2011; Rapport et al., 2013). The first type of cognitive training involves a so called process-based and domain general approach in which cognitive abilities are practiced implicitly by repeated practice. Treatment literature for ADHD is mostly focused on this first type of intervention. A crucial assumption for these type of interventions is that extensive training strengthens the common and overlapping neural EF network which in turn leads to improvements in untrained tasks or activities that rely on the same neural network (Klingberg, 2010). Usually these interventions target one single cognitive ability such as working memory (e.g. Klingberg et al., 2002) or attention (e.g. Tamm, Epstein, Peugh, Nakonezny & Hughes, 2013). However, there are also interventions that target several executive functions (e.g. Dovis, van der Oord, Wiers & Prins, 2015). Despite this broad range of interventions, most of them are aimed at improving working memory capacity originating from a broad scope of neuroimaging studies that have shown that neural mechanisms such as dorsolateral prefrontal and parietal association cortices (which partly overlap with the prefrontal regions implicated in ADHD pathology) can be altered after working memory training (e.g. Olesen, Westerberg & Klingberg, 2004). The second type of cognitive training involves a compensatory based approach in which the cognitive strengths of the individual are emphasized. Most compensatory interventions contain some sort of strategy training in which children learn to use strategies such as repeatedly rehearse tobe-remembered information, creating a story or sentence from words or generate a visual image (Holmes, Gathercole & Dunning, 2010). Teaching children to use strategies should happen explicitly and intensively during a period in which the use of strategies can be automated. But most importantly it should be aimed at improving metacognition which means that children should learn why, when and how to use a specific strategy (Meltzer, 2014 in Goldstein & Naglieri, 2014). Children with learning disabilities or working memory problems seldom select and apply an effective strategy when a situation warrants its use, probably because these children are deficient in metacognition (Dehn, 2008). Teaching strategies to children could also invoke changes in social emotional well-being as children will gain more 13

16 General introduction insight in their cognitive strengths and weaknesses and become part of the solution to remediate deficits. They learn that they can exert influence on situations, which could result in the child feeling more empowered (Otero, Barker & Naglieri, 2014). Another way to compensate for cognitive deficits, next to providing strategies, is adapting the environment of the child. This might include changing the physical or social environment (i.e. seating arrangement in the classroom), modifying tasks (i.e. shorter or more explicit) or changing the way adults interact with children (Dawson & Guare, 2014, in Goldstein & Naglieri, 2014). For instance, increasing teacher awareness of working memory problems and encouraging them to adapt their approach to teaching could help to reduce the working memory loads in the classroom (Holmes, Gathercole & Dunning, 2010). Near and far transfer effects The key question for both types of interventions is whether practice with certain skills induces near transfer (i.e. improvement in tasks that rely on identical cognitive processes that are targeted by the intervention) and far transfer (i.e. improvements in tasks and domains other than the trained process). Cogmed Working Memory Training (CWMT: Klingberg, 2002) is one of the most widely implemented and investigated cognitive training paradigms in children with ADHD. Studies that investigated the efficacy of CWMT in children with ADHD showed that children improved on trained working memory tasks (Chacko, Bedard et al., 2014; Gray et al., 2012, Green et al., 2012; Hovik, Saunes, Aarlien & Egeland, 2013; Klingberg et al., 2002; Klingberg et al., 2005) and untrained working memory tasks (Holmes, Gathercole, Place et al., 2010; Hovik et al., 2013). Treatment effects were also found on measures of attention (Klingberg et al., 2002; Klingberg et al., 2005), parent ratings of ADHD related behavior (Beck, Hanson, Puffenberger, Benninger & Benninger, 2010; Klingberg et al., 2005) and parent ratings of executive functioning (Beck et al., 2010). In the few studies that have also taken into account academic outcome measures (e.g. Chacko, Bedard et al., 2014; Egeland et al., 2013; Gray et al., 2012, Green et al., 2012), treatment effects were found on off task behavior (Green et al., 2012) and reading (Egeland et al., 2013). 14

17 Chapter 1 Despite these promising results, several meta-analyses (Cortese et al., 2015; Orban, Rapport, Friedman & Kofler, 2015; Rapport et al., 2013) have shown that the evidence of beneficial effects on behavior, academic and non-trained cognitive skills of cognitive interventions such as CWMT are insufficiently supported. It has been suggested that results from previous studies should be interpreted with caution due to both theoretical and methodological flaws such as lack of consistency in methodological experimental methods, use of single tasks as evidence for change of abilities and lack of alignment between hypothesized models of therapeutic benefit and outcomes. Moreover, the most frequently addressed methodological issue concerns the use of an inadequate control group (Chacko et al., 2013; Melby-Lervåg & Hulme, 2013; Morrison & Chein, 2011; Shipstead, Hicks & Engle, 2012; Shipstead, Redick & Engle, 2010; Shipstead, Redick & Engle, 2012). Within the scope of CWMT effect studies in children with ADHD, some studies have used non-active (e.g. waiting list, treatment as usual) control groups (Beck et al., 2010; Egeland et al., 2013; Hovik et al., 2013) which overcomes simple test-retest effects (Morrison & Chein, 2011; Shipstead, Hicks et al., 2012), but hinders blinding (Sonuga-Barke, Brandeis, Holtmann, & Cortese, 2014). Others (Green et al., 2012; Klingberg et al., 2002; Klingberg et al., 2005; van Dongen-Boomsma, Vollebregt, Buitelaar, & Slaats-Willemse, 2014) used lowdemand, non-adaptive placebo versions of the active condition, which required considerably less time and effort than the active condition and also diminished the amount and quality of interaction with the training aide (most often a parent) and CWMT coach (Chacko et al., 2013). Other shortcomings from most previous studies included the lack of a broad range of functional outcomes and long-term follow-up measures (Cortese et al., 2015; Sonuga- Barke et al., 2014). Especially in terms of academic outcome measures, the inclusion of long term assessment is crucial as the child will need to exploit his or her improved working memory capacity and this will only become visible after a lengthy period (Gathercole, 2014). Conclusively, although near and far transfer results were inconsistent and generally unsupported, it was also suggested that there still was potential 15

18 General introduction therapeutic utility for cognitive training in children with ADHD if caveats in intervention and study designs were addressed. In terms of designing new approaches to training, it was suggested that transfer could possibly be improved if conventional working memory training is followed by a period of training for transfer that provides practice in applying new skills and strategies acquired through intensive training to more practical working memory taxing situations such as the classroom (Gathercole, 2014). Additionally Chacko, Kofler and Jarrett (2014) hypothesized that new-generation neurocognitive interventions together with adjunctive skill-based approaches could ameliorate the behavioral, academic and interpersonal manifestations of the disorder. Importantly, they also suggested that maximal outcomes most likely require adult-mediated (e.g. parent and teachers) supportive instructional and behavioral skills practice in context such as a classroom or home setting. They suggested that these improved neurocognitive interventions may provide the cortical foundation to improve the children s ability to benefit from the adjunctive skill-based intervention. Development of a new intervention Paying Attention in Class Need for novel intervention In the last couple of years, there has been a great increase of children with EF related behavior and learning difficulties (such as children with ADHD) in regular educational settings in the Netherlands. At the start of this thesis in 2011, several interventions that could potentially improve EFs in children were available. However, as aforementioned, evidence of effects on far transfer such as academic outcome measures was scarce. Additionally, many of the available interventions were implemented at home while in the meantime there was a great need for evidence-based guidelines to properly support these children in the classroom. Evidence-based and standardized interventions implemented within a school context that could both support the teacher and the child with EF problems were scarce. Therefore, our research group developed a new training called Paying Attention in Class (PAC) which contains a process-based working memory training with 16

19 Chapter 1 an additional innovative classroom embedded compensatory approach that actively involves the teacher. To a great extent, the content of this new intervention is line with the recommendations regarding new approaches to training that were recently made by others (Gathercole, 2014; Chacko, Kofler & Jarrett, 2014). Content of the new training The PAC intervention contains three key elements. First of all, the intervention offers psycho-education about five executive functions that are important in a learning situation: attentional control, planning skills, working memory, goal-directed behavior and metacognition. The psycho-education is offered through an audio-book, using a brain castle metaphor, in which children are introduced to the so called brain guards (i.e. strategies such as repeat instruction or visualize) or brain bandits (i.e. pitfalls such as distraction or acting to fast). Every session the audio-book ends with a different cue (depending on which executive function was discussed), for example I repeat what is said. Second, the intervention contains three paper and pencil adaptive working memory tasks: a visual spatial span task, a listening recall span task, and an instruction paradigm task (30 trials in total), which are practiced on a daily basis in order to improve working memory capacity. In the listening recall task, the coach reads aloud a certain amount of sentences and the child has to evaluate whether the particular sentence is true or false. After this, the child has to reproduce the last word of each sentence in the correct order. The visual spatial span task is a paradigm of the Corsi block-tapping task (Corsi, 1972), which consists of a template with ten small blocks. The child has to tap the same cubes as the coach in the reversed sequence. The instruction task was based on a previously described analog task (Gathercole, Durling, Evans, Jeffcock & Stone, 2008) and consists of a paper template and cards that contains pictures of school related items. The coach reads aloud an instruction that the child has to execute, for example Point to the big circle and pick up the small blue pen. Each working memory task ends after ten executed trials. At the end of each session, the child fills out a high score list for each task to keep track of his performance. 17

20 General introduction The third key element of the intervention is the central role of facilitating generalization to the classroom-situation. First of all, the strategies and pitfalls introduced through the audio-book described above are illustrated and practiced by performing school related tasks such as arithmetic, in a workbook during the session. The second way to improve generalization to the classroom is realized by a registration card which the child brings back to class. This card contains the cue of the day (for example, I repeat what is said ) and is meant to remind the child of the requirement of practicing the cue in the classroom. The card also informs the teacher about the cue so that he/she can monitor or stimulate the child to practice. Finally, the teacher is closely involved in the process by informing him/her of the protocol and by giving him/her an active part in the process. Teachers receive a written manual, which contains information about how to recognize EF problems in the classroom and information about the intervention itself. Furthermore, they are asked to report daily whether the child applies the cue in class through structured observation forms. A more extensive description of the intervention can be found in the manual (Van der Donk, Tjeenk-Kalff & Hiemstra-Beernink, 2015). Individual differences in training and transfer gains As was previously mentioned in this chapter, several review studies have indicated both theoretical and methodological shortcomings that possibly contribute to the inconsistent findings in transfer measures. However, little to no attention has been paid to the potential influence of individual differences on treatment outcomes. By merely looking at the differences between pretest and posttest performance, as was the case in most studies so far, important information of individual differences in training and transfer gains was neglected. Given the complex clinical and pathophysiological heterogeneity of ADHD (Willcutt et al., 2012), it is quite plausible that certain subgroups or individuals with ADHD benefit more from cognitive training than others however this area has been left unexplored so far. 18

21 Chapter 1 Predictors and moderators One way of obtaining more knowledge about the influence of individual differences on treatment outcomes is by identifying which baseline variables could predict or moderate treatment outcome. This could improve both the efficacy and effectiveness of treatment in real-world clinical settings as specific treatments could be given to specific subgroups of children under select treatment contexts, so that any one form of treatment will have its maximum impact (Prins, Ollendick, Maric & MacKinnon, 2015). In addition to providing guidelines for clinicians in terms of treatment decision making, identifying moderators will also help to clarify the best choice of inclusion - or exclusion criteria or the best choice of stratification to maximize power for future randomized controlled trials (Kraemer, Wilson, Fairburn & Agras, 2002). There is cumulating evidence for the importance of recognizing the role of individual differences in cognitive training. Variables such as age, genetic predisposition, motivation, personality traits and initial cognitive ability have been shown to influence treatment outcome measures in adult non ADHD samples (Von Bastian & Oberauer, 2013; Titz & Karbach, 2014; Jaeggi, Buschkuehl, Shah & Jonidas, 2014; Karbach & Unger, 2014). Two accounts have been proposed to explain the individual differences in training related performance gains. First, the magnification account (also known as the Matthew effect) assumes that individuals who are already performing very well will also benefit most from cognitive interventions as they have more efficient cognitive resources to acquire and implement new strategies and abilities. Second, the compensation account assumes that high performing individuals will benefit less from cognitive interventions, because they already function at the optimal level which leaves less room for improvement. In contrast, low-performing individuals will benefit more from cognitive training as there is more room for improvement for them. Evidence points in the direction for a magnification effect for strategy based interventions and a compensation effect for process-based interventions (for overview see Titz & Karbach, 2014; Karbach & Unger, 2014). 19

22 General introduction Although studies have established predictor and moderator variables for medication and behavioral interventions for children with ADHD (Hinshaw, 2007; MTA cooperative group, 1999b; Owens et al., 2003), the field of cognitive training interventions falls behind in this area. So far it has only been suggested that variables such as prior treatment, motivation, use of medication or initial working memory skills could be predictors or moderators of treatment (Shah, Buschkuehl, Jaeggi & Jonides, 2012; Rutledge, van den Bos, McClure & Schweitzer, 2012; Chacko et al., 2013; Shinaver, Entwistle & Söderqvist, 2014). Mediators Although identifying predictors and moderators is an important first step in improving our understanding of the influence of individual differences on treatment outcomes, it still does not tell us by which mechanisms those variables exert their influence. Therefore, another important factor to take into account is performance gain during training. For example, it has been shown that individual differences in training gains can moderate transfer effects for typically developing children (Jaeggi, Buschkuehl, Jonides & Shah, 2011) and children with intellectual disabilities (Söderqvist, Nutley, Ottersen & Klingberg, 2012). More specifically, learning curves of individuals should be taken into account as the learning curve of the group can be distorted if there is large variability in learning rate (Jolles & Crone, 2012). Given the complex clinical and pathophysiological heterogeneity of ADHD (Willcutt et al., 2012) it is plausible to assume that these learning rates also vary greatly for children with ADHD. However to our knowledge, this has not been investigated so far in children with ADHD who followed a cognitive intervention. Aims and outline of this thesis The first aim of this thesis is to determine whether cognitive training, in terms of near and far transfer measures, is effective for school-aged children with ADHD. Second, it investigates whether transfer to the academic setting can be improved with an innovative classroom embedded approach. Third, we wanted to obtain finer-grained knowledge of of factors that might influence 20

23 Chapter 1 the efficacy of training such as underlying mechanisms, individual differences and training features. To this end, four empirical studies were conducted. The aim of the first study, described in chapter 2, was to replicate and extend previous studies of Cogmed Working Memory Training (CWMT) in children with ADHD. The effects on neurocognitive functioning, academic performance, behavior in class, behavior problems and quality of life were determined directly after treatment and six months after treatment. These effects were compared with the effects of a new executive function compensatory intervention called Paying Attention in Class (PAC). One hundred and five children diagnosed with ADHD (both medicated and medication naïve) between the age of 8 and 12 years were randomly assigned to CWMT or PAC. Based on the sample of our randomized controlled trial, chapter 3 describes the results of a study that was aimed at investigating whether certain subgroups of children could benefit more from cognitive training in general (i.e. identifying predictor variables) or would be more likely to benefit from one treatment over another (i.e. identifying moderator variables). Outcome measures included neurocognitive assessment, parent and teacher rated questionnaires of executive functioning behavior and academic performance. Use of medication, comorbidity, subtype of ADHD and initial verbal - and visual working memory skills were considered potential predictors or moderator variables. The study described in chapter 4 is aimed at extending our understanding of the individual differences in both near and far transfer treatment outcomes measures that were observed for the new PAC intervention in our randomized controlled trial. Therefore, an additional group of 116 children with ADHD between the age of 8 and 12 years received the PAC intervention. We investigated which demographical, clinical and baseline neurocognitive characteristics could predict individual treatment response six months after treatment, based on a clinical significant improvement in working memory. Additionally, we investigated whether this clinically significant improvement 21

24 General introduction in working memory was a prerequisite to obtain improvements in far transfer measures of neurocognitive functioning, academic performance, behavior in class, behavior problems and quality of life. Non-responders were offered additional CWMT which was followed with an extra assessment of neurocognitive functioning and academic performance. Chapter 5 focuses on the individual differences in learning curves of the process-based working memory training component of the PAC intervention. It was investigated how these individual learning curves influenced transfer measures and whether certain baseline variables (age, intelligence, parentrated externalizing behavior problems and presence of a learning disability) could predict those learning curves. Based on the same sample of children described in chapter 4, a latent growth curve model (LGCM) analysis was performed. Working memory skills (near transfer) and academic performance measures (far transfer) were assessed before and directly after treatment. Finally, the main findings of each chapter in this thesis are summarized in chapter 6, followed by a general discussion with clinical implications and directions for future research. 22

25 Chapter 2 Cognitive training for children with ADHD: a randomized controlled trial of Cogmed Working Memory Training and Paying Attention in Class Marthe van der Donk, Anne-Claire Hiemstra-Beernink, Ariane Tjeenk-Kalff, Aryan van der Leij & Ramón Lindauer Frontiers in Psychology, 2015, 6, 1-13.

26 Cognitive training for children with ADHD: A randomized controlled trial Abstract The goal of this randomized controlled trial was to replicate and extend previous studies of Cogmed Working Memory Training (CWMT) in children with ADHD. While a large proportion of children with ADHD suffer from academic difficulties, only few previous efficacy studies have taken into account long term academic outcome measures. So far, results regarding academic outcome measures have been inconsistent. Hundred and two children with ADHD between the age of 8 and 12 years (both medicated and medication naïve) participated in current randomized controlled trial. Children were randomly assigned to CWMT or a new active combined working memoryand executive function compensatory training called Paying Attention in Class. Primary outcome measures were neurocognitive functioning and academic performance. Secondary outcome measures contained ratings of behavior in class, behavior problems and quality of life. Assessment took place before, directly after and six months after treatment. Results showed only one replicated treatment effect on visual spatial working memory in favor of CWMT. Effects of time were found for broad neurocognitive measures, supported by parent and teacher ratings. However, no treatment or time effects were found for the measures of academic performance, behavior in class or quality of life. We suggest that methodological and non specific treatment factors should be taken into account when interpreting current findings. Future trials with well-blinded measures and a third no treatment control group are needed before cognitive training can be supported as an evidence-based treatment of ADHD. Future research should put more effort into investigating why, how and for whom cognitive training is effective as this would also potentially lead to improved intervention- and study designs. 24

27 Chapter 2 Introduction Attention-Deficit/Hyperactivity Disorder (ADHD) is a developmental psychiatric disorder that has its onset in early childhood and is characterized by inattention, impulsivity and/or hyperactivity (APA, 2000). Multimodal treatment approaches, for instance psycho stimulant medication in combination with behavioral treatment, are recommended (Taylor et al., 2004). Despite the fact that this multimodal approach has been shown to be effective in reducing ADHD symptoms (Van der Oord, Prins, Oosterlaan & Emmelkamp, 2008a; MTA Cooperative Group, 1999a), it seems that these effects can not be sustained beyond 24 months (Jensen et al., 2007). Furthermore in regard to stimulant medication, some children experience serious side effects (Graham & Coghill, 2008) and there is growing concern among parents about the unknown long term effects (Berger, Dor, Nevo & Goldzweig, 2008). Finally, it has been shown that current multimodal approach does not lead to improvements in academic performance (Van der Oord et al., 2008a; Raggi & Chronis, 2006), a key area of functioning in every day life which is often disturbed in children with ADHD (Loe & Feldman, 2007). These limitations have led to a growing demand for alternative nonpharmacological interventions for children with ADHD. Of great interests are interventions that target the underlying cognitive deficits which are assumed to mediate ADHD causal pathways. Targeting those underlying cognitive deficits would potentially lead to greater transfer and generalization to functioning in every day life (Sonuga-Barke, Brandeis, Holtmann & Cortese, 2014). Within the domain of cognitive interventions, working memory (WM) training has received most attention as a potential effective intervention for children with ADHD for several reasons. First of all, WM (i.e., the function of actively holding in mind and manipulating information relevant to a goal) is a necessary mechanism for many other complex tasks such as learning, comprehension and reasoning (Baddeley, 2007). Second, it is assumed that WM deficits are part of the causal pathway to ADHD symptoms (Barkley, 1997;Willcut, Doyle, Nigg, Faraone & Pennington, 2005). It is estimated that 81% of children with ADHD have a deficit in the working 25

28 Cognitive training for children with ADHD: A randomized controlled trial component (central executive) of working memory (Rapport, Orban, Kofler & Friedman, 2013), in contrast to the less impaired memory component (phonological and visuospatial storage/rehearsal). One of the most widely implemented and investigated interventions that targets WM is Cogmed Working Memory Training (CWMT). The rationale behind this training is that by adaptively and intensively training both the storage and storage plus manipulation components of WM, improvements will transfer to other cognitive functions such as attention as a function of underlying overlapping neural networks (Klingberg, 2010). So far, nine studies (Klingberg, Forssberg & Westerberg, 2002; Klingberg et al., 2005; Holmes et al., 2010; Gray et al., 2012, Green et al., 2012; Chacko, Bedard et al., 2014; Hovik, Saunes, Aarlien & Egeland, 2013; Egeland, Aarlien & Saunes, 2013; van Dongen-Boomsma, Vollebregt, Buitelaar, & Slaats-Willemse, 2014) that investigated the efficacy of CWMT in children with ADHD reported neurocognitive outcome measures. Six of these studies showed treatment effects on trained working memory tasks (Klingberg et al., 2002; Klingberg et al., 2005; Gray et al., 2012, Green et al., 2012; Chacko, Bedard et al., 2014; Hovik et al., 2013) and two studies have also shown treatment effects on untrained working memory tasks (Holmes et al., 2010; Hovik et al., 2013). Within the literature this latter often refers to near transfer, i.e. improvement in untrained tasks that rely on identical cognitive processes that are targeted by the intervention. Furthermore, treatment effects have also been found on measures of attention (Klingberg et al., 2002; Klingberg et al., 2005), parent ratings of ADHD related behavior (Klingberg et al., 2005; Beck, Hanson, Puffenberger, Benninger & Benninger, 2010) and parent ratings of executive functioning (Beck et al., 2010). It has been suggested (e.g. Klingberg, 2010) that this should be interpreted as evidence for far transfer, i.e. improvements in tasks that tap cognitive processes other than the trained process. Despite these promising results, there are several meta-analyses (Melby-Lervåg & Hulme, 2013; Rapport et al., 2013; Cortese et al., 2015) that are skeptical about the putative effects of working memory interventions such as CWMT, mainly regarding the far transfer measures such as academic performance. 26

29 Chapter 2 Interestingly, within the scope of CWMT efficacy studies in children with ADHD, only few have also taken into account academic outcome measures (Gray et al., 2012, Green et al., 2012; Chacko, Bedard et al., 2014; Egeland et al., 2013). This is remarkable both from a scientific and clinical perspective, as interventions that can alleviate the encountered academic problems for children with ADHD are needed. Up till now, studies that did investigate the effects on academic performance found treatment effects on off task behavior (Green et al., 2012) and reading (Egeland et al., 2013). Despite these promising results and on the other hand the critical notes from previous meta-analyses (Melby-Lervåg & Hulme, 2013; Rapport et al., 2013; Cortese et al., 2015), we do suggest that replication of previous CWMT studies in children with ADHD is necessary. There is still no consistent pattern of results, mainly in regard to far transfer measures such as academic performance. It has been noted that previous effect studies suffered from both theoretical and methodological flaws and several suggestions have been made to optimize future research. The most frequently addressed methodological issue concerns the use of an inadequate control group (Chacko et al., 2013; Morrison & Chein, 2011; Shipstead, Redick & Engle, 2010; Shipstead, Redick & Engle, 2012; Shipstead, Hicks & Engle, 2012; Melby-Lervåg & Hulme, 2013). Within the scope of CWMT effect studies in children with ADHD, some studies have used non-active (e.g. waiting list, treatment as usual) control groups (Hovik et al., 2013; Egeland et al., 2013; Beck et al., 2010) which hinders blinding (Sonuga-Barke et al., 2014) and only overcomes simple test-retest effects (Morrison & Chein, 2011; Shipstead, Hicks et al., 2012). Others (Klingberg et al., 2002; Klingberg et al., 2005; Green et al., 2012; van Dongen-Boomsma et al., 2014) used lowdemand, non-adaptive placebo versions which require considerably less time and effort then the active condition which also diminishes the amount and quality of interaction with the training aide (most often a parent) and CWMT coach (Chacko et al., 2013). Furthermore, in regard to academic outcome measures in previous CWMT studies in children with ADHD, only the study of Egeland and colleagues (2013) included long term assessment. Gathercole (2014) recently suggested that long term assessment of standardized 27

30 Cognitive training for children with ADHD: A randomized controlled trial academic ability tests are crucial as the child will need to exploit his or her improved WM capacity and this will only be visible after a lengthy period. Others (Sonuga-Barke et al., 2014; Cortese et al., 2015) also suggested that future trials should include a broader range of functional outcomes and longterm follow-up. In current study we will replicate and, moreover, extend previous CWMT studies in children with ADHD between the age of 8 and 12 years by investigating the effects on neurocognitive functioning, academic performance, behavior in class, behavior problems and quality of life. As has been suggested (Chacko et al., 2013; Morrison & Chein, 2011; Shipstead et al., 2010; Shipstead, Hicks et al., 2012; Shipstead, Redick et al., 2012; Melby-Lervåg & Hulme, 2013), we will compare these effects with an active control group whose experience is closely matched to the training group in terms of effort (adaptive WM tasks in response to performance), time (equal interaction time with the coach) and performance related feedback. This active control group receives a cognitive training called Paying Attention in Class which was developed by the authors. This training consists of a working memory - and a compensatory executive function training. Next to adaptive WM tasks, this intervention also targets a broader set of executive functions that are impaired in children with ADHD with a main focus on how to use those executive functions in the classroom. The following research questions were addressed in this study: (1) What are the effects of CWMT on measures of neurocognitive functioning, academic performance, behavior in class, behavior problems and quality of life? and (2) Is an active control intervention equally effective as CWMT? Method Participants Children were recruited in two different ways for this study. First, clinical care providers from two clinical care departments of the De Bascule (Academic Centre for Child and Adolescent Psychiatry, Amsterdam) referred eligible children to the researcher. Second, healthcare staff members (usually 28

31 Chapter 2 remedial teacher or school psychologist) of schools in the region of Amsterdam contacted the researcher when they had eligible children. In both cases, the researcher visited the school for an information meeting to extensively inform the staff members. Parents of children who met criteria for participation were approached and informed by the school staff member. Eligible participants were (a) children between the age of 8 and 12 years, (b) diagnosed with Attention-Deficit/Hyperactivity Disorder by a professional according to the guidelines of the Diagnostic and Statistic Manual of Mental Disorders DSM-IV (APA, 2000). Children with comorbid Learning Disabilities (LD) and/or Oppositional Defiant Disorder (ODD) were also included. Children on medication were only included when they were well adjusted to their medication, which meant that they were not participating in a medication trial, and type and dosage of medication was unchanged at least 4 weeks prior to the start and during the training. Exclusion criteria were (a) presence of psychiatric diagnoses other than ADHD/LD/ODD, (b) Total Intelligence quotient < 80, (c) significant problems in the use of the Dutch language and (d) severe sensory disabilities (hearing/vision problems). Parents filled out an application package containing a written informed consent form, questionnaires of demographic- and background information and the Dutch translation of the Social Communication Questionnaire (SCQ) (Warreyn, Raymaekers & Roeyers, 2004) to screen for autism spectrum disorder. The Lifetime version of the SCQ consists of 40 questions that have to be answered with yes or no. A total raw score of 15 or higher indicates a likelihood of the presence of autism spectrum disorder and is recommended as a cutoff-score. Children with a total score of 15 or higher were excluded from this study. The Attention/Hyperactivity, Oppositional Defiant Disorder and Conduct Disorder modules of the Diagnostic Interview Schedule for Children IV (DISC-IV) (Steenhuis, Serra, Minderaa & Hartman, 2009) were administered by the research assistant(s) by telephone to confirm ADHD diagnose and to rule out for potential Conduct Disorder. Parents were also asked to send a copy of the diagnostic psychiatric report of their child to establish the subtype of ADHD and rule out other potential 29

32 Cognitive training for children with ADHD: A randomized controlled trial psychiatric problems that met exclusion criteria. The expert view, based on the diagnostic psychiatric report, was leading for establishing the subtype of ADHD. If the subtype was not described in the report, we used the Attention/Hyperactivity module of the DISC-IV (Steenhuis et al., 2009) to establish the subtype. A short version of the WISC-III-nl (Wechsler, 2005) with the subtests Similarities, Block Design, Picture Completion and Vocabulary was administered to estimate the Total Intelligence quotient if there were no prior recordings available. At baseline, there were no significant differences between the two groups for the demographical and clinical characteristics (Table 1) except for type of education. The Paying Attention in Class group contained significantly more children from special primary schools (e.g. children with mild learning- or behavior difficulties) but no children from special education schools (e.g. children with severe behavior or psychiatric problems). Procedure The ethics approval for this study was obtained from the Medical Ethical Committee (2011_269) at the Academic Medical Centre in Amsterdam, the Netherlands. After enrollment children were randomly allocated to either the Cogmed Working Memory Training or the experimental Paying Attention in Class intervention by a researcher independent of the research team. The Clinical Research Unit of the Academic Medical Centre composed a randomisation list, stratified by age (8 to 10 years and 11 to 12 years) with a block size of six. The independent researcher assigned the children in predetermined random order and 1:1 allocation. Subsequently, the independent researcher informed the training aides and Cogmed coach about the allocated condition for each child. Parents and teachers were not explicitly informed about the allocation, however the interventions were so dissimilar in appearance and application that parents and teachers can not be marked as blind raters. Prior to treatment they were invited to participate in an information meeting at school where they were informed about the contents of the interventions. Two to three week prior to treatment, parents and teachers received the questionnaires mentioned above via or hard copy on request. One week prior to treatment, a member 30

33 Chapter 2 of the research team (who was blind for the allocation) administered the neuropsychological tasks from each child at a silent (if available) room at school. Post treatment assessment took place within one week after the last training session and follow-up assessment took place after six months. The treatment sessions were completed during morning school hours, aligned with teachers, for both intervention groups. Training periods were planned in between school holidays so that training sessions would not be interrupted for a longer period of time. Children in both intervention groups received daily small rewards such as stickers or extra playtime from the coach. In addition, they received a small present (e.g. pencil or toy) after each week of training, regardless their improvements in trained tasks. Table 1. Demographic and clinical characteristics CWMT (n=50) PAC (n=50) p (t, X² or Fisher s exact test) Age, M (SD) in years 9.8 (1.3) 10.0 (1.3) ns Gender Male, n (%) 35 (70) 37 (74) ns Full-Scale IQ, M (SD) (15.1) 99.2 (12.9) ns Medication for ADHD, n (%) 26 (55.3) 29 (61.7) ns ADHD diagnose, n (%) Combined Inattentive Not Otherwise Specified Comorbid disorders, n (%) Dyslexia Dyscalculia Oppositional Defiant Disorder Enrollment, n (%) Clinical care School Type of education, n (%) Regular primary Special primary Special education SES, n (%) Low < Average High > Ethnicity, n (%) Mother Dutch Father Dutch 29 (58) 15 (30) 6 (12) 8 (21.1) 0 2 (5.3) 7 (14) 43 (86) 44 (88) 2 (4) 4 (8) 10 (24.4) 6 (14.6) 25 (61) 41 (87.2) 35 (76.1) 35 (70) 10 (20) 5 (10) 15 (35.7) 2 (4.8) 0 14 (28) 36 (72) 43 (86) 7 (14) 0 6 (13.6) 12 (27.3) 26 (59.1) 36 (73.5) 31 (63.3) ns ns ns X² (2) = 6.789, p Note.CWMT = Cogmed Working Memory Training;PAC = Paying Attention in Class;SES = social economic status ns ns ns 31

34 Cognitive training for children with ADHD: A randomized controlled trial Interventions Cogmed Working Memory Training Cogmed Working Memory Training is a computerized training program aimed to train working memory. It consists of a variety of game-format tasks that are adaptive, which means that difficulty level is being adjusted automatically to match the working memory span of the child on each task. The program includes 12 different visuospatial and/or verbal working memory tasks, eight of these tasks (90 trials in total) are being completed every day (Klingberg et al., 2005). Children followed the standard CWMT protocol which means following the computer training program for 5 weeks, 5 times a week, approximately 45 minutes a day. The program was provided via the internet on a laptop in a separate room. Children were trained individually at school, guided by a trained developmental psychologist (training aid) who was supervised by a certified Cogmed Coach. Teachers were invited to attend an information meeting in which the content of CWMT was explained by first author, it was communicated that teachers did not have an active role during treatment if children received CWMT. Paying Attention in Class Paying Attention in Class (PAC) is an experimental combined working memory- and compensatory training that has been developed by members of our research team. Children are trained individually outside the classroom for 5 weeks, 5 times a week, approximately 45 minutes a day; the same duration as in the CWMT protocol. This PAC intervention contains three key elements; first of all, this intervention offers psycho education about executive functions that are related to classroom behavior. By making children more aware of these executive functions needed for adequate classroom behaviors, they obtain more insight in their own learning behavior. The psycho education addresses five executive functions, based on information processing and are important in a learning situation namely: attentional control, planning skills, working memory, goal-directed behavior and metacognition. For each executive function, five sessions in the protocol are devoted to that topic. For instance in regard to attentional control, it is 32

35 Chapter 2 explained to children that sitting straight in your chair or taking a deep breath might help to focus on the task. The psycho education is offered through an audio-book, with a brain castle metaphor. It is explained that only by following the right journey (first pay attention, make a plan, remember the task etc) in your head, i.e brain castle, you will manage to finish a task in the classroom. During this journey, the audio-book introduces them to the so called brain guards (i.e. strategies such as repeat instruction or visualize) or brain bandits (i.e. pitfalls such as distraction or acting to fast). The brain castle and it s guards and bandits are also visualized with drawings, plastic cards and stickers. Every day the audio-book ends with a different cue (depending on which executive function is discussed), for example I repeat what is said. This cue will be repeated throughout the session by the coach if necessary and the cue has to be practiced within a neuropsychological - and school task related exercise. Second, this intervention contains three paper and pencil adaptive working memory tasks: a visual spatial span task, a listening recall span task, and an instruction paradigm task (30 trials in total) which are practiced on a daily basis to improve working memory capacity. The sequence of each trial is extended after two correct trials. In the listening recall task, the coach reads aloud a certain amount of sentences and the child has to evaluate and tell whether the particular sentence is true or false. After this, the child has to reproduce the last word of each sentence in the correct order. The visual spatial span task is a paradigm of the Corsi block-tapping task (Corsi, 1972) which consists of a template with ten small blocks. The child has to tap the same cubes as the coach but then in the reversed sequence. The instruction task was based on a previously described analog task (Gathercole, Durling, Evans, Jeffcock & Stone, 2008) and consists of a paper template and cards that contains pictures of school related items. The coach reads aloud an instruction that the child has to execute for example Point to the big circle and pickup the small blue pen. For each next level one action or one extra item was added so the next sentence could be Pickup the large yellow book and a scissor and put them on the small square. Each working memory task was ended after ten executed trials. At the end of each session, the child fills out a high score list for each task to keep track of their performance. 33

36 Cognitive training for children with ADHD: A randomized controlled trial The third key element of this intervention is the central role of optimizing generalization to the classroom-situation. First of all, the strategies and pitfalls introduced through the audio-book described above will be illustrated and practiced by performing school related tasks, such as arithmetic, in a workbook during the session. The coach stimulates the child to use the cue from the audio-book and the coach also monitors whether the child uses any of the brain guards or whether the child encounters brain bandits. Performance on these school related tasks is not important, in stead reflection on the process is stimulated by the coach. The second way to improve generalization to the classroom is realized by a registration card which the child brings back to class. This card contains the cue of the day (for example, I repeat what is said ) and is meant to remember the child to practice the cue in the classroom. It will also inform the teacher about the cue so that he/she can monitor or stimulate the child to practice. Finally, we closely involved the teacher in the process by informing him/her with the protocol and by giving him/her an active part in the process. Teachers received a written manual, which contained information about how to recognize working memory problems in the classroom and information about the intervention itself. Furthermore, they were asked to daily record whether the child applied the cue in class through structured observation forms. The structured observation forms contained four specific statements, for instance The child is able to repeat the instruction, that had to be rated on a 4 point Likert scale. Subsequently, the coach reviewed this observation form the next day which gave the coach information whether the child visibly applied the cue in the classroom. Standardization interventions Developmental psychologists were trained as training aides according to the CWMT protocol (Gerrits, van der Zwaag, Gerrits-Entken & van Berkel, 2012) and also trained as therapists for the Paying Attention in Class intervention. During an interactive three hour course, provided by a member of the research team, the developmental psychologists were introduced in the theoretical background and practical implications of both interventions. The Paying Attention in Class intervention consists of a written manual for 34

37 Chapter 2 the trainer with clear instructions for each task/component and daily score sheets for the working memory tasks. Since the psychologists trained both children in the CMWT group as children in the PAC group, they were asked not teach the specific PAC skills to the children (i.e. not apply the psycho education) in the CWMT group. A total of thirty-one psychologists and five CWMT coaches were deployed in this study. Treatment adherence For both interventions the developmental psychologists received weekly supervision by a certified Cogmed Coach and clinical staff member of the Bascule in which they discussed the progress and clinical difficulties. Also the trainers filled out a daily diary per child for observations and special circumstances. Finally the Cogmed Training Web and the Paying Attention in Class workbook were used to monitor the results of the training. These three documents were used to create a checklist for evaluating treatment compliance. Measures Neurocognitive assessment and academic performance were the primary outcomes of this study. Behavior in class, behavior problems and quality of life were the secondary outcome measures. Assessment took place at school in a separate room at three consecutive moments: at baseline, directly after treatment and six months after treatment. Compliance For both groups, we used the number of completed training sessions and improvements on the trained tasks as a measure for compliance. Treatment compliance was defined as completing twenty or more sessions, as has been reported in previous studies (Klingberg et al., 2005). For the individuals in the CWMT group, we used the Improvement Index as a measure of improvement on trained tasks. This index is generated by the program and reflects the difference between the Start Index (mean of three best trials on day two and three of the training based on two tasks) and the Max Index (mean of the best three trials on the best two days of training based on two tasks). For 35

38 Cognitive training for children with ADHD: A randomized controlled trial the individuals in the PAC group we reported three different improvement indexes namely a visual spatial index, a listening recall index and an instruction index, referring to the improvements on the three trained tasks. Primary outcomes Neurocognitive assessment included tasks that measure attention (Creature Counting and Score!: Manley, Robertson, Anderson & Nimmo-Smith, 2004), verbal working memory (Digit Span:Wechsler, 2005; Comprehension of Instruction and Word List Interference: Zijlstra, Kingma, Swaab & Brouwer, 2010), visual spatial working memory (Span Board: Wechsler & Naglieri, 2008), planning skills (Six Part Test BADS-C: Tjeenk-Kalff & Krabbendam, 2006) and inhibition (Inhibition:Zijlstra et al., 2010). Finally, parents and teachers filled out the Dutch version of The Behaviour Rating of Executive Functions (BRIEF) questionnaire (Smidts & Huizinga, 2009). This questionnaire consists of 75 items which can chart the following executive functions: inhibition, shifting, emotional control, initiation, working memory, planning and organization, organization of materials and monitoring. These clinical scales form two broader indexes: the Behavioral Regulation Index (i.e. the scales Inhibit, Shift and Emotional Control) and the Metacognition Index (i.e. the scales Initiate, Working Memory, Plan/Organize, Organization of Materials and Monitor). An overall score, the Global Executive Composite, can also be calculated. T-scores of 65 and above are considered as a clinical score. Academic performance was measured with tests for word reading fluency, automated math and spelling. Word reading fluency was measured with the Een Minuut Test (Brus & Voeten, 1973), this test consists two parallel cards which each hold 116 words. The child receives the instruction to read out loud (fast and accurate) as many as possible words in one minute. The Tempo Test Automatiseren (De Vos, 2010) was used to measure the degree of automated math. The test consists of four subtests: addition, subtraction, multiplication and division calculations. For each subtest, the child has to make as many as possible sums in two minutes with a maximum of 50. The PI dictee (Geelhoed & Reitsma, 1999) was used to measure spelling skills and consists of two parallel versions (A & B). Each version consists of 135 words that are divided in nine blocks of fifteen words each. For each word, 36

39 Chapter 2 a sentence is read aloud and the child is asked to write down the repeated word. From a time-saving point of view, not all blocks were administered. The starting point was the educational age of the child and if there were three or more mistakes in that block, the previous block was also administered. The test was ended if the child made eight or more mistakes in one block. All raw scores were converted into a Learning Efficiency Quotient (educational age equivalent divided by the educational age) which allows for comparison across grade and age. We also performed secondary analysis in terms of accuracy (% correct) for the word reading fluency and automated math task as these tasks had a time restriction. We calculated an accuracy score for each point in time by dividing the raw scores of correct answers through the raw scores of total amount of produced words or sums and multiplying this answer by 100. As we had no Learning Efficiency Quotient scores for these raw scores, we added a variable age at assessment as a covariate in the model for analysis. Secondary outcomes Behavior in class was reported by the teacher using the Learning Condition Test: this is a 70 item questionnaire that measures Direct Learning Conditions (concentration, motivation, work rate, task orientation, working according to a plan, persistency) and Indirect (social orientation, social position in class and relationship with peers and teacher) Learning Conditions (Scholte & Van der Ploeg, 2009). Items can be rated on a 5 point Likert scale, a high score indicates a negative prognoses. Behavior problems were assessed by both teacher and parents using The Child Behavior Checklist for Ages 6-18 (Verhulst, van der Ende & Koot, 1996) and Teacher s Report Form for Ages 6-18 (Verhulst, van der Ende & Koot, 1997). We reported the scale Attention Problems since improved attention is one of the putative transfer effects of working memory training; a T-score of 65 and above is considered as problematic. We also reported the scale Externalizing Problems which consist of the two problem-scales rule breaking behavior and aggressive behavior; a T-score of 60 is considered as problematic. 37

40 Cognitive training for children with ADHD: A randomized controlled trial Quality of Life was measured with the Dutch translation of the Kidscreen-27 questionnaire (Ravens-Sieberer et al., 2007) and was completed by parents and the child. It covers five dimensions of quality of life: physical well-being, psychological well-being, autonomy & parents relations, social support & peers and school environment. The raw scores are converted into T-scores: a higher score reflects a higher quality of life. Statistical methods The Intention-To-Treat (ITT) approach was used to compare treatment effects. The Statistical Package for Social Sciences, version 19 (IBM SPSS 19), was used for the statistical analysis. Demographic and clinical characteristics were analyzed with independent t-tests for continuous variables and Chi-square and Fisher exact tests for dichotomous variables. Outliers were removed if they had a z-score of < or > 3.29 and were replaced with the second highest value. A linear mixed model was used for each outcome variable as a function of Time, Condition and Time-by-Condition interaction. Secondary analyses were performed with age and gender as covariates. Missing data was considered missing at random and was not imputed because using linear mixed model analyses has the benefit of using every observation for each participant if a baseline score is present. The covariance type for each outcome measure was based on the smallest Akaike s Information Criterion. The significance level was set at p =.05 (two-tailed). A Bonferroni correction was performed to evaluate the effect of multiple testing which resulted in a significance level of p =.003 for the neurocognitive outcome measures (n=15) and a significance level of p =.005 for the academic performance measures (n=11). In addition to these analyses, Cohen s d was calculated as an effect size by subtracting the difference between groups for the change scores (post baseline and follow up baseline for both groups), dividing that by the pooled standard deviations of both groups at baseline. A paired samples t-test was conducted on the mean scores of the Start- and Max Index to test whether the children in the CWMT improved significantly on the improvement index. Paired samples t-tests were also conducted for the visual spatial index, listening recall index and instruction index for the children in the PAC group. Independent t-tests at baseline showed that groups did not differ 38

41 Chapter 2 on any of the outcome measures prior to treatment, however there was a trend for Spelling p =.057 possibly due to the fact that were almost twice as much children with Dyslexia in the PAC condition. The difference in Dyslexia between the two groups was non-significant however. Results Between January 2012 and May 2013, a total of 115 children were assessed for eligibility; ten children were excluded because they did not meet inclusion criteria or for other reasons (Figure 1). One hundred and five children were included and randomized, 52 children were allocated to the CWMT and 53 were allocated to the PAC intervention. Three children from the PAC intervention group and two children from the CWMT group did not start treatment after allocation because either they met exclusion criteria after all or they were included in a different research project due to time scheduling problems. This resulted in 50 children starting with CWMT and 50 children starting with PAC. Compliance measures Of the 50 children who followed Cogmed Working Memory Training, 47 children (94.%) met the compliance criteria of twenty or more complete sessions. Paired samples t-test showed that children in the CWMT group improved significantly on the Improvement Index with a mean Max Index of (SD=12.71) and a mean Start Index of (SD=9.26), t(49)= , p <.001. Of the 50 children who followed the Paying Attention in Class training, 46 workbooks were available for analysis of compliance. Forty-two children (91.3 %) met the compliance criteria of twenty or more complete sessions (i.e. psycho education, tasks in workbook and working memory tasks). Paired samples t-test showed that children improved significantly on the visual spatial index with a mean of 3.5 (SD=.74) at the start of training and a mean of 5.42 (SD=1.35) at the end of training, t(47)= , p <.001. Children also improved significantly on the listening recall index with a mean of 2.45 (SD=.72) at the start of training and a mean of 4.40 (SD=1.21) at the 39

42 Cognitive training for children with ADHD: A randomized controlled trial end of training, t(46)=11.758, p <.001. Finally, children improved significantly on the instruction index with a mean of 3.54 (SD=1.01) at the start of training and a mean of 8.29 (SD=1.96) at the end of training, t(47)=18.24, p <.001. Figure 1. CONSORT flow diagram Enrollment Assessed for eligibility (n=115 ) Excluded (n=10) Not meeting inclusion criteria (n=7) Other reasons (n=3) Randomized (n=105) Allocation Allocated to Cogmed (n=52) Received Cogmed (n=50) Did not receive allocated intervention (n=2) 1 participant was included in other project on own request and 1 participant met exclusion criteria Allocated to Paying Attention in Class (n=53) Received Paying Attention in Class (n=50) Did not receive allocated intervention (n=3) 2 participants were included in other project on own request and 1 participant met exclusion criteria Follow-Up Lost at follow-up at 6 months (n=0) Discontinued intervention (n=3) treatment was too demanding Lost to follow-up at 6 months (n=1) Discontinued intervention (n=1 ) treatment was too demanding Analysed (n=50) Analysis Analysed (n=50) Primary outcomes Neurocognitive assessment. As can be seen in Table 2, a significant effect of time at post treatment was found for attention (Creature Counting, correct answers; p <.001), verbal working memory (Word List Interference Remember; p <.001, Comprehension of Instruction; p <.001), visual spatial 40

43 Chapter 2 working memory (Span Board; p <.001), inhibition (Inhibition correct answers; p <.001 and time; p <.001), parent rated Behavioral Regulation Index (p =.003) and Metacognition Index (p <.001). A significant effect of time at post treatment for Score! (sustained attention) was also found, however this was a decrease. At follow-up, significant effects of time were found for verbal working memory (Word List Interference Remember; p <.001, Comprehension of Instruction; p <.001), visual spatial working memory (Span Board, p <.001), planning (Six Part Test; p <.001), inhibition (Inhibition correct answers; p <.001 and time; p <.001) and teacher rated Metacognition Index (p =.003). A significant group effect was found for the Span Board task (p <.001, d 1 = 0.87; d 2 = 0.49) in favor of CWMT. An interaction effect was also found for the Span Board task (p <.001). When the forward and backward condition for the Span Board task were analyzed separately, results showed that there was only a significant group (p <.001) and interaction (p <.001) effect for the Forward condition. Academic performance. There were no significant time, group or interaction effects on the Learning Efficiency Quotient scores of word reading fluency (table 3). Results showed one effect of time at follow up for the subtest division of the automated math task (p =.005), however this was a decrease of performance. It should be noted here that sample size of the multiplication and division subtests at baseline was a lot smaller than the sample size of the multiplication and division subtests at follow up. The subtests multiplication and division were not administered for children in lower grades as they do not acquire these multiplication and division skills yet. Results revealed a trend group effect (p =.036) and trend effect of time at follow up (p =.045) for spelling. As children in the CWMT group already performed better at baseline, we suspected that Dyslexia moderated the results. When Dyslexia was entered in the model as a covariate, the trend effect of group was no longer present (p =.150). 41

44 Cognitive training for children with ADHD: A randomized controlled trial Table 2. Results on neurocognitive assessment Baseline Post-treatment Follow-up CWMT PAC CWMT PAC CWMT PAC p Effect time pre-post p Effect time pre-fu p Effect group p Interaction effect d 1 PAC) d 2 (CWMT- (CWMT- PAC) Score! a Creature Counting Correct Time a b.015 b Digit Span b.004 b.009 b.018 b Span board a.000 a.000 a.000 a WLI Repeat Remember b.000 a.005 b a Six part test b.000 a COI a.000 a Inhibition switching Mistakes Time a.000 a.000 a.000a BRIEF parents BRI MCI a a.033 b BRIEF teacher BRI MCI b a Note. CWMT=Cogmed Working Memory Training; PAC=Paying Attention in Class; WLI=word list interference; COI=comprehension of instruction; BRIEF=Behaviour Rating of Executive Functions; BRI=Behavioral Regulation Index; MCI=Metacognition Index. Raw scores where used for amount of correct answers and time for the Inhibition task; Span board and BRIEF scores are expressed in T-scores; all other scores are expressed in standard scores. d 1 = difference between groups for the change scores post to baseline for both groups, divided by the pooled standard deviations of both groups at baseline. d 2 = difference between groups for the change scores follow up to baseline for both groups, divided by the pooled standard deviations of both groups at baseline. a p <.003 (significant after Bonferroni correction) b p <.05 42

45 Chapter 2 For the accuracy scores (see table 4) results showed a significant group effect in favor of CWMT (p =.003) on word reading fluency, but without a significant interaction effect (p =.312). Further inspection of the data revealed that children from the CWMT group already significantly performed better at baseline (p =.004) than the children in the PAC group possibly due to the fact that were almost twice as much children with Dyslexia in the PAC condition. We therefore again entered Dyslexia as a covariate in the model and found that the group effect was no longer significant (p =.046) after Bonferroni correction. Finally, we found no significant time, group or interactions effects for the accuracy scores of the automated math task. Secondary outcomes Behavior in class. Analyses for the Direct Learning Condition scale showed no significant effects of time (post treatment; p =.395, follow-up; p = 1.000), group (p = 0.060) or interaction (p = 0.068). Non-parametrical tests were performed for the Indirect Learning Conditions scale since data was not equally distributed. We only found a significant decrease for the CWMT group from pre treatment (M = 60.23) to follow-up (M = 57.27), p =.022. However, this decrease was not significantly different from the PAC group (p =.975). Behavior problems. Parent ratings of Attention Problems showed a significant effect of time at post treatment (p <.001) and follow-up (p <.001). There was no significant group (p =.593) or interaction effect (p =.138). The parent rated scale of Externalizing Problems also showed a significant effect of time at post treatment (p <.001) and follow-up (p <.001) but no significant group (p =.627) or interaction effect (p =.243). Teacher rated Attention Problems also showed a significant effect of time at post treatment (p =.007) and follow-up (p =.001) but no significant group(p = 0.149) or interaction effect (p =.558). No significant time, group or interaction effect was found for the scale Externalizing Problems as rated by teachers. Quality of Life. We found no significant time, group or interaction effects for any of the five dimensions of quality of life that were rated by parents or the child. 43

46 Cognitive training for children with ADHD: A randomized controlled trial Table 3. Learning efficiency quotients of academic performance measures Baseline Post-treatment Follow-up CWMT PAC CWMT PAC CWMT PAC p Effect time pre-post p Effect time pre-fu p Effect group p Interaction effect d 1 PAC) d 2 PAC) WRF Automated math Addition Subtraction Multiplication Division a Spelling b.036 b Note. CWMT = Cogmed Working Memory Training; PAC = Paying Attention in Class; WRF = Word reading fluency. All scores are expresses in a learning efficiency quotient. d 1 = difference between groups for the change scores post to baseline for both groups, divided by the pooled standard deviations of both groups at baseline. d 2 = difference between groups for the change scores follow up to baseline for both groups, divided by the pooled standard deviations of both groups at baseline. a p <.005 (significant after Bonferroni correction) b p <.05 Table 4. Accuracy scores of Word reading fluency and Automated math Baseline Post-treatment Follow-up CWMT PAC CWMT PAC CWMT PAC p Effect time pre-post p Effect time pre-fu p Effect group p Interaction effect d 1 PAC) d 2 (CWMT- (CWMT- (CWMT- (CWMT- PAC) WRF a Automated math Addition Subtraction Multiplication Division Note. CWMT = Cogmed Working Memory Training; PAC = Paying Attention in Class; WRF = Word reading fluency. All scores reflect the percentage of correct answers. d 1 = difference between groups for the change scores post to baseline for both groups, divided by the pooled standard deviations of both groups at baseline. d 2 = difference between groups for the change scores follow up to baseline for both groups, divided by the pooled standard deviations of both groups at baseline. a p <.005 (significant after Bonferroni correction) 44

47 Chapter 2 Discussion The aim of this study was to replicate and extend previous studies of CWMT in school-aged children with ADHD. This was the first randomized controlled trial that contained an active control group in which children received adaptive WM tasks in response to performance, equal interaction time with the coach and performance related feedback. Therefore, in contrast to previous effect studies of CWMT in children with ADHD, the experiences of the trained and control group were more similar in terms of effort and expectations in current study. Another strong aspect of current study was the fact that, next to broad neurocognitive measures, it included long term (six months) assessments of areas that reflect functioning in everyday life i.e. academic performance, behavior in class, behavior problems and quality of life in a noteworthy large sample. Although results showed an effect of time on verbal WM, attention, inhibition, planning, parent and teacher ratings of executive functioning and ADHD related behavior, no superior effect of CWMT was found on these measurements in comparison to the effects of the PAC intervention. No significant time or treatment effects were found for academic performance, behavior in class and quality of life. We were only able to replicate one treatment effect on visual spatial WM as was also found by previous efficacy studies of CWMT in children with ADHD (Klingberg et al., 2002; 2005; Gray et al., 2012; Hovik et al., 2013). Our results showed that the treatment and interaction effect was only apparent for the Forward condition of the Spatial Span task which suggests that CWMT only had a superior effect on short term memory in comparison to the PAC intervention, as was previously pointed out by Rapport and colleagues (2013). Most trained tasks within CWMT contain visual spatial (working) memory elements which strongly resembles the Spatial Span task that was used for the assessment of visual spatial working memory. In contrast, the PAC intervention contains only one trained task that resembles the Spatial Span task. Therefore we suggest that this treatment effect should be viewed as a practice effect and not a measure of (near) transfer. We were not able to replicate treatment effects that were previously 45

48 Cognitive training for children with ADHD: A randomized controlled trial found on verbal working memory (Holmes et al., 2010; Hovik et al., 2013), measures of attention (Klingberg et al., 2002; Klingberg 2005; Egeland et al., 2013), parent ratings of ADHD (Klingberg et al., 2005; Beck et al., 2010) and executive functioning (Beck et al., 2010) and measures of academic performance (Green et al., 2012; Egeland et al., 2013). We suggest that there are several explanations for the fact that current study could not replicate treatment effects of CWMT that were found in previous studies. First of all, regarding the neurocognitive measures, we suggest that the difference in control groups added to these inconsistencies. For instance, previous studies have used no-contact control groups such as treatment as usual (Hovik et al., 2013; Egeland et al., 2013) which corrects for testretest effects. However, it does leave the possibility open that the trained and control group approached the post assessment differently in terms of motivation (Shipstead, Hicks et al., 2012). This same argument also accounts for the studies that used low-demand, non adaptive control groups (Klingberg et al., 2002; Klingberg et al., 2005). Improvements on post training measures might reflect the belief that training should have a positive influence on cognition (Morrison & Chein, 2011). It is questionable whether the use of a low-demand, non adaptive control group sufficiently convinces participants that they are engaged in cognitive training (Shipstead, Hicks et al., 2012). As results did indicate effects of time, we suggest that non specific treatment factors partially might explain current findings. We suggest that positive reinforcement during training should be considered as a plausible mechanism. Next to models that view executive dysfunction as a causal model for ADHD, there are also models that emphasize the sub-optimal reward systems (delay aversion/motivational style) as a second and co-occurring causality for ADHD (Sonuga-Barke, 2003). Dovis and colleagues (2012) showed that incentives significantly improved WM performance of children with ADHD and the intensity of the incentive determined the persistence of performance over time. In our study, children in both groups received performance related feedback during training and were encouraged during performance. In addition, they received daily small rewards at the end of each session (e.g. stickers or playtime) and a small present on a weekly basis. It is plausible that 46

49 Chapter 2 the encouragements and incentives obtained during training altered their motivation in regard to performance. Despite the strong design of current study, it should be noted that this study did not contain a no treatment control group (e.g. waiting list) as a third arm for allocation. Therefore we can not rule out other possible confounders such as test-retest effects, passage of time or therapeutic benefit. Choosing and developing control groups remains challenging for future trials as ethical constraints make it difficult to implement no treatment groups and there still is no consensus about how a control group should be designed (Von Bastian & Oberauer, 2013). Regarding the results on academic outcome measures, we suggest that the heterogeneity of the used samples make it difficult to interpret results across CWMT studies. For instance, while current study included both inattentive and combined subtype children, others (Egeland et al., 2013) only included children with the combined subtype. Another factor that could contribute to the inconsistencies in results concerns the inclusion of children with comorbid learning difficulties. For instance, just as current study, Gray and colleagues (2013) used a sample of children with comorbid learning disabilities, others (Chacko, Bedard et al., 2014; Egeland et al., 2013) did not report whether they included children with comorbid learning difficulties. Recently it has been suggested (Sonuga-Barke et al., 2014) that the response to different forms of training should be compared between clinical subtypes and neuropsychological subgroups. Furthermore, we suggest that future research should pay closer attention to individual differences such as age, biological factors, personality and initial cognitive ability as these factors have been mentioned as potential moderators of treatment effect (Von Bastian & Oberauer, 2013; Jaeggi, Buschkuehl, Jonidas & Shah, 2011; Jolles & Crone, 2012). For instance, it was suggested that WM training might be more effective for subgroups of ADHD, for instance ADHD plus WM problems (Chacko et al., 2013). This would reflect the room for improvement hypothesis in which children with a lower ability at the start of training (for instance WM) show larger improvement on training gains as there would be more room for improvement than children with more normal ability levels who will reach their ceiling capacity much faster. A study of Holmes and colleagues (2009) 47

50 Cognitive training for children with ADHD: A randomized controlled trial might support this view as they showed that mathematical ability improved in children with low WM skills after following WM training. Next to paying more attention to individual differences we also suggest, in line with current comments of Gathercole (2014), that future research should take a closer look into how to assess academic performance. Many previous studies contained standardized ability tests for complex skill domains such as reading and mathematics. According to Gathercole (2014) the problem with these standardized ability tests is that they tap cumulative achievements which makes them strongly dependent on prior learning and relatively insensitive to recent changes in learning capacities. Determining the true and distinctive effect of training in terms of academic outcome measures remains challenging as there is one complicating factor that is often overlooked. While test-retest effects and maturation (passage of time) are often taken into account, it is much harder to control for the potential new skills that children have been exposed to in between assessment periods. In addition, children in lower grades are most likely more frequently exposed to new skills during a certain time period in comparison to children in higher grades. One possible way to overcome this problem is by following the example of a study from Holmes and Gathercole (2013). They used National Curriculum assessments in English and math to calculate the sublevel improvements for the relevant academic year. Conclusively, despite the fact that our results are in line with most recent meta-analyses (Rapport et al., 2013; Cortese et al., 2015), we suggest that more information can be gained from future trials if individual differences and solid academic outcomes measures are taken into account. Finally, regarding the effects on parent and teachers ratings of ADHD related behavior and executive functioning, we again suggest that the difference in control groups added to the inability to replicate treatment effects of previous CWMT studies. It has been previously suggested that non-adaptive placebo control interventions (e.g. Klingberg et al., 2005) require considerably less time and effort from the coach (usually parent) than active conditions. This has direct implications for interpreting parent-rated improvements as it diminishes the quantity and quality of parent-child interaction (Chacko et al., 48

51 Chapter ). Also, studies that used non-active (e.g. waiting list, treatment as usual) control groups (Beck et al., 2010) might have created bias as these type of control groups hinder blinding (Sonuga-Barke et al., 2014). It is possible that post-test change may reflect expectations that were created by the act of receiving treatment rather than actual changes that were brought about by treatment (Morrison & Chein, 2011; Shipstead, Hicks et al., 2012). In current study, parents were not involved in the delivery of the interventions and the interaction time with the coach was equal for children in both groups. Therefore, we suggest that treatment effects on parent ratings of ADHD (Klingberg et al., 2005; Beck et al., 2010) and executive functioning (Beck et al., 2010) in previous studies should be interpreted with caution. However, although parents were not actively informed about treatment allocation in current study they can not be considered objective raters as it was communicated that both interventions were active. A meta-analysis of Sonuga-Barke and colleagues (2013) showed that effects of ADHD ratings after cognitive interventions dropped to non significant if outcomes of probably blinded raters were considered. This same argument might also explain current effects of time on teacher ratings. Both interventions were delivered at school during school hours so teachers were reminded on a daily basis that children were receiving treatment. Furthermore, teachers were invited to attend an information meeting that contained information about working memory problems in the classroom and information about the interventions. From a clinical perspective, we can only encourage the involvement of teachers in such intensive interventions. However from a scientific point of view it remains challenging how to incorporate teachers perspective. We suggest that future studies should incorporate classroom observation rated by blinded and objective persons. As was suggested by Green and colleagues (2012), teachers are probably less objective as they already formed a general impression of the behavior patterns of a child and they may not be sensitive in detecting positive changes. Conclusions In summary, when compared to an active intervention, a superior effect of CWMT could only be found on a trained visual spatial working memory 49

52 Cognitive training for children with ADHD: A randomized controlled trial task. Although children in both groups improved on broad measures of neurocognitive functioning supported by both parent and teacher ratings, these results should be interpreted with caution as they might be related to methodological and non specific treatment factors. We suggest that future trials with well-blinded measures, a third no treatment control group and adequate (far) transfer measures are needed before cognitive training can be supported as an evidence-based treatment of ADHD. Furthermore, we suggest that future studies should be aimed at gaining more insight in why and how cognitive training is effective with possible support from neuroimaging studies. This might shed some light on the question why some of the transfer measures are improved and others are not and may subsequently lead to improved intervention designs. Another important area to explore regards the area of who could benefit most from cognitive training. This concern would be of high clinical value in terms of treatment adherence, financial resources and effort resources from children, parents, teachers and health care professionals. 50

53 Chapter 3 Predictors and moderators of treatment outcome in cognitive training for children with Attention-Deficit/Hyperactivity Disorder Marthe van der Donk, Anne-Claire Hiemstra-Beernink, Ariane Tjeenk-Kalff, Aryan van der Leij & Ramón Lindauer Journal of Attention Disorders, 2016, 1-14.

54 Predictors and moderators of treatment outcome Abstract The aim of this study was to explore whether clinical variables and initial cognitive abilities predict or moderate (far) transfer treatment outcomes of cognitive training. A total of 98 children (aged 8-12 years) with ADHD were randomly assigned to Cogmed Working Memory Training or a new cognitive training called Paying Attention in Class. Outcome measures included neurocognitive assessment, parent and teacher rated questionnaires of executive functioning behavior and academic performance. Predictor/ moderator variables included use of medication, comorbidity, subtype of ADHD and initial verbal - and visual working memory skills. Parent and teacher ratings of executive functioning behavior were predicted and moderated by subtype of ADHD. Word reading accuracy was predicted by subtype of ADHD and comorbidity. Use of medication and initial verbal - and visual spatial working memory skills only predicted and moderated near transfer measures. Cognitive training can be beneficial for certain subgroups of children with ADHD, individual differences should be taken into account in future trials. 52

55 Chapter 3 Introduction The last decade cognitive training has become a popular nonpharmacological intervention for children with Attention-Deficit/Hyperactivity Disorder (ADHD). Despite the large amount of effect studies of cognitive training in children with ADHD, there still is no clear consensus about the effects of cognitive training. Especially effects in terms of far transfer measures, i.e. improvements in tasks that tap cognitive processes other than the trained process, are disputed (Rapport, Orban, Kofler & Friedman, 2013; Cortese et al., 2015). Methodological issues such as inadequate and varying control groups, inadequate measurements of abilities, large variability of assessed skills and varying treatment protocols complicate the interpretation of (far) transfer effects (Morrison & Chein, 2011; Shipstead, Redick & Engle, 2012). However, given the clinical and pathophysiological heterogeneity of ADHD, there is also growing acknowledgment that the inconsistencies in far transfer effects might be due to the fact that only certain subgroups of children with ADHD benefit from cognitive training. Investigating which patients can be expected to benefit most from cognitive training in general (i.e. identifying predictor variables) and which patients would be more likely to respond to one treatment over another (i.e. identifying moderator variables) could provide guidelines for clinicians in terms of treatment decision making (Kraemer, Wilson, Fairburn & Agras, 2002). We recently investigated the efficacy of Cogmed Working Memory Training (CWMT), compared to an active control group that received a new combined working memory- and executive function compensatory training ( Paying Attention in Class ) in a large sample of children with ADHD (Van der Donk, Hiemstra-Beernink, Tjeenk-Kalff, Van der Leij & Lindauer, 2015). Children in both treatment groups improved on measures of attention, working memory, inhibition and planning. These results were supported by parent and teacher rated improvements in executive functioning and ADHD behavior. CWMT was superior effective on visual spatial working memory. No time or treatment effects were found on academic outcome measures. The strong 53

56 Predictors and moderators of treatment outcome properties of our previous RCT, a large sample size and two active cognitive interventions, offers an opportunity to explore a number of predictors and moderators and thereby improving the field of cognitive interventions in children with ADHD. The aim of present study was to explore whether certain clinical variables and initial cognitive abilities predicted or moderated far transfer measures of our previous randomized controlled trial (Van der Donk et al., 2015). We examined three clinical variables: use of medication, comorbidity and subtype of ADHD; and two initial cognitive abilities: verbal working memory and visual spatial working memory baseline performance. Although empirical work regarding moderators of cognitive training outcome in children with ADHD is nonexistent, we chose our potential predictors and moderators on the basis of theory and existing empirical findings to the greatest extent possible. In addition, analyses of current study should be viewed as hypothesis-generating and not hypothesis-testing as identifying moderators will help to clarify the best choice of inclusion - or exclusion criteria or the best choice of stratification to maximize power for future randomized controlled trials (Kraemer et al., 2002). Therefore we did not specify any hypothesis regarding the direction or strength of the predictor or moderating effects. Previous studies in children with ADHD have indicated several moderators of treatment outcome for medication management, intensive behavior therapy and a combination of those two (The MTA cooperative group, 1999b; Owens et al., 2003; Hinshaw, 2007). For instance Owens and colleagues (2003) found that severity of initial ADHD symptoms, parental depressive symptomatology and child IQ moderated treatment outcome of medication management and combination treatments. For cognitive training in general and not specific for children with ADHD factors such as age, genetic predisposition, motivation, personality, prior treatment and initial cognitive ability have been mentioned as potential moderators for training gains and transfer measures (Von Bastian & Oberauer, 2013; Titz & Karbach, 2014; Shah, Buschkuehl, Jaeggi & Jonidas, 2012; Jaeggi, Buschkuehl, Shah & Jonidas, 2014; Karbach & Unger, 2014). However to our knowledge, there is currently no study that has 54

57 Chapter 3 investigated these potential moderators in a sample of children with ADHD who have followed cognitive training, only suggestions have been provided. Regarding the first clinical variable, use of medication, it has been suggested that use of medication during training might enhance the benefits of CWMT (Shinaver, Entwistle & Söderqvist, 2014). This idea was based on a study of Holmes and colleagues (2010) that showed that CWMT led to improvements in working memory performance that were above and beyond the effects of stimulant treatment alone in a sample of children with ADHD. In addition, others (Rutledge, van den Bos, McClure & Schweitzer, 2012) also suggested that the possible enhancing effects of medication in cognitive training should be explored. They proposed two theoretically driven mechanisms for these plausible enhancing effects of medication. First, as both stimulant and non stimulant medications affect dopamine and norepinephrine, they hypothesized that performance on a cognitive task is likely to be improved by improving working memory. On the other hand, they suggested that medication enhances the sensitivity to rewards which in turn could increase the intrinsic or extrinsic rewards for participating in training. In terms of the potential influence of comorbidity on treatment outcome in cognitive training, Chacko and colleagues (2013) suggested that children with learning disabilities might benefit more from CWMT. They refer to a study of Dahlin (2011) which showed that CWMT led to improvements in reading skills in children with learning disabilities. Chacko and colleagues (2013) hypothesized that academic achievements may be beneficially impacted by CWMT as working memory (i.e., the function of actively holding in mind and manipulating information relevant to a goal) plays a crucial role in academic achievements (e.g. Gathercole, Pickering, Knight & Stegmann, 2004) and is the trained target in CWMT. While moderator analysis of subtype of ADHD is generally absent in previous treatment outcome studies (The MTA cooperative group, 1999b; Owens et al., 2003), there is evidence that subtypes respond differently to medication (for overview see Diamond, 2005). Given the differences in cognitive -, behavioral 55

58 Predictors and moderators of treatment outcome - and underlying neurobiological profiles between subtypes (Diamond, 2005), we propose that different subtypes might also respond differently to cognitive training. In a recent review Sonuga-Barke, Brandeis, Holtmann and Cortese (2014) also stated that future trials should compare the response of clinical subtypes to different forms of cognitive training. Regarding initial cognitive ability, two accounts have been proposed to explain the individual differences in training related performance gains in cognitive training. First, the magnification account (also known as the Matthew effect) assumes that individuals that are already performing very well will also benefit most from cognitive interventions as they have more efficient cognitive resources to acquire and implement new strategies and abilities. Second, the compensation account assumes that high performing individuals will benefit less from cognitive interventions, because they already function at the optimal level which leaves less room for improvement. In contrast, low-performing individuals will benefit more from cognitive training as there is more room for improvement for them. Evidence points in the direction for a magnification effect for strategy based interventions and a compensation effect for process based interventions (for overview see Titz & Karbach, 2014; Karbach & Unger, 2014). In line with this compensation account, Chacko and colleagues (2013) suggested that children with ADHD plus working memory problems might benefit more from CWMT. This is based on the idea that not all children with ADHD suffer from working memory problems (Willcutt, Doyle, Nigg, Faraone & Pennington, 2005) and as CWMT is supposed to have effects on ADHD symptoms by improving working memory, working memory deficits might be an important requirement for the training to be effective. Furthermore, although not in a sample of children with ADHD, previous studies of CWMT (Holmes, Gathercole & Dunning, 2009; Dahlin, 2011; Dahlin, 2013; Bergman-Nutley & Klingberg, 2014) have shown that children with cognitive deficits (attention- or working memory deficits) improve on academic outcomes measures after training. Based on these findings and the fact that CWMT is generally viewed as a process-based intervention (e.g. Rapport et al., 2013), we expected that children with initial low working memory skills would benefit more from this intervention in terms of far transfer measures. 56

59 Chapter 3 To summarize, although it has been suggested that trials should compare the response of clinical subtypes and neuropsychological subgroups of children with ADHD to different forms of cognitive training (Sonuga-Barke et al., 2014), so far only suggestions of potential moderators are available and empirical evidence is lacking. Based on the suggestions from others (Chacko et al., 2013; Shinaver et al., 2014; Sonuga-Barke et al., 2014) we decided to focus on the clinical and neurocognitive heterogeneity of ADHD. Using data from our previous randomized controlled trial, the following research questions were addressed in current study: (1) Do clinical variables (use of medication, comorbidity and subtype of ADHD) and initial cognitive abilities (verbal working memory and visual spatial working memory baseline performance) predict neurocognitive and academic performance outcome measures? (2) Do clinical variables and initial cognitive abilities moderate neurocognitive and academic performance outcome measures? Method Participants Eligible participants were (a) children between the age of 8 and 12 years, (b) diagnosed with Attention-Deficit/Hyperactivity Disorder (all subtypes) by a professional according to the guidelines of the Diagnostic and Statistic Manual of Mental Disorders DSM-IV (APA, 2000). Children with comorbid Learning Disorders (LD) and/or Oppositional Defiant Disorder (ODD) according to the guidelines of the Diagnostic and Statistic Manual of Mental Disorders DSM-IV (APA, 2000) were also included. Children on medication were only included when they were well adjusted to their medication, which meant that they were not participating in a medication trial, and type and dosage of medication was unchanged at least 4 weeks prior to the start and during the training. Exclusion criteria were (a) presence of psychiatric diagnoses other than ADHD/LD/ODD, (b) Total Intelligence quotient < 80, (c) significant problems in the use of the Dutch language and (d) severe sensory disabilities (hearing/vision problems). A total of 115 children were assessed for eligibility; fourteen children did not meet inclusion criteria and were excluded. One 57

60 Predictors and moderators of treatment outcome hundred and one children were included and randomized to either CWMT (n=49) or the PAC intervention (n=52). After allocation two children from the PAC intervention group and one child from the CWMT group were transferred to a different research project due to time scheduling problems. Eventually, 48 children followed CWMT and 50 children followed the PAC intervention. Dropout rate was low with three children discontinuing CWMT and one child discontinuing the PAC intervention, treatment was too demanding for these children. For further details of the demographic characteristics, see Table 1. Table 1. Demographic characteristics CWMT (n=48) PAC (n=50) p (t, X² or Fisher s exact test) Age, M (SD) in years 9.8 (1.3) 10.0 (1.3) ns Gender Male, n (%) 33 (69) 37 (74) ns Full-Scale IQ, M (SD) (14.7) 99.2 (12.9) ns Enrollment, n (%) Clinical care School Type of education, n (%) Regular primary Special primary Special education SES, n (%) Low < Average High > Ethnicity, n (%) Mother Dutch Father Dutch 7 (15) 41 (85) 40 (87) 2 (4) 4 (9) 9 (22.5) 6 (15.0) 25 (62.5) 40 (88.9) 34 (77.3) 14 (28) 36 (72) 42 (86) 7 (14) 0 6 (13.6) 12 (27.3) 26 (59.1) 36 (73.5) 31 (63.3) ns X² (2) = 6.739, p Note. CWMT = Cogmed Working Memory Training; PAC = Paying Attention in Class; SES = social economic status ns ns ns Procedure The study was part of a prospective randomized controlled trial (Van der Donk et al., 2015). The ethics approval for this study was obtained from the Medical Ethical Committee (2011_269) at the Academic Medical Centre in Amsterdam, the Netherlands. The trial was registered at the Dutch National Trial Register, trial number NTR3415. Children were recruited in two different 58

61 Chapter 3 ways for this study. First, clinical care providers from two clinical care departments of the De Bascule (Academic Centre for Child and Adolescent Psychiatry, Amsterdam) referred eligible children to the researcher. Second, healthcare staff members (usually remedial teacher or school psychologist) of schools in the region of Amsterdam contacted the researcher when they had eligible children. In both cases, the researcher visited the school for an information meeting to extensively inform the staff members. Parents of children who met criteria for participation were approached and informed by the school staff member. After informed consent was obtained from parent(s), children were allocated to either 25 sessions of CWMT or 25 sessions of a new cognitive training called Paying Attention in Class. Treatment took place at school and the sessions were completed during morning school hours, aligned with teachers, for both intervention groups. Training periods were planned in between school holidays so that training sessions would not be interrupted for a longer period of time. Assessment took place prior to treatment, directly after treatment and 6 months after treatment. The assessment consisted of neurocognitive and academic performance measures for the child and questionnaires that were filled out (via ) by parents and teachers. A member of the research team (who was blind for the allocation) administered the neurocognitive and academic measures from each child at a reasonable silent room at school. Interventions Both interventions consisted of 25 sessions that were offered on a daily basis during school hours. Developmental psychologists (N=31) were trained as training aides according to the CWMT protocol (Dutch version: Gerrits, van der Zwaag, Gerrits-Entken, & van Berkel, 2012) and also trained as therapists for the Paying Attention in Class intervention. The psychologists were trained by a member of the research team and received weekly supervision from a certified Cogmed Coach (N=5). Since the psychologists trained both children in the CMWT group as children in the PAC group, they were asked not to teach the specific PAC skills to the children in the CWMT group. 59

62 Predictors and moderators of treatment outcome Cogmed Working Memory Training Cogmed Working Memory Training is a computerized training program aimed to train working memory. It consists of a variety of game-format tasks that are adaptive, which means that difficulty level is being adjusted automatically to match the working memory span of the child on each task. The program includes 12 different visuospatial and/or verbal working memory tasks, eight of these tasks (90 trials in total) are being completed every day (Klingberg et al., 2005). Children followed the standard CWMT protocol which means following the computer training program for 5 weeks, 5 times a week, approximately 45 minutes a day. The program was provided via the internet on a laptop in a separate room. Children were trained individually at school, guided by a trained developmental psychologist (training aid) who was supervised by a certified Cogmed Coach. Teachers were invited to attend an information meeting in which the content of CWMT was explained by first author, it was communicated that teachers did not have an active role during treatment if children received CWMT. Paying Attention in Class Paying Attention in Class (PAC) is an experimental combined working memory- and compensatory training that has been developed by members of our research team. Children are trained individually outside the classroom for 5 weeks, 5 times a week, approximately 45 minutes a day; the same duration as in the CWMT protocol. This PAC intervention contains three key elements; first of all, this intervention offers psycho education about executive functions that are related to classroom behavior. By making children more aware of these executive functions needed for adequate classroom behaviors, they obtain more insight in their own learning behavior. The psycho education addresses five executive functions, based on information processing and are important in a learning situation namely: attentional control, planning skills, working memory, goal-directed behavior and metacognition. For each executive function, five sessions in the protocol are devoted to that topic. For instance in regard to attentional control, it is explained to children that sitting straight in your chair or taking a deep breath might help to focus on the task. The psycho education is offered through an audio-book, with a brain 60

63 Chapter 3 castle metaphor. It is explained that only by following the right journey (first pay attention, make a plan, remember the task etc) in your head, i.e brain castle, you will manage to finish a task in the classroom. During this journey, the audio-book introduces them to the so called brain guards (i.e. strategies such as repeat instruction or visualize) or brain bandits (i.e. pitfalls such as distraction or acting to fast). The brain castle and it s guards and bandits are also visualized with drawings, plastic cards and stickers. Every day the audiobook ends with a different cue (depending on which executive function is discussed), for example I repeat what is said. This cue will be repeated throughout the session by the coach if necessary and the cue has to be practiced within a neuropsychological - and school task related exercise. Second, this intervention contains three paper and pencil adaptive working memory tasks: a visual spatial span task, a listening recall span task, and an instruction paradigm task (30 trials in total) which are practiced on a daily basis to improve working memory capacity. The sequence of each trial is extended after two correct trials. In the listening recall task, the coach reads aloud a certain amount of sentences and the child has to evaluate and tell whether the particular sentence is true or false. After this, the child has to reproduce the last word of each sentence in the correct order. The visual spatial span task is a paradigm of the Corsi block-tapping task (Corsi, 1972) which consists of a template with ten small blocks. The child has to tap the same cubes as the coach but then in the reversed sequence. The instruction task was based on a previously described analog task (Gathercole, Durling, Evans, Jeffcock & Stone, 2008) and consists of a paper template and cards that contains pictures of school related items. The coach reads aloud an instruction that the child has to execute for example Point to the big circle and pickup the small blue pen. For each next level one action or one extra item was added so the next sentence could be Pickup the large yellow book and a scissor and put them on the small square. Each working memory task was ended after ten executed trials. At the end of each session, the child fills out a high score list for each task to keep track of their performance. 61

64 Predictors and moderators of treatment outcome The third key element of this intervention is the central role of optimizing generalization to the classroom-situation. First of all, the strategies and pitfalls introduced through the audio-book described above will be illustrated and practiced by performing school related tasks, such as arithmetic, in a workbook during the session. The coach stimulates the child to use the cue from the audio-book and the coach also monitors whether the child uses any of the brain guards or whether the child encounters brain bandits. Performance on these school related tasks is not important, in stead reflection on the process is stimulated by the coach. The second way to improve generalization to the classroom is realized by a registration card which the child brings back to class. This card contains the cue of the day (for example, I repeat what is said ) and is meant to remember the child to practice the cue in the classroom. It will also inform the teacher about the cue so that he/she can monitor or stimulate the child to practice. Finally, we closely involved the teacher in the process by informing him/her with the protocol and by giving him/her an active part in the process. Teachers received a written manual, which contained information about how to recognize working memory problems in the classroom and information about the intervention itself. Furthermore, they were asked to daily record whether the child applied the cue in class through structured observation forms. The structured observation forms contained four specific statements, for instance The child is able to repeat the instruction, that had to be rated on a 4 point Likert scale. Subsequently, the coach reviewed this observation form the next day which gave the coach information whether the child visibly applied the cue in the classroom. Measures Predictors / Moderators Medication use Based on the application form filled out by the parents, children were divided into two groups: those on medication during training (predominantly stimulants) and those without medication during training. 62

65 Chapter 3 Comorbidity Children were divided into two groups: comorbidity either present or not present, based on parental report on the application form. In the present group, no distinction was made between type or amount of comorbidities as otherwise sample sizes of the different groups would have been very small. The present group consisted of children with the following comorbid diagnoses: Dyslexia (n = 25), Dyscalculia (n = 2), Learning Disorder NOS (n = 2), Oppositional Defiant Disorder (n = 2), Disorder of Written Expression (n = 1) and Developmental Coordination Disorder (n = 2). ADHD subtype Children were divided into two groups: combined type or inattentive type, none of the children were diagnosed with the hyperactive/impulsive subtype. Parents were asked to send a copy of the diagnostic psychiatric report of their child, this expert view was leading for establishing the subtype of ADHD. Based on these reports, subtype of ADHD could be established for seventy-six children. For the remaining twenty-two children the subtype was not described in the report and information was obtained from the Attention/ Hyperactivity module of the Diagnostic Interview Schedule for Children IV (Steenhuis, Serra, Minderaa & Hartman, 2009) that was administered by the research assistant by telephone. There was a small group of children (n = 4) that were diagnosed with the subtype Not Otherwise Specified. Furthermore, there was also a small group of children (n = 7) of which the psychiatric report did not mention a specific subtype and of which the DISC-IV did not confirm any subtype. Analyses revealed that as a group, the Not Otherwise Specified subtype children and undefined subtype children (total of n = 11), performed significantly lower on attention problems at baseline (F(2)=4,607, p =.012) than children with the combined type and inattentive type. Therefore the Not Otherwise Specified subtype - and undefined subtype children (n = 11) were viewed as having sub-threshold problems and were excluded for further subtype moderator analyses. 63

66 Predictors and moderators of treatment outcome Initial verbal working memory To reduce error variance, initial verbal working memory skills were assessed by a composite score that was created of the baseline standard scores of the Digit Span (Subtest WISC-III-nl; Wechsler, 2005), Comprehension of Instruction and Word List Interference - Remember task (Subtests Nepsy-II nl; Zijlstra, Kingma, Swaab & Brouwer, 2010). Analysis showed that all three variables correlated significantly with each other (Digit Span and Comprehension of Instruction, r =.42, p <.001; Digit Span and Word List Interference Remember, r =.37, p <.001; Comprehension of Instruction and Word List Interference Remember, r =.24, p =.017). The mean score of this composite score Initial verbal working memory was recoded in a nominal group variable based on the normal distribution of standard scores. A standard score between 0 and 7 was considered below average (two standard deviations below average), between 8 and 12 was considered average (one standard deviation below and one standard deviation above average) and 13 or larger was considered above average (two standard deviations above average). Initial visual spatial working memory The T-scores of the Span Board task (Subtest Wechsler Non Verbal-nl; Wechsler & Naglieri, 2008) were recoded in a nominal group variable based on the normal distribution of T-scores to create the variable Initial visual spatial working memory. A T-score of 39 or below was considered below average (two standard deviations below average), a T-score between 40 and 60 was considered average (one standard deviation below and one standard deviation above average) and a T-score of 61 or higher was considered above average (two standard deviations above average). As is shown in Table 2, there were no statistically significant differences between the two treatment groups on the predictor/moderator variables pre treatment. Measurement of Treatment Outcome Neurocognitive outcome measures Neurocognitive assessment included tasks that measure attention (Creature Counting and Score!, Test of Everyday Attention for Children; Manley, Robertson, Anderson & Nimmo-Smith, 2004), verbal working memory 64

67 Chapter 3 Table 2. Baseline comparison of predictor/moderator variables Predictor/moderator variables Medication during training Yes No Missing Comorbid disorder, n Yes No ADHD diagnose, n Combined Inattentive Initial verbal WM, n Low Average Above average Initial visual spatial WM, n Low Average Above average CWMT (n=48) PAC (n=50) p ( t/ X²) Note. CWMT = Cogmed Working Memory Training; PAC = Paying Attention in Class; WM = working memory ns ns ns ns ns (Digit Span; Wechsler, 2005), visual spatial working memory (Span Board; Wechsler & Naglieri, 2008), planning skills (Six Part Test BADS-C, Tjeenk-Kalff & Krabbendam, 2006) and inhibition (Mistakes and time from subtest Inhibition; Nepsy-II-nl, Zijlstra et al., 2010). Parents and teachers filled out the Dutch version of The Behaviour Rating of Executive Functions (BRIEF) questionnaire (Smidts & Huizinga, 2009). This questionnaire consists of 75 items which can chart the following executive functions: inhibition, shifting, emotional control, initiation, working memory, planning and organization, organization of materials and monitoring. These clinical scales form two broader indexes: the Behavioral Regulation Index (i.e. the scales Inhibit, Shift and Emotional Control) and the Metacognition Index (i.e. the scales Initiate, Working Memory, Plan/ Organize, Organization of Materials and Monitor). A T-score of 65 and above is considered as a clinical score. Academic outcome measures Academic performance was measured with tests for word reading fluency, automated math and spelling. Word reading fluency was measured with the 65

68 Predictors and moderators of treatment outcome Een Minuut Test (Brus & Voeten, 1973), this test consists two parallel cards that each hold 116 words. The child receives the instruction to read out loud (fast and accurate) as many as possible words in one minute. The Tempo Test Automatiseren (De Vos, 2010) was used to measure the degree of automated math. The test consists of four subtests: addition, subtraction, multiplication and division calculations. For each subtest, the child has to make as many as possible sums in two minutes with a maximum of 50. Also a total score of the four subtests and a total score of the addition and subtraction subtests can be calculated. As almost half of the children (n =47) in our sample were not able to perform multiplication and division calculations because of their young age, we chose to use only the total score of the addition and subtraction subtests as a outcome measure for automated math. The PI dictee (Geelhoed & Reitsma, 1999) was used to measure spelling skills and consists of two parallel versions (A & B). Each version consists of 135 words that are divided in nine blocks of fifteen words each. For each word, a sentence is read aloud and the child is asked to write down the repeated word. From a time-saving point of view, not all blocks were administered. The starting point was the educational age of the child and if there were three or more mistakes in that block, the previous block was also administered. The test was ended if the child made eight or more mistakes in one block. All raw scores of the academic performance measures were converted into a Learning Efficiency Quotient (educational age equivalent divided by the educational age) which allows for comparison across grade and age. We also performed secondary analysis in terms of accuracy (% correct) for the word reading fluency and automated math task as these tasks had a time restriction. We calculated an accuracy score for each point in time by dividing the raw scores of correct answers through the raw scores of total amount of produced words or sums and multiplying this answer by

69 Chapter 3 Statistical analysis The Statistical Package for Social Sciences, version 19 (IBM SPSS 19), was used for the statistical analysis and data was analyzed based on the intention to treat principle. Linear mixed model analysis was used with the dependent variables: attention, verbal working memory, visual spatial working memory, planning, inhibition, the Behavioral Regulation Index and the Metacognition Index of the BRIEF parent and teacher questionnaire, word reading, automated math and spelling. Outliers were removed if they had a z-score of < or > 3.29 and were replaced with the second highest value. Each predictor/moderator (Medication use, Comorbidity, Subtype of ADHD, Initial verbal working memory and Initial visual spatial working memory) and all interaction with time and treatment were entered as independent variables. Gender and age at the beginning of training were entered as covariates. Missing data was considered missing at random and was not imputed because using linear mixed model analyses has the benefit of using every observation for each participant if a baseline score is present. The significance level was set at p =.05 (two-tailed). A predictor would be established if there is a significant Time x Predictor interaction, indicating that for different levels of the predictor, the interventions lead to similar effects over time. For determining a moderator, our objective was to establish a significant Time x Treatment x Moderator interaction indicating that, for different levels of the moderator, the interventions lead to significantly different effects over time. Results Overall treatment outcome Effects of time were found on measures of attention (Creature counting: p <.001), visual spatial working memory (p <.001), inhibition (Mistakes: p <.001, Time: p <.001) and parent rated executive function behavior (Behavioral Regulation Index: p =.002, Metacognition Index: p <.001) at post treatment. At follow up, effects of time were found for measures of verbal- (p =.009) and visual spatial working memory (p <.001), planning (p <.001), inhibition (Mistakes: p <.001, Time: p <.001) and teacher rated executive function 67

70 Predictors and moderators of treatment outcome behavior (Metacognition Index: p =.001). Only one treatment effect in favor of CWMT was found on a measure of visual spatial working memory F(2, ) = 9.939, p <.001. No time or treatment effects were found on academic outcome measures. Predictor/Moderator analyses of clinical variables Use of medication A linear mixed model analysis indicated no significant predictive effects of use of medication on any of the neurocognitive measures, parent and teacher ratings of executive functioning or academic outcome measures. In terms of moderating effects for use of medication, the results of the linear mixed model analysis showed one significant interaction effect on the visual spatial working memory task, F(2, ) = 3.853, p =.023. Directly after treatment, children on medication benefitted most from CWMT in terms of visual spatial working memory and this effect was maintained at follow up. Children without medication also benefitted more from CWMT directly after treatment, however this effect was not found at follow up. Secondary analysis showed that, for the forty-five children who used medication during training, type of medication was changed for ten children at follow up. In addition, for the forty children who did not use medication during training, four children did use medication at follow up. The results are displayed in figure 1 and 2. No moderating effects were found for parent and teacher ratings of executive functioning or academic performance measures. Comorbidity A linear mixed model analysis revealed no significant predictive effects of comorbidity on any of the neurocognitive measures or parent and teacher ratings of executive functioning. However results did indicate one predicting effect on the academic performance measure Word Reading accuracy, F(2, ) = 3.624, p =.029. Directly after treatment, children without comorbidity increased on word reading accuracy while children with comorbidity decreased on accuracy. This interaction effect was no longer present at follow up. No other predicting nor moderating effects of comorbidity were found on any of the other outcome measures. 68

71 Chapter 3 T-SCORE VISUAL SPATIAL SPAN PRE POST FU CWMT PAC Figure 1. Children without medication during training: treatment effects on visual spatial working memory. CWMT = Cogmed Working Memory Training; PAC = Paying Attention in Class T-SCORE VISUAL SPATIAL SPAN PRE POST FU CWMT PAC Figure 2. Children with medication during training: treatment effects on visual spatial working memory. CWMT = Cogmed Working Memory Training; PAC = Paying Attention in Class Subtype of ADHD A linear mixed model analysis revealed no predicting effects of subtype of ADHD on the neurocognitive measures. Results did show a significant predicting effect on the Behavioral Regulation Index of the BRIEF both rated by parents (F(2, ) = 6.310, p =.002) and teachers (F(2, ) = 3.951, 69

72 Predictors and moderators of treatment outcome p =.021) with the same direction. Children with the ADHD-C subtype showed a decrease of behavioral regulation problems, both directly after treatment and at follow up. In contrast, children with the ADHD-I subtype showed a steep decrease of problems directly after treatment but an increase of problems at follow up. It should be noted here that although children with the ADHD-C subtype responded better to treatment, over time they still showed more problems than children with the ADHD-I subtype. Another predicting effect of subtype of ADHD was found on Word Reading accuracy, F(2, ) = p =.037, children with ADHD-C subtype improved on word reading accuracy directly after treatment and this improvement was maintained at follow up. However, children with the ADHD-I showed a decrease of Word Reading accuracy directly after treatment but improved at follow up and even outperformed children with the ADHD-C subtype. Results also revealed a moderating effect of subtype of ADHD on the BRIEF teacher rated scales Behavioral Regulation Index (F(2, ) = 4.626, p =.011) and Metacognition Index (F(2, ) = 4.126, p =.018). The direction of the interaction effect is similar for both indexes, children with the ADHD-C subtype showed a decrease of problems over time (both directly after treatment and at follow up) with no difference between the intervention groups. However children with the ADHD-I subtype from the CWMT group showed a decrease of problems over time while children who followed the PAC intervention showed an increase of problems at follow up. In summary, on the short term children with the ADHD-I subtype benefitted more from cognitive training in general in terms of parent and teacher rated behavioral regulation problems. In addition, children with the ADHD-I subtype who followed CWMT benefitted most on the long term in terms of teacher rated behavioral regulation - and metacognition problems (results for the Behavioral Regulation Index are shown in figure 3 and 4 and results for Metacognition Index are shown in figure 5 and 6). It should be noted here that data was no equally distributed, particularly with a large standard deviation (SD = 22) for children in the PAC group (n = 10). We found no other moderating effects of subtype of ADHD on other outcome measures. 70

73 Chapter 3 70 CWMT PAC T-SCORE BRI - T PRE POST FU Figure 3. Children with ADHD-C subtype: treatment effects on teacher rated behavioral regulation problems. CWMT = Cogmed Working Memory Training; PAC = Paying Attention in Class; BRI-T = Behavioral Regulation Index rated by teachers T-SCORE BRI - T CWMT PAC 40 PRE POST FU Figure 4. Children with ADHD-I subtype: treatment effects on teacher rated behavioral regulation problems. CWMT = Cogmed Working Memory Training; PAC = Paying Attention in Class; BRI-T = Behavioral Regulation Index rated by teachers 75 CWMT PAC T-SCORE MCI- T PRE POST FU Figure 5. Children with ADHD-C subtype: treatment effects on teacher rated metacognition problems. CWMT = Cogmed Working Memory Training; PAC = Paying Attention in Class; MCI-T = Metacognition Index rated by teachers 71

74 Predictors and moderators of treatment outcome 75 CWMT PAC T-SCORE MCI- T PRE POST FU Figure 6. Children with ADHD-I subtype: treatment effects on teacher rated metacognition problems. CWMT = Cogmed Working Memory Training; PAC = Paying Attention in Class; MCI-T = Metacognition Index rated by teachers Predictor/Moderator analyses of initial cognitive abilities Initial verbal working memory A linear mixed model analysis revealed one predicting effect of initial verbal working memory on attention (Creature counting - Time), F(4, ) = 3.000, p =.020. Children with below average and average initial verbal working memory skills became faster over time on this attention task, while performance of children with above average initial verbal working memory skills decreased over time. No other predicting effects of initial verbal working memory on parent and teacher ratings of executive functioning or academic performance measures were found. Results revealed one moderating effect of initial verbal working memory on visual spatial working memory, F(4, ) = 2.462, p =.047. Children with above average initial verbal working memory skills improved over time, with no difference between the interventions. Performance of children with average initial verbal working memory skills also improved over time for both intervention groups, however children who followed CWMT showed a larger improvement than children who followed PAC. The most pronounced interaction effect however, took place for children with below average initial verbal working memory skills, performance of children who followed the PAC intervention decreased slightly over time while children who followed CWMT showed a significant improvement over time. Results are displayed in figure 7, 8 and 9. It should be noted that the below average group consisted of a very small amount 72

75 Chapter 3 of children (n = 5) with only one child who followed CWMT. We found no moderating effects of initial verbal working memory on parent and teacher ratings of executive functioning or academic performance measures. Initial visual spatial working memory Results revealed one significant predictive effect of Initial visual spatial working memory on the visual spatial working memory task, F(4, ) = 8.747, p <.001. Children with below average and average initial visual spatial working memory skills showed improvements over time, while children with above average initial visual spatial working memory skills showed a decrease of performance over time. Although the above average group showed a decrease over time, they still outperformed the average and below average group at all time points. No other predicting or moderating effects of initial visual spatial working memory were found. T-SCORE VISUAL SPATIAL SPAN PRE POST FU CWMT PAC Figure 7. Children with initial above average verbal working memory skills: treatment effects on visual spatial working memory. CWMT = Cogmed Working Memory Training; PAC = Paying Attention in Class 73

76 Predictors and moderators of treatment outcome T-SCORE VISUAL SPATIAL SPAN PRE POST FU CWMT PAC Figure 8. Children with initial average verbal working memory skills: treatment effects on visual spatial working memory. CWMT = Cogmed Working Memory Training; PAC = Paying Attention in Class T-SCORE VISUAL SPATIAL SPAN PRE POST FU CWMT PAC Figure 9. Children with initial below average verbal working memory skills: treatment effects on visual spatial working memory. CWMT = Cogmed Working Memory Training; PAC = Paying Attention in Class Discussion In present study we explored whether a number of clinical variables and initial cognitive abilities predicted or moderated neurocognitive and academic performance outcome measures after cognitive training in children with ADHD. Current study showed that subtype of ADHD both predicted and moderated parent and teacher ratings of executive functioning behavior. Furthermore, word reading accuracy was predicted by subtype of ADHD and comorbidity. Use of medication, initial verbal - and visual spatial working memory skills only predicted and moderated near transfer measures. 74

77 Chapter 3 First of all we looked at the influence of the clinical variables: use of medication, comorbidity and subtype of ADHD. Use of medication did not predict any outcome measure, cognitive training in general whether it is process- or more skills oriented - is equally effective for medicated and medication naïve children. However, results did indicate one moderating effect: the superior effect of CWMT on visual spatial working memory was maintained at follow up for children who used medication during training but not for medication naïve children. Previously, Holmes and colleagues (2010) compared the effects of stimulant medication and CWMT in terms of working memory performance and found that stimulant medication only had an effect on visual spatial working memory performance while CWMT led to improvements in all aspects of working memory. This could imply that the children who used medication during training in current study already performed better on visual spatial working memory at baseline. Therefore these children plausibly had more efficient cognitive resources available to process the highly visual spatial training stimuli of CWMT. So at least to the extend of visual spatial working memory, current results are in line with Shinaver and colleagues (2014) suggestion that medication could enhance the benefits of CWMT. However one question that remains is why this enhancing effect of medication is limited to visual spatial working memory. One plausible explanation would be that most of the trained tasks within CWMT tap into the domain of visual spatial working memory, therefore improvements in visual spatial working memory are generally viewed as a practice effect. In order to truly disentangle the effects of medication and effects of CWMT on visual spatial working memory, future studies should consider including a third group of children who receive medication but no training. Comorbidity only had a predictive effect on word reading accuracy on the short term. Irrespective of type of training, performance on word reading accuracy improved at all time points for children without comorbid disorders while children with comorbid disorder showed a decrease of accuracy post treatment and an increase at follow up. It should be considered here that the comorbidity group in current study almost entirely consisted of children diagnosed with a Learning Disorder (n=30 out of n=34). This would imply that, at least on the short term, children with ADHD and comorbid learning 75

78 Predictors and moderators of treatment outcome disabilities do not benefit from cognitive training in terms of academic outcomes measures, as would be in concordance with a study of Gray and colleagues (2012). However, this doesn t mean that cognitive training should be discouraged for children with ADHD and comorbid learning difficulties. As on the one hand, current results also showed that children with comorbid learning disabilities improve in word accuracy on the long term, which highlights the necessity to include long term assessments of academic performance measures. Additionally, both current study and the study of Gray and colleagues (2012) did not differentiate between type of learning disability. Gray and colleagues (2012) suggested that this would be an interesting predictor variable for future research. Conclusively, future studies with larger sample sizes should include long term assessments of academic outcome measures and a population with a broader range of types of comorbid learning disabilities. Of all predicting/moderating variables, subtype of ADHD played the most profound role in determining treatment outcome. More interestingly, it affected only far transfer outcome measures, i.e. both parent and teacher ratings of executive functioning behavior and word reading accuracy. Results for word reading accuracy showed that, although in absence of an overall time or treatment effect, children with the ADHD-C subtype benefitted most on the short term however on the long term the opposite occurred; children with the ADHD-I benefitted most. One plausible explanation for this postponed effect is the fact that a large proportion of children with the ADHD-I subtype are affected by very slow reaction time and slow processing speed, characteristics that correlate with poor working memory skills (Diamond, 2005). Therefore it might take more time for the beneficial effects of cognitive training to unfold for this group of children. However, this postponed effect for the ADHD-I group was not observed for the spelling and automated math task. While reading decoding primarily depends on phonological shortterm memory and verbal working memory, automated math and spelling tasks require other and more complex working memory systems such as the central executive (for overview see Dehn, 2008). Working memory deficits in children with the ADHD-I subtype are most prominent in auditory processing 76

79 Chapter 3 (Diamond, 2005) which would imply that cognitive training only promotes this specific deficient system but no other working memory systems. In terms of the parent and teacher rated behavioral regulations problems it was shown that children with the ADHD-I subtype could temporarily benefit more from cognitive training in general. The fact that both parents and teachers report these results makes the evidence compelling. These behavioral regulation problems can be viewed as the hot aspects of executive functioning. According to Zelazo and Müller (2011) hot executive functioning is required for problems that are characterized by high affective involvement or demand flexible appraisals of the affective significance of stimuli (p. 586). Castellanos, Sonuga-Barke, Milham and Tannock (2006) proposed that hyperactive/impulsivity symptoms reflect those hot executive function deficits. In contrast, cool executive functions such as working memory are more likely to be elicited by relatively abstract decontextualized problems (Zelazo and Müller, 2011, p. 586) and can be associated with attention problems according to Castellanos and colleagues (2006). Based on this perspective we suspect that children with the ADHD-C benefitted less from cognitive training due to a more heterogeneous origin with both cool and hot executive function deficiencies. Additionally, children with the ADHD-I subtype in the CWMT group also benefitted most on the long term regarding teacher rated behavioral regulation - and metacognition problems. This was a rather surprising though promising finding as studies that investigated the efficacy of CWMT in children with ADHD so far haven t been able to establish effects on teacher rated executive function behavior. Future studies with larger sample sizes of different subtypes and well blinded assessments of executive function behavior are necessary to further investigate this potential beneficial effect for the ADHD-I subtype. Finally, initial verbal and visual spatial working memory skills only predicted and moderated near transfer measures. Irrespective of type of training, children with initial below average or average working memory (either verbal or visual spatial) skills benefitted over time in terms of performance on an attention - and visual spatial working memory task while performances 77

80 Predictors and moderators of treatment outcome decreased over time for children with initial above average working memory skills. We also found an additional moderating effect on the visual spatial working memory task; children with initial below average or average verbal working memory skills benefitted most from CWMT while there was no difference between interventions for the above average group. These last mentioned findings confirm the hypothesis of Chacko and colleagues (2013) that children with ADHD plus working memory problems could benefit more from CWMT. Consistent with the compensation account, these results suggest that high performing individuals benefit less from training, possibly due to the fact that they already function at the optimal level at the beginning of training which leaves less room for improvement. Previous studies that investigated the effects of process based interventions similar to CWMT also detected this compensation effect (for overview see Titz & Karbach, 2014). Unfortunately, no beneficial effects of initial low working memory skills were found in terms of improvements in academic outcome measures. We suggest that there are several reasons why current predicting and moderating effects of initial working memory skills were limited to an attention - and visual spatial working memory task. First of all, tasks that were used to assess initial working memory skills and tasks that were used to assess attention - and visual spatial working memory all tap into the domain general component of working memory (i.e. central executive). So these predictor/ moderator variables and outcome measures to some extend measured the same construct. In addition, features of the task that measured visual spatial working memory (Span Board) overlap with features of trained tasks in both interventions. This overlap is greatest for CWMT as this intervention contains multiple tasks that tap into the domain of visual spatial (working) memory. Therefore the observed improvements can be viewed as practice - and near transfer effects. Second, we suspect that the variety of cognitive - and behavioral impairments associated with ADHD might suppress the ability to benefit from cognitive training in terms of academic outcome measures. It is plausible that only children with a single cognitive impairment, and no co-occuring psychiatric disorder, benefit from cognitive training in terms of academic outcome measures as was found in previous studies (Holmes et al., 2009; Bergman-Nutley & Klingberg, 2014). 78

81 Chapter 3 Limitations Some limitations of this study need to be considered. First, we did not correct for multiple testing which, given the large amount of outcome measures, might have led to unjustified demonstrated effects (Type I error). On the other hand, a Bonferroni correction would increase the probability of a type II error leading to more conservative results which would be undesirable given the explorative character of this study. Second, in regard to interpreting the results of the parent and teacher rated questionnaires one should hold in mind that parents and teachers were aware that children received active treatment so it is plausible that findings are inflated due to expectancy effects. In addition, teachers from children in the PAC intervention received information on how to recognize executive function problems in the classroom so it is plausible that these teachers improved on detecting these problems and therefore rated them more critically for children who followed the PAC intervention. Third, despite the fact that we used a composite score for the variable initial verbal working memory skills, it should be mentioned that initial verbal - and visual working memory skills are susceptible to random or systematic error as they only reflect one point in time. Future trials should use composite scores of initial working memory skills that include multiple domains of working memory (e.g. differentiate between short term memory and central executive) and multiple time points. Fourth, we did not consider the possible conjoint effect of moderators. There is a high likelihood of collinearity between variables, for instance the potential overlap between ADHD-I subtype and comorbid learning difficulties. Fifth, although we carefully selected the predictors and moderators, there are numerous other factors that deserve attention in future trials. Other factors to consider would be personality, motivational style, treatment history or more demographic factors such as age. Finally, while we focused on the relationship between baseline variables and outcome measures, we still do not know by which mechanism those moderators exert their influence on treatment outcome. We suggest that potential mediators of cognitive training, such as personal growth curve, should be explored in future trials. 79

82 Predictors and moderators of treatment outcome Conclusion Current study has shown that treatment outcome measures of cognitive interventions for children with ADHD can be influenced by clinical variables and initial cognitive abilities. From a scientific point of view, this might explain the inconsistent results found in previous studies as inclusion and exclusion criteria varied greatly which inevitably leads to more individual differences that could influence outcome measures. Future trials should therefore for example consider screening participants for ADHD related deficits (such as working memory) before including them in trials. Current study also ameliorates the clinical perspective of cognitive training when the results are viewed as a starting point for providing guidelines to clinicians. Most importantly the results imply that cognitive training is not a one size fits all treatment. For example, when the main aim is to merely improve children s attention- and visual spatial working memory skills, clinicians should take into account that use of medication during training and low initial verbal working memory skills might lead to greater gains for process based intervention such as CWMT. Additionally, when the main aim is to improve executive function behavior at home or in school, one should hold in mind that children with the ADHD-I subtype could profit most from training. Finally, clinicians should take into account that improvements in word reading accuracy seem to be postponed for children with comorbid (predominantly learning) disorders and children with the ADHD-I subtype. 80

83 Chapter 4 The influence of individual differences on treatment outcomes of cognitive training in a sample of children with Attention- Deficit/Hyperactivity Disorder Marthe van der Donk, Anne-Claire Hiemstra-Beernink, Ariane Tjeenk-Kalff, Aryan van der Leij & Ramón Lindauer Manuscript under review

84 The influence of individual differences on treatment outcomes Abstract The aim of present study was to extend our understanding of individual differences in treatment outcome of an innovative cognitive training called Paying Attention in Class (PAC) within a sample of children with Attention-Deficit/Hyperactivity Disorder. First of all, it was determined which demographical, clinical or neurocognitive characteristics predict individual treatment response, which was based on clinically significant improvement in working memory. Additionally, it was investigated whether treatment response in terms of working memory improvement was a prerequisite to obtain improvements in far transfer measures of neurocognitive functioning, academic performance, behavior in class, behavior problems and quality of life. A total of 164 children between the age of 8 and12 years followed this new PAC training. Non-responders were offered additional Cogmed Working Memory Training (CWMT). Results showed that initial sustained attention, initial verbal working memory skills and teacher rated metacognition problems predicted treatment response. Transfer for non-responders was limited to non-trained working memory tasks and parent rated questionnaires, and providing CWMT did not lead to additional improvements in far transfer measures. Partial responders and responders improved on most far transfer measures with group effects in favor of the responders on several outcome measures. Current findings imply that individual differences should be taken into account when determining the efficacy of cognitive training for children with ADHD. These results indicate that children with a certain vulnerable cognitive and behavioral profile are in dire need of adapted treatment approaches. 82

85 Chapter 4 Introduction Recent meta-analyses and review studies regarding the effects of cognitive training for children with Attention-Deficit/Hyperactivity Disorder (e.g. Rapport, Orban, Kofler & Friedman, 2013; Sonuga-Barke et al., 2013; Cortese et al., 2015; Orban, Rapport, Friedman & Kofler, 2015) have shown that the evidence of beneficial effects on behavioral, academic and non trained cognitive skills is insufficiently supported. Although a lot of studies have paid attention to improving treatment efficacy by addressing caveats in treatment and study designs (Orban et al., 2015; Cortese et al., 2015; Sonuga-Barke et al., 2013), little is known about the influence of individual differences on cognitive training outcomes in ADHD. Given the clinical and pathophysiological heterogeneity of ADHD (Wilcutt et al., 2012), it is plausible that certain individuals benefit more from cognitive training than others. Medication studies in children with ADHD have shown that factors such as age, severity of the disorder, intelligence and comorbid anxiety determine treatment response (Buitelaar, van der Gaag, Swaab-Barneveld & Kuiper, 1995; Van der Oord, Prins, Oosterlaan & Emmelkamp, 2008b). However, research on the influence of individual differences on cognitive training outcome in children with ADHD is lacking. It has been suggested though that several individual differences could affect training gain and transfer effects such as etiology, prior treatment, motivation, use of medication during training, ADHD subtype, developmental age and initial working memory skills (Shah, Buschkuehl, Jaeggi & Jonides, 2012; Shinaver, Entwistle & Söderqvist, 2014; Chacko et al., 2013). In a previous randomized controlled trial (Van der Donk, Hiemstra-Beernink, Tjeenk-Kalff, Van der Leij & Lindauer, 2015), we established the effects of a new Paying Attention in Class (PAC) intervention in a sample of school aged children with ADHD and compared it to the effects of Cogmed Working Memory Training (CWMT). The new PAC intervention, which was developed by our research group, is fundamentally different from most other processbased cognitive interventions (such as CWMT) as the PAC training contains 83

86 The influence of individual differences on treatment outcomes an additional innovative classroom embedded compensatory approach. The main goal of this additional compensatory module is optimizing generalization to the learning environment by, on the one hand, providing the child with psycho-education about executive functions that are important in a learning situation and, on the other hand, increasing teacher awareness of EF problems and encourage them to adapt their approach to teaching. Results of this randomized controlled trial showed that children in both treatment groups (i.e. CWMT and PAC) improved on measures of attention, working memory, inhibition and planning. These results were supported by parent and teacher rated improvements in executive functioning and ADHD related behavior. No improvements were found on any of the academic- or behavior in class outcome measures. However, we also observed a large variability in individual transfer gains for children in the PAC intervention group; while certain individuals benefited from the intervention both in terms of near and far transfer measures, others only mainly improved on working memory tasks and parent rated questionnaires. These findings encouraged us to investigate whether individual differences at baseline could account for these observed differences in both near and far transfer training gains. In order to extend our understanding of these individual differences in treatment outcomes, an additional group of 116 children with ADHD received the new PAC intervention. The first aim of this study was to establish which demographical, clinical and baseline neurocognitive characteristics could predict individual treatment response. Response to treatment was determined based on a clinical significant improvement on working memory. By merely looking at the statistical significant differences between pre- and post treatment measures (as happens within the golden standard RCT designs) valuable information is lost on the variability of response to treatment within the sample. In addition, statistical significance tests do not provide information on whether treatment has been clinically significant, i.e. whether an individual shifted from the dysfunctional to the functional population (Jacobson & Truax, 1991). The second aim of this study was to investigate whether this clinically significant improvement in working memory was a prerequisite to obtain improvements in far transfer measures. Therefore, we 84

87 Chapter 4 established the effects of the PAC intervention on neurocognitive functioning, academic performance, behavior in class, behavior problems and quality of life for the different response groups (i.e. non-responders, partial responders and responders). Finally, both from an ethical and clinical perspective as the PAC intervention was experimental, the third aim of this study was to investigate if non-responders benefit from additional CWMT treatment. This intervention has been shown to improve working memory skills in children with ADHD (e.g. Klingberg, Forssberg & Westerberg 2002, Klingberg et al., 2005; Holmes et al., 2010). Method Participants The data were obtained from children who participated in two separate studies. The first study included 48 children who were allocated to the Paying Attention in Class (PAC) intervention from our previous randomized controlled trial (Van der Donk et al., 2015). For the second study, 125 children were assessed for eligibility; nine children did not meet inclusion criteria and were excluded. A total of 116 children were included and started with the PAC intervention. Eligible participants were (a) children between the age of 8 and 12 years, (b) diagnosed with ADHD (all subtypes) by a professional according to the guidelines of the Diagnostic and Statistic Manual of Mental Disorders DSM-IV (APA, 2000). Children with comorbid Learning Disabilities (LD) and/or Oppositional Defiant Disorder (ODD) according to the DSM-IV (APA, 2000) were also included. Children on medication were only included when they were well adjusted to their medication, which meant that they were not participating in a medication trial, and type and dosage of medication was unchanged at least 4 weeks prior to the start and during the training. Exclusion criteria were (a) presence of psychiatric diagnoses other than ADHD/LD/ODD, (b) Total IQ < 80, (c) significant problems in the use of the Dutch language and (d) severe sensory disabilities (hearing/vision problems). 85

88 The influence of individual differences on treatment outcomes Procedure The ethics approval for this study was obtained from the Medical Ethical Committee (2011_269) at the Academic Medical Centre in Amsterdam, the Netherlands. Children were recruited in two different ways for this study. First, clinical care providers from two clinical care departments of the De Bascule (Academic Centre for Child and Adolescent Psychiatry, Amsterdam) referred eligible children to the researcher. Second, healthcare staff members (usually remedial teacher or school psychologist) of schools in the region of Amsterdam contacted the researcher when they had eligible children. In both cases, the researcher visited the school for an information meeting to extensively inform the staff members. Parents of children who met criteria for participation were approached and informed by the school staff member. After informed consent was obtained, parent(s) and teachers were invited to participate in an information meeting at school where they were informed about the content of the intervention. Two to three weeks prior to treatment, parents and teachers received the questionnaires via or hard copy on request. Parents filled out an application package containing a written informed consent form, questionnaires of demographic- and background information and the Dutch translation of the Social Communication Questionnaire (SCQ; Warreyn, Raymaekers & Roeyers, 2004) to screen for autism spectrum disorder. The Lifetime version of the SCQ consists of 40 questions that have to be answered with yes or no. A total raw score of 15 or higher indicates a likelihood of the presence of autism spectrum disorder and is recommended as a cutoff-score, a total of eight children exceeded this cutoff-score. On the Behavior Rating Inventory of Executive Function (BRIEF; Smidts & Huizinga, 2009), these children showed significantly more parent rated problems at baseline regarding cognitive flexibility (U = , z = , p =.028) and monitoring (U = , z = , p =.031) and rule breaking scale of the Child Behavior Checklist (Verhulst, van der Ende & Koot, 1996), U = , z = , p =.020. Therefore, we decided to remove data from these children from subsequent analyses. See Figure 1 for a flow chart of current study. 86

89 Chapter 4 Parents were also asked to send a copy of the diagnostic psychiatric report of their child to establish the subtype of ADHD and rule out other potential psychiatric problems that met exclusion criteria. A short version of the WISC-IIInl (Wechsler, 2005) with the subtests Similarities, Block Design, Vocabulary and Picture Completion was administered to estimate the Total Intelligence quotient if there were no prior recordings available. One week prior to treatment, neurocognitive - and academic performance was assessed by a member of the research team in a silent room at school. Treatment took place at school and all sessions were completed during morning school hours. Training periods were planned in between school holidays, to ensure that training sessions would not be interrupted for a longer period of time. Post treatment assessment took place within one week after the last training session and follow-up assessment took place after six months. Children who were classified as non-responders at follow up assessment were offered CWMT at home under supervision by a certified CWMT coach. For these children, neurocognitive - and academic performance was again assessed after finishing CWMT. Interventions Paying Attention in Class PAC is an experimental combined working memory- and compensatory training that has been developed by our research team. Children were trained individually at school (though outside the classroom) for 5 weeks, 5 times a week, approximately 45 minutes a day. This PAC intervention contains three key elements. First of all, this intervention offers psycho-education about five executive functions that are important in a learning situation: attentional control, planning skills, working memory, goal-directed behavior and metacognition. The psycho-education is offered through an audiobook, using a brain castle metaphor, in which children are introduced to the so called brain guards (i.e. strategies such as repeat instruction or visualize) or brain bandits (i.e. pitfalls such as distraction or acting to fast). Every day the audio-book ends with a different cue (depending on which executive function was discussed), for example I repeat what is said. This cue will be repeated throughout the session by the coach if necessary and the cue has to be practiced within a neuropsychological - and school task related exercise. 87

90 The influence of individual differences on treatment outcomes Figure 1. Flow diagram Enrollment Assessed for eligibility study 2 (n=125) Excluded (n=9) Not meeting inclusion criteria Followed Paying Attention in Class study 1 (n=48) Exceeded cut off score SCQ (n=8) Treatment Paying Attention in Class (n=156) Study 1 (n=46) Study 2 (n=110) Lost at follow-up (n=1) Discontinued intervention (n=5) treatment was too demanding Follow-up Analysis Analysed (n=150) Second, this intervention contains three paper and pencil adaptive working memory tasks: a visual spatial span task, a listening recall span task, and an instruction paradigm task (30 trials in total), which are practiced on a daily basis in order to improve working memory capacity. In the listening recall task, the coach reads aloud a certain amount of sentences and the child has to evaluate whether the particular sentence is true or false. After this, the child has to reproduce the last word of each sentence in the correct order. 88

91 Chapter 4 The visual spatial span task is a paradigm of the Corsi block-tapping task (Corsi, 1972), which consists of a template with ten small blocks. The child has to tap the same cubes as the coach in the reversed sequence. The instruction task was based on a previously described analog task (Gathercole, Durling, Evans, Jeffcock & Stone, 2008) and consists of a paper template and cards that contains pictures of school related items. The coach reads aloud an instruction that the child has to execute, for example Point to the big circle and pickup the small blue pen. Each working memory task was ended after ten executed trials. At the end of each session, the child fills out a high score list for each task to keep track of his performance. The third key element of this intervention is the central role of facilitating generalization to the classroom-situation. First of all, the strategies and pitfalls introduced through the audio-book described above will be illustrated and practiced by performing school related tasks such as arithmetic, in a workbook during the session. The second way to improve generalization to the classroom is realized by a registration card which the child brings back to class. This card contains the cue of the day (for example, I repeat what is said ) and is meant to remember the child to practice the cue in the classroom. It will also inform the teacher about the cue so that he/she can monitor or stimulate the child to practice. Finally, we closely involved the teacher in the process by informing him/her with the protocol and by giving him/her an active part in the process. Teachers received a written manual, which contained information about how to recognize working memory problems in the classroom and information about the intervention itself. Furthermore, they were asked to daily record whether the child applied the cue in class through structured observation forms. A more extensive description of the intervention can be found in the publication of our randomized controlled trial (Van der Donk et al., 2015). Standardization intervention and treatment adherence Developmental psychologists with a master degree were trained as therapists for the PAC intervention. During an interactive three hour course, provided by a member of the research team, the therapists were introduced to the 89

92 The influence of individual differences on treatment outcomes theoretical background and practical implications of the intervention. The PAC protocol consists of a written manual for the therapist with clear instructions for each task/component and daily score sheets for the working memory tasks (Van der Donk, Tjeenk-Kalff & Hiemstra-Beernink, 2015). The therapists received supervision sessions on a weekly basis by a clinical staff member, in which progress and clinical difficulties were discussed. The PAC workbook was used to monitor the results of the training. A total of 67 psychologists and four supervisors collaborated in this study. Cogmed Working Memory Training CWMT is a computerized working memory training program. It consists of a variety of game-format tasks that are adaptive, which means that difficulty level is being adjusted automatically to match the working memory span of the child within each task. The program includes 12 different visual spatial and/or verbal working memory tasks, and eight of these tasks (90 trials in total) are being completed every day (Klingberg et al., 2005). Children followed the standard CWMT protocol: 5 weeks of training, 5 times a week, for approximately 45 minutes a day. Children were trained at home, guided by their parents who were supervised by a certified Cogmed Coach. Measures All information of the demographical (age, gender, social economic status, type of education) and clinical variables (Total IQ, use of medication, subtype of ADHD and presence of Learning Disorder) was obtained from the information package that parents filled out during enrollment of the study. Primary outcome measures Neurocognitive assessment included tasks that measure attention (Creature Counting and Score!: Manley, Robertson, Anderson & Nimmo-Smith, 2004), verbal working memory (Digit Span: Wechsler, 2005; Comprehension of Instruction and Word List Interference: Dutch translation Zijlstra, Kingma, Swaab & Brouwer, 2010), visual spatial working memory (Span Board: Wechsler & Naglieri, 2008), planning skills (Six Part Test BADS-C: Dutch translation Tjeenk- Kalff & Krabbendam, 2006) and inhibition (Inhibition: Zijlstra et al., 2010). Parents 90

93 Chapter 4 and teachers filled out the Dutch version of The Behavior Rating Inventory of Executive Functions (BRIEF) questionnaire (Smidts & Huizinga, 2009). This questionnaire consists of 75 items which can chart the following executive functions: inhibition, shifting, emotional control, initiation, working memory, planning and organization, organization of materials and monitoring. These clinical scales form two broader indices: the Behavioral Regulation Index (i.e. the scales Inhibit, Shift and Emotional Control) and the Metacognition Index (i.e. the scales Initiate, Working Memory, Plan/Organize, Organization of Materials and Monitor). An overall score, the Global Executive Composite, can also be calculated. T-scores of 65 and above are considered clinical. Academic performance was measured with tests for word reading fluency, automated math and spelling. Word reading fluency was measured with the Een Minuut Test (Brus & Voeten, 1973), which consists of two parallel cards (version A & B) which each hold 116 words. The child receives the instruction to read out loud (fast and accurate) as many words as possible in one minute. The Tempo Test Automatiseren (De Vos, 2010) was used to measure the degree of automated math. The test consists of four subtests: addition, subtraction, multiplication and division calculations. For each subtest, the child has to make as many sums as possible in two minutes, with a maximum of 50. Additionally, a composite score of all four subtests and a composite score of the addition and subtraction subtests can be also calculated; both composite scores were used as outcome measure in current study. The PI dictee (Geelhoed & Reitsma, 1999) was used to measure spelling skills and consists of two parallel versions (A & B). Each version consists of 135 words that are divided in nine blocks of fifteen words each. For each word, a sentence is read aloud and the child is asked to write down the repeated word. From a time-saving point of view, not all blocks were administered. The starting point was determined by the educational age of the child and only if there were three or more mistakes in that block, the previous block was also administered. The test was ended if the child made eight or more mistakes in one block. All raw scores were converted into a Learning Efficiency Quotient (educational age equivalent divided by the educational age) which allows for comparison across grade and age. 91

94 The influence of individual differences on treatment outcomes Secondary outcome measures Behavior problems were assessed by both teacher and parents using The Child Behavior Checklist for Ages 6-18 (Verhulst et al., 1996; translation from Achenbach & Rescorla 2001) and Teacher s Report Form for Ages 6-18 (Verhulst, van der Ende & Koot, 1997). We reported the scale Attention Problems, for which a T-score of 65 and above is considered as problematic, and the scale Externalizing Problems which consists of the rule breaking behavior scale and the aggressive behavior scale; a T-score of 60 is considered as problematic. Behavior in class was reported by the teacher using the Learning Condition Test (Scholte & Van der Ploeg, 2009): this is a 70 item questionnaire that measures Direct Learning Conditions (concentration, motivation, work rate, task orientation, working according to a plan, persistency), Social Embedding (social orientation and social position in class) and Relations (relationship with peers and teacher). Items are rated on a 5 point Likert scale, with higher scores indicating negative prognoses. Quality of Life was measured with the Dutch translation of the Kidscreen-27 questionnaire (Ravens-Sieberer et al., 2007) and was completed by parents and the child. It covers five dimensions of quality of life: physical well-being, psychological well-being, autonomy & parents relations, social support & peers and school environment. The raw scores are converted into T-scores: a higher score reflects a higher quality of life. Treatment Response Response to treatment was determined at the follow up assessment six months after treatment and was based on four working memory outcome measures; the verbal working memory task Digit Span (Wechsler, 2005), the visual spatial working memory task Span Board (Wechsler & Naglieri, 2008) and the working memory scale from the Dutch version of BRIEF questionnaire (Smidts & Huizinga, 2009) rated by both teacher and parent(s). For each of these measures a Reliable Change Index (Jacobson & Truax, 1991) was calculated. This index can be calculated as follows: follow up score pretest score / Sdiff (standard error of difference between the two test scores). The 92

95 Chapter 4 Sdiff can be calculated according to this formula: (2(SE) 2 ) and SE = s 1 (1- r xx ); with denoting standard deviation of the test; and r xx denoting the reliability of the test. A Reliable Change Index greater than 1.96 is unlikely to occur (p <.05) without actual change. Based on these results the sample was split in three different groups: non-responders, partial responders or responders. Children were considered non-responders to treatment if there was no reliable change on all four measures and if they had a below average score on the Digit Span (standard score < 8) and/or Span Board (T-score < 40) task at follow up (n = 32). Children with no reliable change on all four measures but with average scores on the Digit Span and/or Span Board task at follow up were considered partial responders (n = 65). In case of a Reliable Change Index larger than 1.96 on at least one of the four measures, children were considered responders to treatment (n = 53). Within this group of responders, 20 children showed a reliable change on the Digit Span task, 21 children showed a reliable change on the Span Board task, 12 children showed a reliable change on the teacher rated working memory scale of the BRIEF and 10 children showed a reliable change on the parent rated working memory scale of the BRIEF. There were no statistical differences between the groups in terms of changes in medication between post treatment and follow up. However, there was a significant correlation between treatment received between post -and follow up assessment and response group (r =.197, p <.05); several partial responders (n = 5) and responders (n = 8) received remedial teaching while no single non-responder received remedial teaching. Therefore, receiving remedial teaching between post -and follow up assessment was entered as a covariate in all subsequent analyses. Statistical methods The Statistical Package for Social Sciences, version 19 (IBM SPSS 19), was used for the statistical analysis. Outliers were removed if they had a z-score of < or > 3.29 and were replaced with the second highest value, multiple imputation was used to deal with missing data. As several variables violated the assumption of homogeneity of variance, we chose to use non parametrical tests to analyze the data. Kruskal-Wallis tests for continuous variables and Chi-square tests for dichotomous variables were used to detect group 93

96 The influence of individual differences on treatment outcomes differences for the demographic-, clinical- and baseline neurocognitive variables. Mann-Whitney tests were subsequently undertaken as post hoc analyses. The significance level was set at p <.05 (two-tailed). Multinomial logistic regression analyses were undertaken with the demographic-, clinicaland baseline neurocognitive variables that showed to have a relationship with treatment response, the non-response category was set as reference category. The model was adjusted for the covariate received remedial teaching between post -and follow up assessment. Regressions were then repeated with the response group as reference category in order to compare the partial responders and the response group. To investigate whether the response groups improved on the outcome measure from baseline to follow up, a Wilcoxon signed-rank test was used. Additionally, Kruskal-Wallis tests were used to determine whether the amount of improvement differed for the response groups. Mann-Whitney tests were used for post hoc analyses. Finally, Wilcoxon signed-rank tests were used to investigate whether the children from the non-response group who followed CWMT (n = 6) improved on the outcome measures as described above. Results Between January 2012 and January 2015, a total of 164 children received the PAC intervention. Dropout rate was low with only five children discontinuing the intervention: PAC was too demanding for these children. There were no statistical differences between the baseline characteristics of the drop out children and the children that finished the training. At follow up, parents of one child refused to cooperate any further, data was removed for this participant. Finally, after the data of the eight children which exceeded cut off scores on the SCQ was removed, analyses were performed on the remaining data for 150 children. Baseline comparison response groups Table 1 and 2 show the differences between the three response groups for the demographical, clinical and neurocognitive variables. Baseline 94

97 Chapter 4 differences were found for total IQ score (H(2) = 7.887, p =.019), parent rated externalizing problems (H(2) = 7.643, p =.022), sustained attention (Score!; H (2) = 8.362, p =.015), verbal working memory composite score (H(2) = , p <.001), parent rated Behavioral Regulation Index (H(2) = 7.054, p =.029) and Metacognition Index (H(2) = 6.051, p =.049) and finally teacher rated Metacognition Index (H(2) = 8.786, p =.012). For the total IQ score, sustained attention and verbal working memory composite score post hoc tests were in the expected direction with non-responders showing lower scores than the partial responders and/or responders. For the parent rated externalizing and executive function problems post hoc tests revealed that partial responders had the least problems in comparison to the non-responders and/or responders. Finally post hoc test for the teacher rated Metacognition Index revealed that responders had more problems in comparison to nonresponders and partial responders. Regression analyses Table 3 displays the results of the multinomial regression analyses. There was a good model fit using a deviance criteria, X²(252, N=155) = , p =.885. The baseline variables verbal working memory skills (X²(2) = , p <.001), sustained attention skills (X²(2) = , p =.012) and teacher rated metacognition problems (X²(2) = , p =.001) predicted group membership. Given that the other variables in the model are hold constant, children with higher sustained attention skills at baseline were more likely to be responders (OR: 1.302, 95% CI [1.043, 1.626]) or partial responders (OR: 1.363, 95% CI: 1.093, 1.701) as compared to the non-responders. The likelihood of being responder was similar as compared to being partial responder (OR: 1.047, 95% CI [.889, 1.233]). Higher verbal working memory skills at baseline were associated with an increase in the likelihood of being responder (OR: 1.884, 95% CI [1.244, 2.852]) or partial responder (OR: 2.304, 95% CI [1.521, 3.488]). The likelihood of being partial responder as compared to being responder was similar (OR: 1.223, 95% CI [.961, 1.557]). Finally, higher teacher rated metacognition problems at baseline were associated with an increase in the likelihood of being responder (OR: 1.072, 95% CI [1.024, 1.123]) or partial responder (OR: 1.070, 95% CI [.897, 1.122]) as compared to 95

98 The influence of individual differences on treatment outcomes being non-responder. The likelihood of being partial responder as compared to being responder was similar (OR:.998, 95% CI [.973, 1.024]). In summary, higher initial sustained attention - and verbal working memory skills and more teacher rated metacognition problems increase the likelihood of being a partial responder or responder. Table 1. Demographic and clinical characteristics classified by responder group Nonresponders n = 32 Partial responders n = 65 Responders n = 53 H or X² p Age at training, Mdn (range) 10 (4) 10 (4) 9 (4) Gender Male, n (%) 28 (87.5) 48 (73.8) 40 (75.5) Full-Scale IQ, Mdn (range) 93.5 (41) 102 (60) (59) * Medication for ADHD, n (%) 21 (67.7) 34 (52.3) 35 (66) ADHD subtype, n (%) ADHD-combined ADHD-inattentive ADHD-NOS Unknown Learning Disorder, n (%) Yes No Type of education, n (%) Regular primary Special primary Special education SES, n (%) Low < Average High > CBCL scale, Mdn (range) Attention problems Externalizing problems TRF scale, Mdn (range) Attention problems Externalizing problems 23 (71.9) 5 (15.6) 1 (3.1) 3 (9.4) 8 (25) 24 (75) 27 (84.4) 5 (15.6) 0 3 (12) 7 (28) 15 (60) 67 (31) 61 (36) 64 (29) 56 (34) 38 (58.5) 17 (26.2) 2 (3.1) 8 (12.3) 16 (24.6) 49 (75.4) 58 (89.2) 5 (7.7) 2 (3.1) 11 (19.6) 12 (21.4) 33 (58.9) 64 (32) 54 (44) 61 (31) 58 (38) 35 (66) 10 (18.9) 3 (5.7) 5 (9.4) 12 (22.6) 41 (74.4) 49 (92.5) 3 (5.7) 1 (1.9) 7 (14.6) 9 (18.8) 32 (66.7) 67 (33) 57 (46) 62 (34) 57 (38) ** Note. ADHD-NOS = Attention Deficit Hyperactivity Disorder Not Otherwise Specified; SES = Social Economic Status; CBCL = Children s Behavior Checklist rated by parents; TRF = Teacher Report Form. * = non-response group < partial responders - and response group ** = partial responders group < non-response - and response group 96

99 Chapter 4 Table 2. Neurocognitive outcome measures for the different responder groups Non-responders Partial responders Responders Baseline Follow up Z Baseline Follow up Z Baseline Follow up Z H baseline between groups H Δ T0-T2 between groups Score! 7.1 (2.1) 7.5 (3) (2.9) 8.5 (3.6) (2.7) 8.9 (3.1) * Creature Counting Correct Time 8.9 (2.8) 9 (3.1) 9.3 (3) 9.8 (2.4) (2.6) 8.6 (3.2) 10.8 (2.7) 10.2 (2.7) ** 8.8 (3.1) 9 (3.1) 9.6 (2.9) 10.5 (2.6) * * Verbal WM 8.3 (1.5) 9.4 (1.5) ** 10.6 (1.8) 11.7 (1.6) ** 9.8 (1.8) 11.3 (1.7) ** * Span Board 42.8 (8.8) 45.1 (8) * 46 (8) 50.7 (6.7) ** 42.4 (9.1) 52.3 (8.3) ** ** Six part test 9.2 (2.4) 9.7 (2.8) (2.1) 10.1 (2.6) ** 8.9 (2.6) 10.1 (2.4) * Inhibition Correct Time 7.9 (4) (25.3) 5.6 (3.3) 91.7 (19) ** ** 8 (4.7) (27.4) 5.2 (2.9) 96.3 (21.3) ** ** 10 (8.1) 110 (20.7) 4.7 (3.3) 94.3 (17) ** ** BRIEF parent BRI MCI 56.7 (9.3) 59.9 (10.7) 54 (9.3) 56.5 (11.1) ** ** 52.8 (8.2) 57 (8.6) 50.6 (9.2) 54.8 (8.8) * * 55.9 (9.6) 61.2 (8.5) 52.1 (10.1) 55.9 (11.4) ** ** 7.054* 6.051* BRIEF teacher BRI MCI 58.8 (13.2) 62.6 (11.7) 59.4 (11.6) 59.2 (8.5) (14.3) 66 (18.8) 59 (18.5) 61 (14.8) * * 61.9 (14) 72.1 (18.3) 54.4 (11.7) 59.5 (14.2) ** ** * * Note. WM = working memory; BRIEF= Behavior Rating Inventory of Executive Functions; BRI = Behavioral Regulation Index; MCI = Metacognition Index * p <.05 ** p <.01 97

100 The influence of individual differences on treatment outcomes Table 3. Multinomial logistic regression predicting the response groups: non-responders, partial responders and responders on the basis of clinical and neurocognitive variables. 95% CI for Odds Ratio B (SE) Odds Ratio Lower Upper a non-response vs. partial responders Intercept (4.06) Total IQ.019 (.02) Externalizing problems (.05) Attention.263 (.109) ** Verbal working memory.830 (.208) *** BRIEF-P BRI.011 (.06) BRIEF-P MCI (.04) BRIEF-T MCI.068 (.02) ** a non-responders vs. responders Intercept (4.15) Total IQ.022 (.02) Externalizing problems (.05) Attention.264 (.113) * Verbal working memory.633 (.212) ** BRIEF-P BRI (.06) BRIEF-P MCI.014 (.04) BRIEF-T MCI.070 (.02) ** b partial responders vs. responders Intercept (2.99) Total IQ (.02) Externalizing problems (.03) Attention.046 (.08) Verbal working memory.201 (.12) BRIEF-P BRI.036 (.04) BRIEF-P MCI (.03) BRIEF-T MCI (.01) Note. BRIEF = Behavior Rating of Executive Functions; P = parent; T = teacher; BRI = Behavioral Regulation Index; MCI = Metacognition Index a Non-response group set as reference category, R² =.354 (Cox & Snell),.403 (Nagelkerke). b Response group set as reference category.*p <.05 **p <.01 ***p <

101 Chapter 4 Treatment effectiveness Primary outcome measures As can been seen in Table 2, the non-responders improved on measures of verbal working memory (z = , p <.001), visual spatial working memory (z = , p =.044), inhibition (Correct z = , p =.005; Time z = , p =.001) and parent rated executive functioning (Behavioral Regulation Index z = , p =.003; Metacognition Index z = , p =.003) six months after treatment. Partial responders improved on measures of verbal working memory (z = , p <.001), visual spatial working memory (z = , p <.001), inhibition (Correct z = , p <.001; Time z = , p <.001) and parent rated executive functioning (Behavioral Regulation Index z = , p =.023; Metacognition Index z =-2.256, p =.024), but also on attention (Creature counting Time z = , p <.001), planning (z = , p <.001) and teacher rated executive functioning (Behavioral Regulation Index z = , p =.048; Metacognition Index z = , p =.017). The responders improved on measures of attention (Creature counting Correct z = , p =.035; Time z = , p =.019), verbal working memory (z = , p <.001), visual spatial working memory (z = , p <.001 ), planning (z = , p =.018), inhibition (Correct z = , p <.001; Time z = , p <.001), parent rated executive functioning (Behavioral Regulation Index z = , p =.010; Metacognition Index z = , p <.001) and teacher rated executive functioning (Behavioral Regulation Index z = , p <.001; Metacognition Index z = , p <.001). None of the groups improved on the measure of sustained attention (Score!). The responders showed a larger improvement on visual spatial working memory (H(2) = , p =.001) and teacher rated metacognition problems (H(2) = 7.858, p =.020) in comparison to the nonresponders (U = , p =.001; U = , p =.009) and partial responders (U = , p =.006; U = , p =.032). In terms of academic outcome measures, it can be seen in Table 4 that the non-responders did not improve on any of the word reading, mathematic, or spelling measures. The partial responders improved on word reading only (z = , p =.025). For the responders there was a trend for improvement on word reading (z = , p =.059) and spelling (z = , p =.059) and a significant improvement on mathematic (Plus and minus z = , p = 99

102 The influence of individual differences on treatment outcomes.039; Total z = -.486, p =.627). The improvements for the responders were not significantly larger than the improvements for the partial responders or nonresponders. Secondary outcome measures In terms of behavior problems, non-responders improved on parent rated attention (z = , p =.001) and externalizing problems (z = , p =.002) six months after treatment, but not on teacher rated attention - and externalizing problems. The partial responders improved both on the parent and teacher rated attention - (Parent z = , p =.002; Teacher z = , p =.003) and externalizing problems (Parent z = , p =.002; Teacher z = , p =.043). Finally, the responders also improved both on the parent and teacher rated attention (Parent z = , p <.001; Teacher z = , p <.001) and externalizing problems (Parent z = , p <.001; Teacher z = , p =.020). The improvements for the responders were not significantly larger than the improvements for the partial responders or non-responders. For the outcome measure behavior in class, results showed that the nonresponders did not improve on any of the subscales. The partial responders improved on the Relation - (z = , p =.004) and Direct Learning Conditions scale (z = , p <.001). The responders also improved on the Relation - (z = , p =.004) and Direct Learning Conditions scale (z = , p <.001). Additional analyses showed that the groups differed in terms of the amount of improvement on the Direct Learning Conditions scale, H(2) = 8.775, p =.012. Non-responders improved less in comparison to the partial responders (U = , p =.009) and responders (U = , p =.007). For the outcome measure quality of life, results showed that the nonresponders improved on the parent rated autonomy & parents relations - (z = , p =.025) and school environment scale (z = , p =.012). Partial responders did not improve on any of the subscales. Responders improved on the psychological well-being - (z = , p =.008), peers - (z = , p =.010) and school environment scale (z = , p =.001). Groups differed in terms of amount of improvement for the subscales psychological well-being 100

103 Chapter 4 (H(2) = 6.574, p =.037), autonomy & parents relations (H(2) = 6.042, p =.049) and school environment (H(2) = 8.024, p =.018). For the psychological wellbeing scale, the responders showed a larger improvement in comparison to the partial responders (U = , p =.010). For the autonomy & parents relations scale, the non-responders showed a larger improvement in comparison to the partial responders (U = , p =.029). And finally, for the school environment scale, the responders showed a larger improvement in comparison to the partial responders (U = , p =.007). For the child rated version, results showed that non-responders improved on the physically well-being subscale (z = , p =.038). No improvements were found for the partial responders and responders. In summary, while the improvements for non-responders are limited to only several neurocognitive outcome measures and parent rated questionnaires, partial responders and responders improve on almost all outcome measures. Responders show larger improvements than partial responders and/or non-responders on verbal working memory, teacher rated metacognition problems, teacher rated direct learning conditions and parent rated psychological well-being and school environment. Treatment for non-responders Six of the 32 children who were classified as non-responders additionally followed Cogmed Working Memory Training (CWMT). Demographical and clinical variables of the children who followed CWMT were not statistically different from the non-response children that did not follow CWMT. However, there was a trend for Total IQ score (U = , p =.053); children who additionally followed CWMT had a larger Total IQ score (Mdn = 105) than children who did not additionally follow CWMT (Mdn = 93.5). Results showed that children improved on visual spatial working memory from pretreatment (Mdn = 40.5) to directly after treatment (Mdn = 55), z = , p <.05. There were no significant improvements on other neurocognitive and academic outcome measures. 101

104 The influence of individual differences on treatment outcomes Table 4. Learning efficiency quotients of academic performance measures. Non-responders Partial responders Responders Baseline Follow up Z Baseline Follow up Z Baseline Follow up Z H Δ T0-T2 between groups Word reading.81 (41).68 (.37) (.53).83 (.50) *.82 (.33).88 (.44) a.2362 Automated math Plus and minus Total.62 (.26).71 (.27).62 (.29).63 (.24) (.27).65 (.21).65 (.26).61 (.23) (.31).72 (.29).76 (.36).68 (.24) * Spelling.65 (.30).60 (31) (.33).73 (.33) (.30).74 (.36) a *p <.05 a trend difference (p =.06) 102

105 Chapter 4 Discussion The aim of the present study was to extend our understanding of individual differences in treatment outcome of an innovative cognitive training called Paying Attention in Class within a sample of children with ADHD. The first aim was to establish which demographical, clinical or neurocognitive characteristics predicted treatment outcome. Based on clinically significant improvements in working memory, we identified 32 non-responders, 65 partial responders and 53 responders to treatment. Predictors of treatment response Results showed that initial sustained attention skills, initial verbal working memory skills and teacher rated metacognition problems predicted the membership of treatment response group. Intelligence and parent rated externalizing and executive function problems differed at baseline between the response groups but were not predictive of treatment response. Children with low initial verbal working memory skills, low initial sustained attention skills and low teacher rated metacognition problems were more likely to be nonresponders. None of the predictors differentiated between the likelihood of being a partial responder as compared to being a responder. In terms of the verbal working memory and attention skills, these results suggest that individuals who perform better on cognitive tasks could benefit more from training which would be in line with the so called magnification account or Matthew effect. This account assumes that individuals who are already performing better at baseline will benefit most from cognitive training, as they will have more efficient cognitive resources to acquire and implement new strategies and abilities (Titz & Karbach, 2014). However, this Matthew effect does not withstand when we consider the finding that children with high teacher rated metacognition problems at baseline were more likely to be responders. Although the responders showed to have more teacher rated metacognition problems at baseline than the non-responders, they also showed the largest improvements in this area. One should hold in mind here that, given the fact that the classification 103

106 The influence of individual differences on treatment outcomes of response groups was partially based on a subscale (working memory) of this Metacognition Index, these results might have been biased. Almost one quarter of the responders in current study (n = 12) were identified as responders based on their teacher rated improvement on working memory. Transfer effects for different response groups The second aim of this study was to investigate whether the clinically relevant improvement in working memory would be a prerequisite to obtain improvements in far transfer measures as well. Although the non-responders improved (statistically) on near transfer measures of verbal and visual spatial working memory, which most likely can be attributed to the daily practice with adaptive working memory tasks, far transfer was limited to parent rated improvements on executive functioning behavior. In contrast to the nonresponders, both the partial responders and responders improved on most far transfer outcome measures. However, the most profound effects were found for the responders; they improved on broad neurocognitive measures, a math task, behavior and executive function problems both rated by teacher and parents, behavior in class and parent rated quality of life scales. Compared to non-responders and partial responders, they benefitted significantly more in terms of visual spatial working memory, teacher rated metacognition problems, direct learning conditions (i.e. concentration, motivation, work rate, task orientation) and the parent rated quality of life scales psychological well-being and school environment. We suggest that there are several explanations for this observed positive relationship between clinically significant improvement in working memory and broader transfer. First of all, it has been repeatedly shown that working memory skills are involved in many cognitive processes such as learning, comprehension and reasoning (Baddeley, 2007). For example, children with low working memory are typically rated by their teachers as being inattentive, highly distractible, forgetful in instructions and failing to complete tasks (Alloway, Gathercole, Kirkwood & Elliot, 2009). Therefore, it is plausible that the observed beneficial effects for responders in terms of teacher rated questionnaires derived from this clinical improvements in working memory. Secondly, it is quite likely that 104

107 Chapter 4 children with larger training gains receive more positive feedback from their environment, which in turn could increase those children s general psychological well-being and self-esteem. Finally, responders higher attention and verbal working memory skills probably made it easier to acquire and implement the provided strategies and skills. In turn this could have given them more insight in their cognitive abilities and showed them that they can exert influence on situations within the classroom. Subsequently, as these children gain more control of their behavior in the classroom they might need less support from their teachers. Irrespective of these potential explanations or underlying working mechanisms, results show that a clinically significant improvement in working memory coincides with improvements in other clinically relevant areas of functioning in everyday life. Additional CWMT for non-responders Finally, results showed that non-responders only improved on a measure of visual spatial working memory after additional CWMT. Given the high visual spatial character of most tasks within CWMT, this improvement in visual spatial working memory could be viewed as a practice effect. These results imply that even a more intensive and additional working memory training like CWMT is not sufficient to compensate for the deficits of these non-responders and therefore they are in dire need of adjusted treatment protocols. A recent study of Ottersen & Grill (2015) showed that children with intellectual disabilities benefit from working memory training, also in terms of non-trained tasks, if training sessions were extended and if the trained tasks were less demanding. With regard to the new PAC intervention, as well as other cognitive training programs, future research should consider adjusting the pace of providing these children with new skills and strategies and also adapting the algorithm of the adaptive executive functioning tasks. Additionally we suggest that children with this vulnerable cognitive and behavioral profile should receive more assistance within the classroom so that they can optimally practice with - and fully benefit from the new acquired skills. 105

108 The influence of individual differences on treatment outcomes Limitations Several limitations of current study have to be addressed. First, the results of the questionnaires should be interpreted with caution as (a) parents and teachers were aware that children received an active treatment, (b) teachers were actively involved in the treatment and (c) standard deviations were quite large for the teacher rated BRIEF. Given the potential crucial role of the teacher, it would be prudent to add objective measures that could rate the amount of teacher involvement in future studies. Second, the current study design did not contain a control group. Therefore the potential influence of effort, practice or parent and teacher expectancy effects cannot be ruled out. Future studies should include adequate control groups and well blinded assessments of (classroom) behavior. Third, we did not correct for multiple testing which could have increased the likelihood of a Type I error. On the other hand, a Bonferroni correction would increase the probability of Type II error which in turn would lead to more conservative results. Fourth, the results of the treatment effects for the non-responders who followed additional CWMT should be interpreted with caution as the sample size was small. In most cases, parents of children who chose not to follow additional CWMT indicated that their children already had a busy schedule of extra curricular activities or parents indicated that training at home would be too strenuous for the family. Finally, it would be of great clinical value to identify nonresponders directly after training instead of 6 months after treatment as we did in current study. Additional treatment could then be initiated immediately which could prevent children from falling behind even more. Conclusion In summary, the current study shows that initial attention and verbal working memory skills and teacher rated metacognition problems at baseline predicted treatment response of a new cognitive training. In addition, the current study has shown that individual differences in clinically significant improvement in working memory influence the overall near and far transfer efficacy of cognitive training for children with ADHD. Future studies and intervention designs should (a) take into account both cognitive and behavioral baseline individual differences and focus on the interaction with 106

109 Chapter 4 the environment in which children need to apply their new acquired skills (e.g. classroom) and, (b) adapt to the needs of children with a vulnerable cognitive and behavioral profile in terms of developing adapted treatment approaches. 107

110

111 Chapter 5 Individual differences in training gains and transfer measures: an investigation of training curves in children with Attention- Deficit/Hyperactivity Disorder Marthe van der Donk, Sietske van Viersen, Anne-Claire Hiemstra-Beernink, Ariane Tjeenk-Kalff, Aryan van der Leij & Ramón Lindauer Manuscript under review

112 An investigation of training curves Abstract Currently, evidence for the beneficial effects of working memory training on transfer measures in children with ADHD is inconsistent. Although there is accumulating evidence for the role of individual differences in training and transfer gains of cognitive training, this area has been left unexplored for children with ADHD. In the current study, an advanced latent growth curve model (LGCM) analysis was used to investigate the individual differences in learning curves of working memory training tasks within a new cognitive intervention Paying Attention in Class (PAC). It was investigated whether certain baseline variables (age, IQ, externalizing behavior problems and presence of learning disability) could predict the learning curves and how these individual learning curves influenced near and far transfer measures. A total of 164 children diagnosed with ADHD, between age 8-12, followed this new PAC intervention. Working memory (near transfer) and academic performance (far transfer) measures were assessed before treatment and directly after treatment. Results showed that individual differences at the start of training were predicted by age and intelligence, but the individual differences in learning curves were not predicted by any of the baseline variables. Children with larger gains on the trained verbal working memory task showed larger gains on a near transfer verbal working memory measure. No effects were found for visuospatial working memory gains and the academic performance measures. Current study shows that training WM is quite complex and has its limitations for children with ADHD. Nonetheless, it highlights that training and transfer gains are affected by many different factors and warrants the need of a more in-depth investigation of individual differences in future studies. 110

113 Chapter 5 Introduction The last decade, there has been a shift of focus for treatment modalities for children with Attention-Deficit/Hyperactivity Disorder (ADHD) from directly ameliorating behavioral symptoms towards improving the underlying neurological substrates and core cognitive deficits assumed to mediate ADHD causal pathways. Although evidence suggests that most children with ADHD have deficits on multiple executive functions (Willcutt, Doyle, Nigg, Faraone & Pennington, 2005), working memory (i.e. actively holding in mind and manipulating information relevant to a goal) deficits have been reported most frequently (Martinussen, Hayden, Hogg-Johnson & Tannock, 2005) and are at the core of this disorder according to some causal models for ADHD (e.g. Castellanos & Tannock, 2002). Studies have shown that working memory (WM) capacity can be improved in children with ADHD by computerized cognitive interventions such as Cogmed Working Memory Training (CWMT), which includes repeated practice of increasingly demanding WM tasks (Klingberg, Forssberg & Westerberg, 2002). Apart from improvements on the trained task and domain (i.e. near transfer), clinicians and researchers are specifically interested in the transferability of training-related improvements in tasks and domains that have not been trained (far transfer). Unfortunately, meta analyses and review studies so far have shown that far transfer effects of cognitive interventions such as CWMT have been inconsistent and insufficiently supported for children with ADHD (e.g. Cortese et al., 2015; Melby-Lervåg & Hulme, 2013; Orban, Rapport, Friedman & Kofler, 2015; Rapport, Orban, Kofler & Friedman, 2013; Sonuga-Barke et al., 2013). However, by focusing on the differences between pretest and posttest performance, as happened in most of these review studies, important information regarding individual differences in treatment outcomes is neglected. There is accumulating evidence for the role of individual differences in cognitive training (Könen & Karbach, 2015). Studies have shown that factors such as age, genetic predisposition, motivation, personality, prior treatment 111

114 An investigation of training curves and initial cognitive ability influence training gains and transfer measures (Jaeggi, Buschkuehl, Shah & Jonidas, 2014; Karbach & Unger, 2014; Titz & Karbach, 2014; Von Bastian & Oberauer, 2013). However, evidence is lacking in terms of the potential influence of these individual differences on cognitive training outcomes in children with ADHD. Some suggest that individual differences in prior treatment, motivation, use of medication during training or initial WM skills could influence cognitive training outcome in children with ADHD (Rutledge, van den Bos, McClure & Schweitzer, 2012; Shah, Buschkuehl, Jaeggi & Jonides, 2012; Shinaver, Entwistle & Söderqvist, 2014). Next to the potential influence of baseline individual difference on transfer measures, there are also indications that individual differences during training affect transfer measures. For instance, a study of Jaeggi, Buschkuehl, Jonidas and Shah (2011) showed that increase on untrained fluid intelligence tasks was only apparent for children who considerably improved on the trained WM task. More specifically, it has been suggested that individual training curves should be taken into account as a large variability in learning rate could bias the average learning curve of the group (Jolles & Crone, 2012) and affect outcome measures (Moreau, 2014). For example, a study of Söderqvist, Nutley, Otterson and Klingberg (2012) in children with intellectual disabilities showed that there was a large inter-individual variance in training progress after a combined training of visual spatial WM and nonverbal reasoning. They found that training progress was predicted by gender, comorbidity and baseline capacity of verbal WM and, more interestingly, larger improvements during training were associated with larger training gains in untrained WM tasks. This within-person approach is even more crucial when the trained individuals come from a heterogonous population (Könen & Karbach, 2015), which would apply to individuals with ADHD considering the clinical and pathophysiological heterogeneity of the disorder (Willcutt et al., 2012). In sum, evidence is lacking for both the potential influence of baseline individual differences and the potential influence of individual training gains on cognitive training outcomes in children with ADHD. Therefore, the current study investigated the influence of individual learning curves on transfer gains 112

115 Chapter 5 on the one hand and assessed the influence of potential predictors on those learning curves on the other hand. Following the suggestion and example of Bürki, Ludwig, Chicherio and Ribaupierre (2014), data of the current study was analyzed by means of latent growth curve modeling (LGCM). This has the benefit of examining both intra-individual (within-person) change over time as well as inter-individual (between-person) variability in intra-individual change. Additionally, it allows for the investigation into antecedents (i.e. predictors) and consequents (i.e. transfer) of change (Preacher, Wichman, MacCallum & Briggs, 2008). In the current study, a sample of 164 children with ADHD received a new cognitive training called Paying Attention in Class (PAC) which contains a process-based WM training with an additional innovative classroom embedded compensatory approach. First of all, based on the WM training data of this intervention, LGCM analysis was applied to examine the individual differences in learning curves. Second, we investigated whether age, intelligence, initial parent-rated externalizing behavior problems and presence of a comorbid learning disability predicted these individual differences in learning curves. Finally, we examined the link between these individual differences in learning curves and transfer effects on verbal and visuospatial WM (near transfer) and academic outcome measures (far transfer). The putative mechanism behind WM training is the assumption that extensive training of WM strengthens the common and overlapping neural executive function network which in turn leads to improvements in untrained tasks or activities that rely on the same neural network (Klingberg, 2010). This would imply that the more WM capacity increases during training, the larger the transfer effects will be. Therefore we expected that children with larger training gains (i.e. steeper learning curves) would show larger benefits in transfer measures. Academic performance measures were used as indications for far transfer given the strong relationship between WM and academic success (for overview see Titz & Karbach, 2014). 113

116 An investigation of training curves Method Participants The data were obtained from children who participated in two separate studies. The first study included 48 children who were allocated to the Paying Attention in Class (PAC) intervention from our previous randomized controlled trial (Van der Donk, Hiemstra-Beernink, Tjeenk-Kalff, Van der Leij & Lindauer, 2015). For the second study, 125 children were assessed for eligibility; nine children did not meet inclusion criteria and were excluded. A total of 116 children were included and started with the PAC intervention. Eligible participants were (a) children between the age of 8 and 12 years, (b) diagnosed with ADHD (all subtypes) by a professional according to the guidelines of the Diagnostic and Statistical Manual of Mental Disorders DSM- IV-TR (APA, 2000). Children with comorbid learning disabilities (LD) and/ or Oppositional Defiant Disorder (ODD) according to the DSM-IV-TR (APA, 2000) were also included. Children on medication were only included when they were well adjusted to their medication, which meant that they were not participating in a medication trial, and type and dosage of medication was unchanged at least 4 weeks prior to the start and during the training. Exclusion criteria were (a) presence of psychiatric diagnoses other than ADHD/LD/ODD, (b) Total IQ < 80, (c) significant problems in the use of the Dutch language and (d) severe sensory disabilities (hearing/vision problems). Procedure The ethics approval for this study was obtained from the Medical Ethical Committee (2011_269) at the Academic Medical Centre in Amsterdam, the Netherlands. Children were recruited in two different ways for this study. First, clinicians from two clinical care departments of De Bascule (Academic Centre for Child and Adolescent Psychiatry, Amsterdam) referred eligible children to the researcher. Second, healthcare staff members (usually remedial teacher or school psychologist) of schools in the region of Amsterdam contacted the researcher when they had eligible children. In both cases, the researcher visited the school for an information meeting to extensively inform the staff members. Parents of children who met criteria for 114

117 Chapter 5 participation were approached and informed by the school staff member. After informed consent was obtained, parent(s) and teachers were invited to participate in an information meeting at school where they were informed about the content of the intervention. Two to three weeks prior to treatment, parents and teachers received several questionnaires via or hard copy on request. Parents filled out an application package containing a written informed consent form, questionnaires of demographic- and background information and the Dutch translation of the Social Communication Questionnaire (SCQ; Warreyn, Raymaekers & Roeyers, 2004) to screen for autism spectrum disorder. The Lifetime version of the SCQ consists of 40 questions that have to be answered with yes or no. A total raw score of 15 or higher indicates a likelihood for the presence of autism spectrum disorder and is recommended as a cutoff-score, a total of eight children exceeded this cutoff-score. Therefore, we decided to remove data of these children from subsequent analyses. See Table 1 for the sample characteristics. Parents were also asked to send a copy of the diagnostic psychiatric report of their child to establish the subtype of ADHD and rule out other potential psychiatric problems that met exclusion criteria. One week prior to treatment, WM - and academic performance was assessed by a member of the research team in a silent room at school. Treatment took place at school and all sessions were completed during morning school hours. Training periods were planned in between school holidays, to ensure that training sessions would not be interrupted for a longer period of time. Post treatment assessment took place within one week after the last training. Intervention Paying Attention in Class PAC is an experimental combined WM and compensatory training that has been developed by our research team. Children were trained individually at school (though outside the classroom) for 5 weeks, 5 times a week, for approximately 45 minutes a day. This PAC intervention contains three key elements; psycho-education about five executive functions that are important in a learning situation, paper and pencil adaptive WM training and facilitating generalization to the classroom-situation. Analyses in the present 115

118 An investigation of training curves study were solely based on the adaptive WM training, a more extensive description of other parts of this intervention can be found in the publication of our randomized controlled trial (Van der Donk et al., 2015). The paper and pencil adaptive WM training contains three tasks: a visual spatial span task, a listening recall span task, and an instruction paradigm task (30 trials in total), which are practiced on a daily basis in order to increase WM capacity. In the listening recall task (hereafter referred to as verbal WM training), the coach reads aloud a certain amount of sentences and the child has to evaluate whether the particular sentence is true or false. After this, the child has to reproduce the last word of each sentence in the correct order. For example Fish have legs; true or false? Tomatoes are usually red; true or false? Repeat the last word of each sentence. The visual spatial span task (hereafter referred to as visuospatial WM training) is a paradigm of the Corsi block-tapping task (Corsi, 1972), which consists of a template with ten small blocks. The child has to tap the same cubes as the coach in the reversed sequence. The instruction task was based on a previously described analog task (Gathercole, Durling, Evans, Jeffcock & Stone, 2008) and consists of a paper template and cards that contains pictures of school related items. The coach reads aloud an instruction that the child has to execute, for example Point to the big circle and pick up the small blue pen. The algorithm was similar for all tasks; after two correct trials at a certain level, children moved on to the next higher level. After two incorrect trials at a certain level, children returned to the adjacent lower level. Each WM task was ended after ten executed trials. At the end of each session, the child and therapist filled out a high score list for each task to keep track of the performance, the raw scores of these high score lists were used for the analyses. After initial data screening it was decided to focus on the listening recall task and visual spatial span task as many children reached a ceiling effect on the instruction task. Compliance of the WM training was met after 20 sessions (4 weeks of training) as has been reported in previous WM studies (e.g. Klingberg et al., 2005). Since data of the fifth and final week of the training (i.e., 5 sessions) was not available for all children, data of this last training week was left out of the analyses. 116

119 Chapter 5 Table 1. Sample Characteristics N = 154 Age at training in months, M (SD) (14.5) Gender Male, n (%) 118 (78.1) Full-Scale IQ, M (SD) (1.11) Medication for ADHD, n (%) 91 (60.3) ADHD subtype, n (%) ADHD-combined ADHD-inattentive ADHD-Not Otherwise Specified Unknown Comorbid Learning Disability, n (%) Yes No Type of education, n (%) Regular primary Special primary Special education Enrollment, n (%) Clinical care School SES, n (%) Low < Average High > Not reported Ethnicity, n (%) Mother Dutch Father Dutch 98 (64.9) 31 (20.5) 6 (4) 16 (10.6) 34 (22.5) 117 (77.5) 138 (88.5) 14 (9) 3 (1.9) 30 (19.2) 125 (80.1) 22 (14.1) 28 (17.9) 82 (52.6) 23 (14.7) 107 (68.6) 102 (65.4) Externalizing behavior problems Parent rating, M (SD) 56.8 (.8) Note. SES = social economic status Measures Predictors The variables age, intelligence, parent-rated externalizing behavior problems and presence of a comorbid learning disability were considered potential predictors of training and treatment outcomes. Age at training was defined by the amount of months between date of birth and the first day of the training. Intelligence quotients were obtained from the received previous diagnostic psychiatric report. A shortened version of the WISC-III-NL (Wechsler, 2005) with the subtests Picture Completion, Similarities, Block Design and Vocabulary was administered to estimate the total intelligence quotient (IQ) if there were 117

120 An investigation of training curves no prior recordings available. Initial externalizing problems were based on the subscale Externalizing Problems of the parent-rated Child Behavior Checklist for Ages 6-18 (Verhulst, van der Ende & Koot, 1996). This subscale consist of the two problem-scales rule breaking behavior and aggressive behavior; a T-score of 60 or higher is considered as problematic. Information about the presence of a learning disability was based on parental reports on the application form and the diagnostic psychiatric report. Children were divided into two groups: learning disability either present or not present. In the present group, no distinction was made between the type of learning disability, as otherwise sample sizes of the different groups would have been very small. The present group consisted of children with the following diagnoses: Dyslexia (n = 32), Dyscalculia (n = 1), Learning Disability NOS (n = 3). Treatment outcomes Near transfer was based on a composite verbal WM score (subtest Digit Span WISC-III: Wechsler, 2005; Comprehension of Instruction and Word List Interference: Dutch translation nepsy-ii: Zijlstra, Kingma, Swaab & Brouwer, 2010) and a visual spatial WM score (subtest Span Board WNV: Wechsler & Naglieri, 2008). Far transfer was based on three academic performance measures: word reading fluency, automated math and spelling. Word reading fluency was measured with the Een Minuut Test (Brus & Voeten, 1973), which consists of two parallel cards (version A & B) which each hold 116 words. The child receives the instruction to read out loud (fast and accurately) as many words as possible in one minute. The Tempo Test Automatiseren (De Vos, 2010) was used to measure the degree of automated math. The test consists of four subtests: addition, subtraction, multiplication and division calculations. For each subtest, the child has to make as many sums as possible in two minutes, with a maximum of 50. Additionally, a composite score of all four subtests and a composite score of the addition and subtraction subtests can be also calculated. In the current study many children were not capable of finishing the multiplication and division calculations due to their young age. Therefore, only the composite score of the addition and subtraction subtests was used as an outcome measure. The PI-dictee (Geelhoed & 118

121 Chapter 5 Reitsma, 1999) was used to measure word level spelling skills and consists of two parallel versions (A & B). Each version consists of 135 words that are divided in nine blocks of fifteen words each. For each word, a sentence is read aloud and the child is asked to write down the repeated word. From a time-saving point of view, not all blocks were administered. The starting point was determined by the educational age of the child and only if there were three or more mistakes in that block, the previous block was also administered. The test was ended if the child made eight or more mistakes in one block. For all outcome measures, pretest-adapted scores were used in the analyses to control for individual differences in untrained transfer tasks at the start of the intervention. These scores were computed by regressing the raw posttest score on the variables raw pretest score and squared raw pretest score and saving the unstandardized residuals (see McGrath et al., 2011 for a comparable approach with age-residuals). This resulted in pretestresidualized scores for verbal WM, visuospatial WM, word reading, spelling and automated math. Statistical methods Two separate LGCMs were fitted to investigate individual differences in learning rate of the verbal WM (listening recall) and visuospatial WM (visual spatial span) training across four weeks of training. In addition, we examined the effect of age, IQ, externalizing behavior problems, and comorbid learning disabilities on the latent growth factors. The outcome measures verbal WM, word reading fluency and spelling were added to assess near and far transfer effects of the individual differences in growth of the trained verbal WM training. For the individual differences in growth of the trained visuospatial WM training, the outcome measures visuospatial WM and automated math were added. We used the following fit measures and rules of thumb to evaluate exact and approximate model fit: Chi-square value (χ 2 ) with associated p-value (good fit = non-significant p-value), RMSEA including 90% Confidence Interval (CI) and pclose (good = <.05, acceptable = <.08, pclose >.05), CFI (good = >.95, acceptable = >.90), and SRMR (good = <.05, acceptable = <.08; Kline, 2011; Little, 2013). The R package lavaan for structural equation modeling (SEM; R Core Team, 2015) was used for the analyses. 119

122 An investigation of training curves Results Data screening Between January 2012 and January 2015 a total of 164 children received the Paying Attention in Class intervention. Data was removed for the eight children that exceeded cut off scores on the SCQ. Dropout rate was low with only five children discontinuing the intervention, treatment was too demanding for these children. For two of those drop out children no data was available in terms of training progress due to very early discontinuation, therefore these participants were completely removed from the dataset. There were no statistical differences between the baseline characteristics of the drop out children and the children that finished the training. See Figure 1 for a flow chart of current study. For the visuospatial WM training, data was analyzed for 154 children. For the verbal WM training, outlier analysis showed that three children were multivariate outliers. Data of these children was removed which resulted in final data analyses of 151 children. Missing data analysis revealed that percentages of missing data were 11% for externalizing behavior, 2.6% for the outcome measures, 13.6% for the verbal WM training sessions and 11.9% for the visuospatial WM training sessions, respectively. Expectation maximization (EM) was used to impute missing data points for the results displayed in Figure 2 and the Appendix. Full information maximum likelihood (FIML) was used to deal with missing data in the LGCM analyses. The number of indicators per growth model was reduced from 20 to 10 to facilitate the estimation procedure; test results of all odd sessions (i.e., day 1, 3, 5, 7,, 19) were used for the analyses and reported here, while the test results of all even sessions (i.e., day 2, 4, 6,.., 20) were used to cross-validate the results. The findings were comparable for both sets of indicators. Latent growth curve modelling The correlations among the verbal and visuospatial WM levels at the 10 (odd) sessions, the predictors (i.e., age, IQ, externalizing behavior, learning disability), and transfer variables (i.e., verbal WM, visuospatial WM, word reading, spelling, automated math) are provided in the Appendix. 120

123 Chapter 5 Figure 1. Flow diagram Enrollment Assessed for eligibility study 2 (n=125) Excluded (n=9) Not meeting inclusion criteria Followed Paying Attention in Class study 1 (n=48) Exceeded cut off score SCQ (n=8) Paying Attention in Class (n=156) Study 1 (n=46) Treatment Study 2 (n=110) Data removed for children (n=2) without available training progress data Post treatment Analysis Visuospatial working memory training: analysed data (n=154) Verbal working memory training: n=3 multivariate outliers analysed data (n=151) The correlations among the verbal WM training sessions were strong, indicating substantial stability of individual differences in verbal WM development. Age correlated significantly with verbal WM levels at all training sessions. For IQ, correlations with the sessions were only significant towards the end of the training (session 11, 13, 15, 17, 19). Externalizing behavior problems only correlated with verbal WM level at the first training session and with age. Learning disability was significantly correlated with age, IQ, word reading and spelling, but not with any of the sessions. In terms of the outcome 121

124 An investigation of training curves measures, untrained verbal WM was significantly correlated with some of the sessions later on in the training (sessions 11, 13, 15, 19). Finally, word reading showed one significant correlation with verbal WM level at training session 11. For the visuospatial WM training, the correlations across sessions were strong, indicating substantial stability of individual differences in visuospatial WM development. Except for session 13, age correlated significantly with all visuospatial WM training sessions and with the predictors IQ, externalizing behavior problems and learning disability. Furthermore, IQ correlated with visuospatial WM at sessions 7 and 13. In terms of the outcome measures, untrained visual spatial WM correlated significantly with visuospatial WM levels at all sessions except session 1. Finally, automated math did not significantly correlate with any of the sessions or predictors. LGCM was used to investigate variation in growth of verbal WM and visuospatial WM capacity across 10 training sessions. In addition, the effect of four predictors was taken into account and both near and far transfer effects of the training were examined. Starting point was a simple latent growth curve model (i.e., initial model) including two growth parameters; an initial status factor (i.e., intercept) and a linear change factor (i.e., linear slope). The intercept denotes the initial verbal or visuospatial WM level of the children at the start of the intervention. Factor loadings were equal for all 10 indicators (i.e., fixed at 1). The linear slope indicates the children s linear growth in verbal or visuospatial WM ability over the (odd) sessions throughout the training. Factor loadings reflected equal intervals between sessions (i.e., 0, 1, 2,, 9). Figure 2 shows the individual verbal and visuospatial WM capacity development as well as the mean levels per WM modality. Verbal WM training The initial growth model for verbal WM ability showed a poor fit to the data, χ 2 (50) = , p <.001, RMSEA =.14, 90% CI [ ], pc l o s e < , CFI =.84, SRMR =.15. Based on the mean training curve displayed in Figure 2, a quadratic factor, indicating nonlinear growth, was added to the model (factor loadings fixed at 0, 1, 4,, 81). This quadratic model demonstrated the following fit, χ 2 (46) = 92.05, p <.001, RMSEA =.08, 90% CI [ ], pclose 122

125 Chapter 5 =.02, CFI =.95, SRMR =.07, and fitted the data significantly better than the model without the quadratic factor, Δχ 2 (4) = , p <.001. The fit of the quadratic model is insufficient when taking into account the significant chisquare and the 90% CI of the RMSEA exceeding.08. However, since there was no indication of systematic patterns of misfit in the parameter estimates and the CFI and SRMR values were considered sufficient to good (Kline, 2011; Little, 2013), this model was found acceptable for the purposes of our analyses. The variance of the intercept (0.27, SE =.05) was significant, indicating that there were individual differences in children s verbal WM levels at the start of the intervention. The variance of the linear slope (.05, SE =.01) was also significant, indicating individual differences in children s linear growth trajectories. Overall, this quadratic growth model explains about 62% of the observed total standardized variance in growth of verbal WM ability across four weeks of training. The prediction model, including age, IQ, externalizing behavior, and learning disability as predictors of training curves for verbal WM ability, fitted the data sufficiently, χ 2 (74) = , p <.001, RMSEA =.07, 90% CI [ ], pclose =.06, CFI =.95, SRMR =.06. The results showed significant effects of age (p <.001) and IQ (p =.003) on the intercept, with older children and more intelligent children having higher initial levels of verbal WM. There were no significant effects on the linear or quadratic slopes. Together, the four predictors explained 17.4% of the variance in the intercept, 7.4% in the linear slope, and 8.2% in the quadratic slope. The outcome model for the verbal WM training, including untrained verbal WM, word reading, and spelling as transfer measures, showed a sufficient fit to the data, χ 2 (110) = , p <.001, RMSEA =.07, 90% CI [ ], pclose =.09, CFI =.93, SRMR =.06. Several additional direct effects from the predictors to the transfer variables were specified based on the modification indices and implemented and evaluated one by one (see Figure 3). This resulted in a good fit for the final model, χ 2 (107) = , p <.001, RMSEA =.06, 90% CI [ ], pclose =.20, CFI =.95, SRMR =.05. The final model is displayed in Figure 3. The results showed a significant effect of the linear growth factor on 123

126 An investigation of training curves Training session Training session Mean VWM level Mean VSWM level Figure 2. Development of individual (light gray) and average (dark gray) verbal (left) and visuospatial (right) working memory levels over the odd training sessions during four weeks of intervention. 124

127 Chapter 5 the verbal WM outcome, indicating that linear growth in trained verbal WM ability has a positive effect on untrained verbal WM ability (i.e., near transfer effect). In addition, older children and children showing higher levels of parent-rated externalizing behavior had higher word reading fluency scores at posttest. Children with a comorbid learning disability were found to have lower spelling levels. Overall, the final model explained 12.6% of the variance in verbal WM, 11.5% of the variance in word reading fluency, and 8.2% of the variance in spelling ability. An overview of the parameter estimates is provided in the Appendix. Visuospatial WM training For the visuospatial WM training, the initial growth model showed a poor fit to the data, χ 2 (50) = , p <.001, RMSEA =.14, 90% CI [ ], pclose <.001, CFI =.82, SRMR =.19. In line with the growth model for the verbal WM training, we added a quadratic factor to the model, indicating nonlinear growth (factor loadings fixed at 0, 1, 4,, 81). This resulted in a sufficient model fit, χ 2 (46) = 81.53, p <.001, RMSEA =.07, 90% CI [ ], pclose =.09, CFI =.96, SRMR =.08, and a significant improvement in fit compared to the previous model with only a linear growth factor, Δχ 2 (4) = , p <.001. The variances of the intercept (0.25, SE =.06) and linear slope (.09, SE =.02) were all significant. This indicates that the children showed significant variability in their visuospatial WM levels at the start of the intervention and that there were individual differences in their linear growth trajectories. Overall, the model explains about 60% of the observed total standardized variance in growth of visuospatial WM ability over four weeks of training. For the prediction model, including age, IQ, externalizing behavior, and learning disability as predictors, model fit was sufficient, χ 2 (74) = , p =.002, RMSEA =.06, 90% CI [ ], pclose =.21, CFI =.96, SRMR =.06. Again, results showed significant effects of age (p <.001) and IQ (p =.02) on the intercept. Older children and children with higher intelligence had higher initial levels of visuospatial WM. There were no effects on the linear or quadratic slopes. Together, the four predictors explained 21% of the variance in the intercept, 3.2% in the linear slope, and 2.1% in the quadratic slope. 125

128 An investigation of training curves Age IQ Ext. Beh. LD VWM Intercept Linear slope.28 Quadratic slope Word reading Spelling VWM T1 VWM T3 VWM T5 VWM T7 VWM T9 VWM T... VWM T19 Figure 3. Final growth model for the verbal WM training, including age, IQ, externalizing behavior, and LD as predictors and verbal WM, word reading, and spelling as outcomes. Note. LD = learning disability, VWM = verbal working memory. (Cor)relations between latent factors and predictors are standardized. To aid visibility, error terms are not displayed. Solid lines and bold numbers indicate significant effects (p <.05). For the final model, visuospatial WM and automated math were included as outcome variables. This model had a good fit to the data, χ 2 (97) = , p =.006, RMSEA =.05, 90% CI [ ], pclose =.45, CFI =.96, SRMR =.06. The modification indices showed that there were no relevant additional effects from the predictors to the outcome variables. The final model is displayed in Figure 4. The results showed a significant effect of the intercept on the visuospatial WM outcome, indicating that individual differences in the initial 126

129 Chapter Age IQ Ext. Beh. LD VSWM Automated math Intercept Linear slope.15 Quadratic slope VSWM T1 VSWM T3 VSWM T5 VSWM T7 VSWM T9 VSWM T... VSWM T19 Figure 4. Final growth model for the visuospatial WM training, including age, IQ, externalizing behavior, and LD as predictors and visuospatial WM and automated math as outcomes. Note. LD = learning disability, VSWM = visuospatial working memory. (Cor)relations between latent factors and predictors are standardized. To aid visibility, error terms are not displayed. Solid lines and bold numbers indicate significant effects (p <.05). visuospatial WM level of the children at the start of the intervention significantly predict untrained visuospatial WM ability one week post intervention. Overall, the final model explained 19.2% of the variance in visuospatial WM and 1.5% of the variance in automated math. An overview of the parameter estimates is provided in the Appendix. 127

130 An investigation of training curves Discussion The aim of this study was to investigate whether individual differences in working memory (WM) training gains (i.e. learning curves) influenced near and far transfer gains in a sample of children with ADHD. It was also investigated whether certain baseline variables could predict the individual differences in training gains. We expected that children with larger training gains (i.e. steeper learning curves) would show larger transfer gains. To our knowledge this is the first cognitive training study in children with ADHD that takes into account the individual differences in training and transfer gains with an advanced latent growth curve modeling (LGCM) analysis. The verbal WM and visuospatial WM training had several results in common. First of all, the average training curve indicated nonlinear growth following a quadratic curve; initial growth rates were high, as indicated by a steep increase during the first few sessions of the training, and showed subsequent decline towards the end of the study resulting in flattening of the learning curve as the training progressed. According to Jolles and Crone (2012) this decreasing slope of performance is typical for the development of these learning curves and could partly be explained by the different aspects of the tasks that are being trained. For example, children could learn a new strategy that improves their performance tremendously in the beginning of the training, but repeated practice with this same strategy does not add much to performance later in training. The second communality between the verbal and visuospatial WM training was that there were individual differences between children in their initial WM levels at the start of training (intercept) and also individual differences in children s growth trajectories (slope). Third, both the verbal and visuospatial WM prediction models showed that older and more intelligent children had higher initial WM levels, but these variables did not predict individual differences in training gains. These predicting effects of age and intelligence are assumed to be a mere reflection of the typical correlation between WM capacity and age (e.g. Alloway, 2011) on the one hand and WM capacity and intelligence on the other hand (see Titz & Karbach, 2014 for an overview). Finally, initial parentrated externalizing behavior problems and comorbid learning disability did not predict the initial levels or training gains of both models. 128

131 Chapter 5 Specifically for the verbal WM training, results of transfer measures showed that children with larger training gains (i.e., steeper training curves) showed larger benefits on the near transfer measure (untrained verbal WM composite score). In addition, there were some direct influences of the baseline variables on the outcome measures. Age and externalizing behavior problems were found to positively affect word reading skills and the presence of a comorbid learning disability affected spelling levels. The near transfer finding is in line with our hypothesis that children with larger training gains would show larger transfer gains. This hypothesis was based on the assumption of Klingberg (2010) that extensive training strengthens the common and overlapping neural executive network which in turn leads to improvements in untrained tasks or activities that rely on the same neural network. As this assumption stems from the neural correlates of working memory, future studies should include neuro-imaging measures. Unfortunately, we did not find any far transfer effects, illustrating the difficulty of studying training effects and their relation to academic as well as underlying skills. In contrast to the verbal WM training, larger visuospatial WM training gains did not lead to larger benefits on a near transfer visuospatial WM measure. However, there was a predictive effect of individual differences in initial visuospatial WM level (intercept) on the untrained visuospatial WM measure, showing that children with higher initial levels of visuospatial WM also had higher visuospatial WM levels one week after the end of the training. A possible explanation for the absence of an effect of the training gains on a near transfer measure could be that it is much more difficult to improve visuospatial WM for children with lower initial visuospatial WM levels, as would hold for children with ADHD, than for children with higher visuospatial WM ability. Visuospatial WM is considered the most impaired executive function in ADHD (Martinussen et al., 2005). It could be that the resources of the children in this study were just not sufficient to further extend their training gains and that a more extensive or different kind of visuospatial WM training is necessary to compensate for this deficit. For example, Cogmed Working Memory Training contains many trained tasks that tap into the domain of visuospatial WM and studies have shown that visuospatial WM skills of children with ADHD 129

132 An investigation of training curves can be improved after this training (i.e. Holmes et al., 2010; Klingberg et al., 2002; 2005). An alternative explanation for the absence of a training effect for visuospatial WM could be that the near transfer visuospatial WM measure was based on a single task that was almost completely similar to the trained task. When using only a single task as a measure of change for an underlying ability, the score could be driven by ability of interest and other systematic and random influences. Therefore, multiple tasks should be used to measure the ability of interest (Shipstead, Redick & Engle, 2012). Conclusively, the findings of the current study are partly in line with our hypothesis. The influence of individual differences in training gains was only apparent for near transfer measures and limited to the domain of verbal WM training. Despite the absence of transfer to academic performance measures, we suggest that there still is room for improvement in terms of assessing these academic outcome measures. One of the theoretical assumptions why an increase in WM capacity should lead to improvements in academic performance is based on the fact that WM is directly involved in many academic skills (see Titz & Karbach, 2014 for an overview). By this account, immediate improvements in academic performance would only be visible if WM has been acting as a bottleneck for performance (Söderqvist & Bergman-Nutley, 2015). The findings of the current study however refute the bottleneck route. Another account that has been proposed, the learning route, refers to the possibility that increases in WM capacity could help children to pay more attention in the classroom and stay more focused on a task, thereby aiding to the learning process (Söderqvist & Bergman-Nutley, 2015). However, confirmation of this last account would require the inclusion of long term assessments as sufficient time is necessary for the enhanced learning process to take place (Gathercole, 2014). As the present study only included direct measures of academic performance as outcomes, there is still a possibility that this learning route adds to improvements in academic performance measures. A final remark that should be made here is that the correlations between the academic outcome and WM measures in current study were quite low (see Appendix). The academic outcome measures that were used in current study were quite basic tasks which required less 130

133 Chapter 5 WM capacity than more complex tasks such as reading comprehension or mathematic problem solving (Dehn, 2008). Limitations Although the current study has extended our understanding of individual differences in training and transfer gains, and therefore improved the field of cognitive training research for children with ADHD, results should be viewed in light of several limitations. First of all, although measures such as attention skills or behavioral ratings are viewed as important measures of far transfer, they were not included in current analyses. LCGM analysis is not without limits and, given the sample size, choices had to be made in terms of relevant predictors and outcomes. Both in the verbal and the visuospatial WM training, the model explained 15-20% of the variance in the initial WM levels and 2-8% in training gains, which means that there still is quite an amount of unexplained variance. The low percentages of explained variance illustrate that WM ability is a rather complex concept and training gains are affected by many different factors. Jaeggi and colleagues (2011) suggested that the observed differences in training gains in their study could be plausibly explained in terms of lack of interest during training or difficulty coping with the frustrations of the task becoming more challenging. This motivational element might be even more important within an ADHD population as some theories of ADHD postulate that motivational problems, to a great extent, are the core deficit of the disorder (Sonuga-Barke, 2003). Although it has been previously suggested that (intrinsic) motivation could influence the treatment outcomes of cognitive training research in children with ADHD (Shah et al., 2012; Van der Oord & Daley, 2015), this factor was not taken into account in the current study. Conclusion The current study has shown that, in line with previous studies (i.e. Melby- Lervåg & Hulme, 2013), training WM is quite complex and has its limitations for children with ADHD. Nonetheless, the study has provided a first glimpse of the importance of individual differences in training and transfer gains and encourages more in-depth future research. In terms of clinical implications, this knowledge can be used by clinicians in terms of what is to be expected 131

134 An investigation of training curves from training. For example, a declining learning curve towards the end of training doesn t necessarily mean that children have reached their limits, rather it is merely a typical development of WM training. In turn this knowledge could encourage the child and therapist to finish the training. Finally, WM training in the form presented in current study might not be qualitatively and quantitatively sufficient to overcome the deficits within this ADHD population. For example, it has been suggested that there should be a better alignment between ADHD deficiencies and targeted executive functions within interventions (Rapport et al., 2013; Orban et al., 2014) or training tasks should be made more ecologically valid by using tasks that resemble the complexity of problematic situations in daily-life (Dovis, 2014). 132

135 Chapter 5 Appendix Correlations between Verbal WM Levels of the Odd Training Sessions, Predictors, and Outcomes Variable VWM T1-2. VWM T VWM T VWM T VWM T VWM T VWM T VWM T VWM T VWM T Age IQ Ext. beh LD verbal WM Reading Spelling Note. VWM = verbal working memory, LD = learning disability, WM = working memory. For learning disability, 0 = no learning disability, 1 = learning disability. Correlations are based on EM imputed data. Bold numbers indicate significant effects (p <.05). 133

136 An investigation of training curves Correlations between Visuospatial WM Levels of the Odd Training Sessions, Predictors, and Outcomes Variable VSWM T1-2. VSWM T VSWM T VSWM T VSWM T VSWM T VSWM T VSWM T VSWM T VSWM T Age IQ Ext. beh LD VSWM Math Note. VSWM = visuospatial working memory, LD = learning disability. For learning disability, 0 = no learning disability, 1 = learning disability. Correlations are based on EM imputed data. Bold numbers indicate significant effects (p <.05). 134

137 Chapter 5 Parameter Estimates of the Final Model of the Verbal WM Training Parameter Unstandardized SE Standardized Outcome means Verbal WM Word reading -9.45* Spelling Predictor means Age 12.15*** IQ 10.13*** Externalizing behavior 10.92*** Learning disability 0.23*** Latent factor means IS 2.44*** LS 0.35*** QS -0.05** Predicted means Verbal WM T1 2.44*** Verbal WM T3 2.75*** Verbal WM T5 2.96*** Verbal WM T7 3.09*** Verbal WM T9 3.12*** Verbal WM T *** Verbal WM T *** Verbal WM T ** Verbal WM T * Verbal WM T Outcome variances Verbal WM 27.23*** Word reading 31.00*** Spelling 94.59*** Predictor variances and covariances Age 2.10*** IQ 1.77*** Externalizing behavior 64.03*** Learning disability 0.17*** Age <-- --> IQ -0.73*** Age <-- --> Externalizing behavior -2.27* Age <-- -->à Learning disability 0.17** IQ <-- --> Externalizing behavior IQ <-- --> Learning disability Ext. behavior <-- --> Learning disability Latent factor residual variances and covariances IS 0.23*** LS 0.05*** QS IS <-- --> LS IS <-- --> QS LS <-- --> QS -0.01***

138 An investigation of training curves Direct effects on IS Age 0.15*** IQ 0.12** Externalizing behavior Learning disability on LS Age IQ Externalizing behavior Learning disability on QS Age IQ Externalizing behavior Learning disability 0.01* on VWM IS LS 15.32* QS on Word reading IS LS QS Age 0.86* Externalizing behavior 0.17** on Spelling IS LS QS Learning disability -6.77** Error variances E T1 0.18*** E T3 0.26*** E T5 0.47*** E T7 0.61*** E T9 0.44*** E T *** E T *** E T *** E T *** E T *** Note. IS = Intercept, LS = linear slope, QS = quadratic slope, WM = working memory. Standardized estimates for residual variances and error variances are proportions of unexplained variance. *p <.05. ** p <.01. *** p <

139 Chapter 5 Parameter Estimates of the Final Model of the Visuospatial WM Training Parameter Unstandardized SE Standardized Outcome means Visuospatial WM -4.90** Automated math Predictor means Age 12.17*** IQ 10.15*** Externalizing behavior 10.78*** Learning disability 0.23*** Latent factor means IS 3.52*** LS 0.43*** QS -0.05* Predicted means Verbal WM T1 3.52*** Verbal WM T3 3.91*** Verbal WM T5 4.20*** Verbal WM T7 4.40*** Verbal WM T9 4.51*** Verbal WM T *** Verbal WM T *** Verbal WM T *** Verbal WM T ** Verbal WM T * Outcome variances Visuospatial WM 4.45*** Automated math 50.12*** Predictor variances and covariances Age 2.09*** IQ 1.79*** Externalizing behavior 63.51*** Learning disability 0.18*** Age <-- --> IQ -0.68*** Age <-- --> Externalizing behavior -2.24* Age <-- --> Learning disability 0.18*** IQ <-- --> Externalizing behavior IQ <-- --> Learning disability Ext. behavior <-- --> Learning disability Latent factor residual variances and covariances IS 0.25*** LS 0.09*** QS

140 An investigation of training curves IS <-- --> LS IS <-- --> QS LS <-- --> QS -0.01*** Direct effects on IS Age 0.17*** IQ 0.13** Externalizing behavior Learning disability on LS Age IQ Externalizing behavior Learning disability on QS Age IQ Externalizing behavior Learning disability on VSWM IS 1.25* LS QS on Automated math IS LS QS Error variances E T1 0.30*** E T3 0.55*** E T5 0.68*** E T7 0.61*** E T9 0.61*** E T *** E T *** E T *** E T *** E T *** Note. IS = Intercept, LS = linear slope, QS = quadratic slope, WM = working memory. Standardized estimates for residual variances and error variances are proportions of unexplained variance.*p <.05. ** p <.01. *** p <

141 Chapter 6 Summary and general discussion

142 Summary and general discussion The last decade, the amount of non-pharmacological treatments for children with ADHD has increased tremendously, especially interventions aimed at improving executive functions (EFs) have found its way into clinical practice. It was thought that targeting these underlying neurological substrates and core cognitive deficits assumed to mediate ADHD causal pathways could potentially lead to greater transfer and generalization to functioning in everyday life. This shift towards the development of interventions that directly target EFs is even more understandable when one considers the impact of these neurocognitive deficits on functioning in everyday life such as academic performance (e.g. Gathercole, Pickering, Knight & Stegmann, 2004) and the inability of worldwide recommended multimodal approaches (e.g. combination of pharmacological and psychological treatments) to improve those ecologically valid outcome measures in children with ADHD (Jensen et al., 2007; Van der Oord, Prins, Oosterlaan & Emmelkamp, 2008a). At the start of this thesis in 2011 there were several EF targeted intervention studies that showed promising findings on both near (i.e. improvement in untrained tasks similar to the domain of the trained task) and far transfer measures (i.e. improvements in domains other than the trained process). However, at that time only few studies also incorporated academic outcome measures which is remarkable given that this key area of functioning in everyday life is often disturbed in children with ADHD (Loe & Feldman, 2007). Simultaneously there was a growing demand for evidence-based implementable interventions in a school setting for children with EF related behavior and learning difficulties (such as children with ADHD) in regular educational settings in the Netherlands. Evidence-based and standardized interventions that could both support the teacher and the child with EF problems were scarce at that time. The current thesis was aimed at determining whether cognitive training would be effective for school-aged children with ADHD (aim 1) and whether transfer, both in terms of classroom behavior as well as academic performance, could be improved with an innovative classroom embedded approach (aim 2). Moreover, it was aimed at obtaining a more finer-grained knowledge of factors that might influence the efficacy of training such as underlying mechanisms, individual differences and training features (aim 3). This chapter 140

143 Chapter 6 summarizes the main findings of the different chapters throughout this thesis and ends with a general discussion and implications for future research and practice. Summary Chapter 2 describes the results of the randomized controlled trial which was aimed at replicating and extending previous Cogmed Working Memory Training (CWMT) studies in children with ADHD by investigating the short and long-term effects on neurocognitive, behavior and academic outcome measures. Children in the active control group received a new cognitive training called Paying Attention in Class (PAC) which contained a working memory and a compensatory executive function training. One hundred and five children with ADHD between the age of 8 and 12 years were randomly assigned to either CWMT or the PAC intervention. For both interventions, children received treatment from trained developmental psychologists during school hours outside the classroom. Results showed that children in both groups improved on measures of attention, working memory and inhibition directly after treatment. These results were supported by improvements found on parent rated executive functioning and parent and teacher rated ADHD related behavior. On the long term, children in both groups improved on measures of working memory, inhibition, planning, parent and teacher rated ADHD related behavior and teacher rated executive functioning. One superior effect of CWMT was found; children who followed CWMT performed better on a visual spatial working memory task. Children did not improve on measures of academic performance, behavior in class and quality of life. Conclusively, both on the short and long-term children improved on broad neurocognitive measures and parent and teachers executive functioning and ADHD related behavior ratings. The aim of the study presented in chapter 3 was to explore whether clinical and initial cognitive abilities predicted or moderated the neurocognitive and academic performance outcome measures of aforementioned RCT. Investigated predictor and moderator variables were use of medication, 141

144 Summary and general discussion comorbidity, subtype of ADHD, initial verbal working memory skills and initial visual spatial working memory skills. Results showed that use of medication and initial verbal - and visual spatial working memory skills predicted and moderated near transfer measures. Irrespective of type of training, children with initial below average or average working memory (either verbal or visual spatial) skills benefitted most over time in terms of performance on an attention - and visual spatial working memory task. CWMT was more beneficial in terms of a visual spatial working memory task for children who used medication during training and children with initial below average or average verbal working memory skills. Subtype of ADHD and comorbidity predicted and moderated far transfer measures. In terms of parent and teacher rated behavioral regulations problems, children with the ADHD- Inattentive subtype temporarily benefitted most from cognitive training in general. Additionally, children with the ADHD-Inattentive subtype in the CWMT group also benefitted most on the long term regarding teacher rated behavioral regulation - and metacognition problems. Finally, in terms of word reading accuracy, children with the ADHD-Combined subtype and children without comorbid disorders (i.e. learning disabilities or other behavioral disorders) benefitted most on the short term from cognitive training in general. In order to extend our understanding of the individual differences in both near and far transfer measures of the new PAC intervention, an additional group of 116 children with ADHD received this intervention. In chapter 4 it was investigated which demographical, clinical and baseline neurocognitive characteristics predicted individual treatment response six months after treatment, based on a clinical significant improvement in working memory. Results showed that initial sustained attention skills, initial verbal working memory skills and teacher rated metacognition problems at baseline predicted individual treatment response. Children with lower sustained attention and verbal working memory skills and less teacher rated metacognition problems were more likely to be non-responders to treatment. Subsequent analyses revealed that non-responders only improved on near transfer measures (verbal and visual spatial working memory) and parent rated questionnaires. In contrast, both the partial responders and responders improved on most far 142

145 Chapter 6 transfer outcome measures with most profound effects for the responders. Responders benefitted significantly more in terms of visual spatial working memory, teacher rated metacognition problems, direct learning conditions (i.e. concentration, motivation, work rate, task orientation) and the parent rated quality of life scales psychological well-being and school environment. These results imply that a clinical (significant) improvement in working memory was an important prerequisite to obtain improvements in far transfer measures as well. A small group of non-responders that additionally followed CWMT only improved on a measure of visual spatial working memory. In chapter 5 we focused on individual differences in performance gains during the working memory training of the PAC intervention and investigated how individual differences in learning curves influenced transfer measures directly after training and which variables could predict those learning curves. Based on the sample described in chapter 4, data of the trained verbal and visuospatial working memory task was analysed with a latent growth curve model (LGCM). Results showed that for both trained tasks, there were individual differences at the beginning of training and individual differences in children s linear growth trajectories. The individual differences at the start of training were predicted by age and intelligence, however the individual differences in learning curves were not predicted by any of the baseline variables. Children with larger gains on the trained verbal working memory task showed larger gains on a near transfer verbal working memory measure. The linear growth trajectories of the trained visuospatial working memory task did not affect the visuospatial outcome measure. Finally, the academic performance outcome measures were not affected by the linear growth trajectories of the trained verbal or visuospatial task. 143

146 Summary and general discussion General discussion Near and far transfer effects of cognitive training The first aim of this thesis was to establish whether cognitive training (i.e. CWMT and PAC), implemented at school, would be effective for school-aged children with ADHD. The most important findings from our RCT were that, irrespective of the type of training, children improved both on the short and long term on several neurocognitive outcome measures. Previous treatment effects of CWMT on verbal working memory (Holmes et al., 2010; Hovik et al., 2013) and attention (Egeland et al., 2013; Klingberg et al., 2002; Klingberg et al., 2005) were not found, only a treatment effect on a visual spatial working memory task could be replicated. In terms of behavioral outcome measures, parents and teachers also reported improvements in executive functioning and ADHD related behavior for both groups indicating transfer to functioning in everyday life. Especially the improvements in teacher rated questionnaires are promising as evidence from previous studies in this area has been scarce. However again, treatment effects of CWMT on parent ratings of ADHD related behavior (Beck et al., 2010; Klingberg et al., 2005) and executive functioning (Beck et al., 2010) could not be replicated. The overall null findings in academic performance measures contrasted with some previous effect studies of CWMT (Egeland et al., 2013; Green et al., 2012) however these results are more in line with recent meta-analyses (Rapport et al., 2013; Cortese et al., 2015; Orban et al., 2015). So what can we conclude from these results? Is cognitive training effective for children with ADHD and should it be added to the list of effective nonpharmacological interventions? Why couldn t we replicate the treatment effects of CWMT that were found in previous studies? And why are our results only partly in line with these recent meta-analyses? How can two interventions that are different in their nature and treatment features lead to almost similar treatment outcomes? To answer these important questions, we need a broader picture of factors that possibly contribute to the effects of cognitive training. Therefore, we have to dig deeper and focus on factors such as underlying mechanisms, individual differences and training features. 144

147 Chapter 6 Figure 1, which is based on the review article of Von Bastian and Oberauer (2013), depicts an oversight of how these factors are integrated with the findings of current thesis and thereby providing a basis for this general discussion. Methodological confounding factors Although previous studies (e.g. Green, Strobach & Schubert, 2014; Morrison & Chein, 2011; Shipstead et al., 2010; 2012) have extensively elaborated on the methodological issues around cognitive training, several of these issues (right side of Figure 1) are readdressed below as they are particularly important for the interpretation of the results presented in current thesis. In contrast to previous effect studies of CWMT, our RCT contained an active control group (PAC) whose experience was closely matched to the training group in terms of effort, active training time and performance related feedback. This overcomes the possibility that the trained and control group approach the post assessment differently in terms of motivation (Shipstead, Hicks et al., 2012) and also ensures that the expectations of parents and teachers is equal for both conditions. It should be mentioned though that our RCT did not contain a third randomized no treatment control group. Therefore confounders such as test-retest effects, passage of time, expectancy and effort effects cannot be ruled out. Parent and teachers were aware that children received an active treatment and teachers were actively involved in the PAC intervention, which could have inflated the parent and teachers ratings. Meta-analyses have shown that effects of ADHD ratings after cognitive interventions drop to non-significant if outcomes of probably blinded raters are considered (Cortese et al., 2015; Sonuga-Barke et al., 2013). Choosing and developing control groups remains challenging for future trials as ethical constraints make it difficult to implement no treatment groups and there still is no consensus about how a control group should be designed (Von Bastian & Oberauer, 2013). Nonetheless, it is advisable that, next to an active control group, future studies include a third no contact randomized control group with well blinded measures of behavioral outcomes. 145

148 Summary and general discussion Figure 1. Factors possibly influencing cognitive training outcomes Intervention specific features *intensity & duration *paradigm *algorithm Progress trained task(s) Verbal working memory Individual differences *age *intelligence *initial cognitive ability *comorbidity *subtype *medication intercept only Specific working mechanisms Enhanced capacity/enhanced efficiency Assessment during treatment: *neuro imaging studies * strategy questionnaires *personality *motivation *biological factors *beliefs in malleability Non-specific working mechanisms *treatment adherence *skills therapist *support teacher and parent = = Treatment outcome Effects cognitive training Methodological confounders Test-resttest effect, passage of time, expectancy effect *assessment far transfer *well-blinded measures *randomized third control group factors that (possibly) contributed to findings of current thesis factors that should be investigated in future research 146

149 Chapter 6 An additional methodological factor that should be considered in light of current findings concerns the way in how far transfer measures were assessed. For cognitive measures, a general recommendation that arises from the literature refers to the use of multiple tasks per cognitive domain (e.g. Morrison & Chein, 2011; Shipstead et al., 2010). Using only one single task merely reflects transfer of task-specific rather than task-general improvements. Despite the fact that we made a well-balanced decision in terms of including cognitive measures, some cognitive domains (visual spatial working memory, planning and inhibition) were represented by one single task and should therefore be interpreted with caution. For example, the replicated treatment effect of CWMT on visual spatial working memory that was found in chapter 2 was based on one single task that shared similar features with many of the trained tasks within CWMT. Findings in terms of academic outcome measures also seem to be greatly dependent on the way how they are assessed. An important strength of our study was that, in contrast to many previous studies, we included long term assessments of academic performance measures. These longterm assessments are necessary as a child will need to exploit his or her improved working memory capacity and this will only be visible after a lengthy period of time (Gathercole, 2014). Nonetheless, no improvements were found on any of the academic performance measures. Many studies, including ours, contained standardized ability tests that tap into cumulative achievements which makes them strongly dependent on prior learning and relatively insensitive to recent changes in learning capacities (Gathercole, 2014). Actual school measures such as national academic tests or grades could possibly offer a better assessment of academic performance during a certain period of time. Another factor that could explain why we didn t find any improvement in academic performance measures is the fact that the correlations between the academic outcome and working memory measures in current study were quite low (chapter 5). The academic outcome measures that were used in current study were quite basic tasks which required less working memory capacity than more complex tasks such as reading comprehension or mathematic problem solving (Dehn, 2008). 147

150 Summary and general discussion Therefore, academic outcomes measures that predominantly depend on working memory capacity should be included in future studies. Conclusively, these methodological considerations bring us a bit closer to understanding the inconsistent findings in transfer effects throughout the cognitive training literature and provide some starting points for future research designs. However, to address the remaining questions regarding the putative underlying mechanisms and individual differences, a further examination of factors that potentially contributed to the observed near and far transfer effects is necessary. Specific working mechanisms The middle part of Figure 1, which refers to the potential underlying working mechanisms, will be discussed hereafter. The putative mechanism behind process-based interventions such as CWMT is based on the assumption that extensive training of a specific cognitive skill strengthens the common and overlapping neural EF network which in turn leads to improvements in untrained tasks or activities that rely on the same neural network (Klingberg, 2010). This would imply that the more the executive function improves during training, the larger the transfer effects will be. So far, there is no direct evidence for this mechanism in ADHD samples (Van der Oord & Daley, 2015) although some studies provide indirect evidence opposing this hypothesis as they found far transfer effects in the absence of near transfer effects (Chacko, Bedard et al., 2014; Dovis et al., 2015; Egeland et al., 2013; Van Dongen-Boomsma et al., 2014). Results of chapter 4, although indirectly, provide some support for this aforementioned hypothesis as it was found that children with the largest gains in working memory skills (responders) after following the PAC intervention showed a broader and stronger pattern of far transfer improvements. In chapter 5 we further elaborated this hypothesis by investigating how improvements in working memory capacity during the PAC training influenced near and far transfer measures. Results showed that children with larger verbal working memory training gains (i.e., steeper training curves) had larger benefits on the untrained verbal working memory composite score, indicating near transfer. However, these training 148

151 Chapter 6 gains did not affect the academic outcome measures and no effects of visuospatial training gains on transfer measures were found. Additionally, both in the verbal and the visuospatial WM training, the model explained 15-20% of the variance in the initial WM levels and 2-8% in training gains. These results provide limited support for this putative neural mechanism of working memory training and also illustrates that training gains are affected by many different factors, hence the large proportion of unexplained variance. As this hypothesis stems from the neural correlates of executive functioning, evidence from neuro-imaging studies in ADHD samples is indispensable and should be incorporated in future trial designs. Another plausible specific mechanism that could account for transfer is the increase of working memory efficiency (Von Bastian & Oberauer, 2013). For example, the acquisition of knowledge and skills (e.g. strategy use) during training could lead to more efficient use of the available working memory capacity. This account has been supported by a recent study of Dunning and Holmes (2014) which showed that training related improvements in working memory were accompanied by implicit and spontaneous changes in use of strategies in healthy adults. Although there are some indications that spontaneous increased use of strategies during process-based working memory training occurs in children with ADHD as well (Holmes et al., 2010), this hasn t been systematically investigated so far. As for the PAC intervention, children practiced with several strategies throughout the training so there is a possibility that this led to a more efficient use of the available working memory capacity. However, one would need questionnaires that could map the strategy use during training to provide evidence for this mechanism. As long as direct evidence of these specific mechanisms of increased working memory capacity and efficiency is lacking, it might be more rewarding considering the potential contribution of non-specific treatment factors to near and far transfer findings. Non-specific working mechanisms Non-specific factors such as therapeutic alliance, the therapist s competence and adherence to treatment protocol refer to dimensions that are shared 149

152 Summary and general discussion by most therapies and have shown to contribute to treatment outcome in psychotherapy (Chatoor & Krupnick, 2001). Especially therapeutic alliance, i.e. the quality of the relationship between the patient and therapist, has been shown to be a reliable predictor of positive treatment outcome in psychotherapy (Ardito & Rabelinno, 2011). So far this has received little attention in cognitive training literature. However, it is likely that this factor made a contribution to the findings in current thesis. CWMT and PAC differ in their nature and treatment features, nonetheless they led to quite similar outcomes. The common factor of both intervention groups was that they received equal amounts of interaction time with a therapist in which children learned to cope with frustrations (due to increasingly demanding tasks) and were encouraged to proceed even if they failed. This may imply that it does not matter what you do (either process-based versus more strategy based training) instead it has been suggested that how an activity is done is more important and that the personal characteristics of those leading a program can have major effects on how beneficial a program is (Diamond & Ling, 2015). Although the treatment protocols were standardized and the therapists received similar training beforehand it is plausible that, given the large amount of therapists that were necessary to conduct the studies in current thesis, treatment alliance varied which in turn could have influenced the efficacy of training. Therefore, we suggest that future trials should monitor these non-specific therapist factors and incorporate these findings in treatment outcome analyses. The Client Direct Outcome Information (CDOI) method of Duncan, Miller and Sparks (2004) is a good example of a model that can used in future research. Another non-specific treatment factor that could have contributed to the observed effects is the use of incentives during training. Next to models that view executive dysfunction as a causal model for ADHD, there are also models emphasize the sub-optimal reward systems as a second and cooccurring causality for ADHD (Sonuga-Barke, 2003). Dovis (2014) has shown that incentives significantly improve working memory performance of children with ADHD and that the intensity of the incentive determines the persistence of performance over time. Feedback-only was not enough for these children 150

153 Chapter 6 to reach optimal performance. Within the studies presented throughout this thesis, children received daily small rewards at the end of each session (e.g. stickers or playtime) and a small present on a weekly basis. In order to investigate to what extend incentives during training could contribute to the effects of cognitive training, future studies should for example vary the amount of incentives between treatment conditions. Altogether, these specific and non-specific working mechanisms seem not exclusive of another rather it is likely that they interact with each other in determining the treatment outcomes. So far throughout this discussion we have focused on the potential working mechanisms and methodological factors that could contribute to the effects of cognitive training, now it is time to focus on the remaining part of Figure 1 and discuss the contribution of intervention specific features and individual differences. Intervention specific features Although intervention specific features were not directly examined in the current thesis, they will be briefly discussed below given the alleged impact of these factors on cognitive training outcomes. Throughout the cognitive training literature, training features vary greatly in terms of paradigms (e.g. single versus multiple targeted skills), intensity (frequency) and duration (dose) of training sessions and adjustment of task difficulty (Von Bastian & Oberauer, 2013). Regarding this first factor, the training paradigm, there still is no consensus of what works best for children with ADHD. Although it has been suggested that training just one single cognitive domain might not be sufficient to reach broad transfer (Moreau & Conway, 2014; Van Dongen- Boomsma et al., 2014), the effect of multiple executive function training also seems elusive so far (Dovis et al., 2015). It has been suggested though that the training paradigms are not in alignment with the neuropsychological deficits that are most impaired in children with ADHD (e.g. Gibson et al., 2011) and therefore future intervention designs should adequately target these broader range of neuropsychological deficits (Cortese et al., 2015; Orban et al., 2015; Rapport et al., 2013). Intensivity and adaptivity have been assumed to be critical elements of working memory training (Klingberg, 2010). However, more recent research disputes the importance of these factors. For example, a 151

154 Summary and general discussion study of Von Bastian and Eschen (2015) found that there were no differences in training or transfer gains between training procedures that differed in task difficulty. In terms of intensity, a recent study of Mawjee and colleagues (2014) in adults with ADHD showed that a shortened-length version of CWMT (15 minutes) led to almost similar results as the standard-length version (45 minutes) even when motivation, engagement and expectancy of change were controlled for. Results from chapter 3 support this notion showing that quantitatively different exposure to working memory training (CWMT contained 90 trials while PAC contained 30 trials) led to quite similar results of transfer. Given that ADHD is a persistent developmental disorder, 25 sessions of training might not be enough to obtain its desired effects. And although CWMT offers an extended protocol (booster sessions), the potential benefit of these booster sessions has not been investigated so far. In summary, the extent to which intervention specific features could contribute to cognitive training gains seems limited and needs more in-depth research. Alternatively, it has been suggested that transfer is most likely to be optimized if the activity takes place in a more ecologically valid setting (Moreau & Conway, 2014; Gathercole, 2014) and besides training EFs directly also addresses the emotional, social and physical needs (Diamond & Ling, 2015). Influence of individual differences Last but certainly not least, the last part of this discussion will focus on the individual differences in training gains, an area that until now was relatively unexplored within the field of cognitive training research for ADHD. In line with suggestions from others (Chacko et al., 2013; Shinaver et al., 2014), results from chapter 3 revealed that children with initial lower working memory skills and children who used medication benefitted more from CWMT. These effects did not generalize beyond the visual spatial span outcome measure, which is considered a trained task of CWMT. Interestingly, clinical variables were found to mainly influence far transfer outcome measures. Irrespective of type of training, children without a comorbid learning disability (LD) benefitted most in terms of word reading accuracy. Word reading accuracy 152

155 Chapter 6 was also influenced by the subtype of ADHD, on the short term children with the ADHD combined (ADHD-C) subtype benefitted most from cognitive training in general. It should be noted though that both for the effect of comorbidity and subtype of ADHD, there were no differences between the subgroups on the long term which emphasizes the importance of including long term assessments of academic outcome measures. These findings might also explain why others (Egeland et al., 2013) found treatment effects on academic outcome measures as only children with the ADHD-C subtype were included with no mentioning of comorbid LDs. The fact that both children with a learning disability and children with the ADHD Inattentive (ADHD-I) subtype could not benefit from training in terms of these academic outcome measures is not surprising. Just as children with a LD (Dehn, 2008), children with the ADHD-I subtype are more likely to suffer from poor working memory skills (Diamond, 2005). Additionally, a comorbid LD is more common in children with ADHD-I (Diamond, 2005) implying that there might have been in overlap in these children. Contrary to the academic outcome measures, children with the ADHD-I subtype benefitted most on the short term from training in terms of parent and teacher rated behavioral regulation problems. Additionally, within the group that followed CWMT, these children also benefitted most on the long term regarding teacher rated behavioral regulation and metacognition problems. This was a rather surprising though promising finding as studies that investigated the efficacy of CWMT in children with ADHD so far were not able to establish effects on teacher rated executive function behavior. The behavioral regulation scale of the BRIEF mainly contains hot aspects of executive functioning (more associated with ventral and medial prefrontal cortex, e.g. inhibition and emotion regulation) while the metacognition scale mainly contains cool aspects of executive functioning (more associated with the lateral prefrontal cortex, e.g. working memory and planning). We suspect that children with the ADHD-C benefitted less from cognitive training due to a more heterogeneous origin with both cool and hot executive function deficiencies. Children with the ADHD-C subtype are usually affected by both cool and hot executive function deficiencies in contrast to children with 153

156 Summary and general discussion ADHD-I how are mostly affected by cool executive function deficiencies. Cool executive functions such as working memory are more likely to be elicited by relatively abstract decontextualized problems (Zelazo & Müller, 2011, p. 586) and can be associated with attention problems according to Castellanos and colleagues (2006). Hot executive functions can be described as the emotional problem solving executive functions (Zelazo & Müller, 2011) and are reflected in hyperactive/impulsivity symptoms (Castellanos et al., 2006). Future studies with larger sample sizes of different subtypes and well blinded assessments of executive function behavior are necessary to further investigate this potential beneficial effect for the ADHD-I subtype. Regarding the PAC intervention, findings were somewhat contradicting for the initial cognitive abilities. Results from chapter 3 indicate that, just as for CWMT, children with initial lower verbal and visual spatial working memory skills in the PAC intervention group benefitted more in terms of an attention and visual spatial working memory task. This implicates a compensation effect which is usually found for process-based interventions, indicating that individuals with the lowest initial cognitive abilities probably benefit most because they have more room for improvement (Titz & Karbach, 2014; Karbach & Unger, 2014). The opposite was found in chapter 4, showing that children with higher initial attention and verbal working memory skills were more likely to benefit from training in terms of a working memory composite score. This implicates a magnification effect which has previously been observed for strategy based interventions indicating that individuals with high initial cognitive abilities might benefit most as more efficient cognitive resources make it easier to acquire and implement new strategies and abilities (Titz & Karbach, 2014; Karbach & Unger, 2014). Two factors should be taken into account when interpreting these contradicting results. First of all, the PAC intervention contained both a process-based and a strategy based training which makes it hard to disentangle the possibility of a compensation or magnification effect. Second, there was a methodological difference between the two studies in terms of how working memory was assessed as predictor (nominal versus continuous variable) and outcome measure (single construct versus composite score). Again, this highlights that one should always consider the way how transfer is 154

157 Chapter 6 assessed when interpreting the results of cognitive training gains. Next to initial attention and working memory skills, there were several other factors found to influence the effects of PAC. For example, significant lower IQ scores were observed for children who did not clinically improve on working memory skills (non-responders) in chapter 4. In chapter 5, both age and IQ were found to predict initial working memory levels of the trained tasks within PAC. However, they were not a significant predictor of the actual improvement. It remains unclear how these factors exactly influence training gains. It has been suggested that it is not so much chronological age that is important but rather neurodevelopmental age (Rutledge et al., 2012) or stage of cognitive development (Jolles & Crone, 2012), i.e. age and earlier experience. Finally, chapter 4 disclosed additional important findings regarding who might, or actually who might not, benefit from cognitive training. Transfer for the non-responders of the PAC intervention, children with initial lower attention and verbal working memory skills, was limited to working memory tasks and parent rated questionnaires. A small group who additionally followed CWMT only improved on a visual spatial working memory task indicating that this is a group of children who cannot profit from cognitive training in its current form and are in dire need of adjusted treatment protocols. Conclusively, results of current thesis have shown that several clinical and cognitive variables influence cognitive training transfer gains which warrants that future studies should shift towards a more individual approach of assessing training gains. Current findings might still be just the tip of the iceberg and individual differences in other variables such as prior treatment, motivation, personality, beliefs, biological factors deserve to be explored in future trials. A single-case experimental design (SCDE), with its repeated assessments during various phases of treatment, is a good example of a more individual approach for determining the therapeutically utility of cognitive training (e.g. Barlow, Nock & Hersen, 2009). 155

158 Summary and general discussion Implications and directions for future research and practice In terms of clinical implications, findings of current thesis show that cognitive interventions are not a quick fix solution of ADHD related problems. Rather it is an ambiguous and complex process that requires effort from the environment in which the effects do not follow a clear-cut path. Despite the fact that dropout rates were low, indicating that implementing cognitive interventions at school is feasible, the current thesis has shown that obtaining transfer to the classroom is not easy and that certain boundary conditions are crucial for optimizing the results of cognitive training. Coping with practical issues such as time scheduling of treatment sessions or a reasonable place to practice have been a major, but not insuperable, challenge in this study. To date, most cognitive interventions have been solely aimed at improving children s executive functions directly. However, the current study has taught us that it is possibly even more important to actively involve the entire environment of the child. Although teachers are generally motivated to support these children within the classroom, in practice it is much harder to burden teachers with this task. Given the implementation of the law Passend Onderwijs in the Netherlands (i.e. education that should fit), which makes schools obligated to provide adequate assistance for children who need extra care, the demand for similar interventions as PAC will continue to increase. An additional law that was recently implemented in the Netherlands, the transformation and transition of youth health care services to the local governments, heavily relies on the social network and requires an intensive collaboration from health care professionals, parents and teachers. Therefore, integrating the roles of all those involved should be the main target of future intervention designs, for example by encouraging parents and teachers to address the problems of the child with the same terminology both at home and at school. Finally, considering the financial investment of individual treatment and the large number of children that struggle with executive function deficiencies in the classroom, the feasibility of a group intervention should also be considered for future research. 156

159 Chapter 6 The main important message for health care professionals is that cognitive training should be viewed as an adjunctive treatment of current guidelines, bearing in mind that it is not a one size fits all treatment. When the main aim is to improve executive function behavior at home or in school, clinicians should hold in mind that children with the ADHD-I subtype could profit most from training. Additionally, obtaining improvements in academic performance measures constitute a greater challenge for children with a comorbid learning disability, ADHD-I subtype or lower initial attentional and working memory skills. Clinicians should make a well balanced decision in terms of whether or which cognitive training might be suitable for a child. Next to evaluating the cognitive and functional impairments of children beforehand, it is also important that professionals discuss the expectations of parents and teachers and assess whether the environment can provide an optimal climate for training. The degree of commitment from parents, children, teachers and clinicians are all important to achieve the best possible outcome. Other factors that should be considered include timing within the academic year (e.g. not around curriculum based assessments periods) or unstable family conditions (e.g. divorcing parents). Conclusion Right from the start of this thesis, it was obvious that the study fulfilled the demand for implementable interventions in a school setting for children with EF related behavior and learning difficulties. Many professionals, teachers, parents and children cooperated enthusiastically and were positive about the applicability of PAC providing both children and teachers practical tools to work with within the classroom. This thesis started out with a fairly simple question namely: is cognitive training effective for children with ADHD? However, answering this question appeared to be anything but simple. When considering all factors discussed above, future research still faces many challenges before cognitive training meets the criteria as an evidence-based treatment for ADHD and can be added to the list of effective non-pharmacological interventions. Future research will, among other things, face challenges in terms of including adequate measures of transfer and adequate control groups. Nonetheless, we have made a good 157

160 Summary and general discussion start by showing that cognitive training, when implemented within the school setting, can effectively improve a fair amount of outcome measures. More importantly, the current thesis has shown that individual differences are crucial both in terms of clinical decision making and in determining the efficacy of cognitive training. This also encourages future researchers to assess training and transfer gains on a more individual level. 158

161 Nederlandse samenvatting

162 Nederlandse samenvatting De laatste jaren is het aanbod van niet farmacologische behandelingen voor ADHD enorm toegenomen. Voornamelijk interventies die gericht zijn op het verbeteren van executieve functies (de regelfuncties van het brein), veelal in de vorm van een cognitieve training, worden in toenemende mate geïmplementeerd in de klinische praktijk. Een cognitieve training kan omschreven worden als het proces van het verbeteren van cognitief functioneren door middel van oefeningen en/of doelbewuste instructies (Jolles & Crone, 2012). De veronderstelling is dat cognitieve interventies niet zozeer het directe gedrag beïnvloeden, maar juist de onderliggende mechanismen aanpakken die het gedrag veroorzaken (o.a. Sonuga- Barke, Brandeis, Holtmann, & Cortese, 2014). Dit zou potentieel ook leiden tot betere transfer en generalisatie van de effecten naar het functioneren in het alledaagse leven. De toename in dit type interventies is begrijpelijk gezien het feit dat executieve functies een belangrijke rol spelen bij het functioneren in het alledaagse leven en het feit dat een wereldwijd aanbevolen multimodale behandelaanpak (meestal een combinatie van medicatiegebruik en gedragstherapie) veelal niet leid tot langdurige verbeteringen (Jensen e.a, 2007) en verbeteringen in ecologisch valide uitkomstmaten zoals het schoolse functioneren (Raggi & Chronis, 2006; Van der Oord, Prins, Oosterlaan & Emmelkamp, 2008). Daarbij heeft ook bezorgdheid over medicatiegebruik vanwege gerapporteerde ernstige bijwerkingen (Graham & Coghill, 2008) en de onbekende lange termijn effecten (Berger, Dor, Nevo, Goldzweig, 2008) geleid tot een toename in de vraag en ontwikkeling van alternatieve niet farmacologische behandelingen voor kinderen met ADHD. Een cognitieve training kan op een aantal manieren aangeboden worden. Allereerst is het mogelijk om cognitieve vaardigheden te trainen door deze expliciet, intensief en adaptief te oefenen (zogenaamde core-trainingen; Morrison & Chein, 2011). De meesten van dit type interventies richten zich op het verbeteren van kern executieve functies zoals werkgeheugen en inhibitie. Een tweede vorm van cognitieve training vindt plaats door cognitieve tekorten juist te compenseren, waarbij de nadruk wordt gelegd 160

163 Nederlandse samenvatting op de sterke cognitieve vaardigheden van het individu. De zwakke vaardigheden worden hierbij omzeild en hebben daardoor minder impact op het functioneren (Dehn, 2008). De meeste compenserende interventies bevatten een strategie training, waarbij er diverse strategieën worden aangeleerd. Het compenseren kan ook plaatsvinden door het aanpassen van de leeromgeving, bijvoorbeeld met hulpmiddelen, of door de leerkracht concrete aanwijzingen te geven (Holmes, Gathercole & Dunning, 2010). De Cogmed werkgeheugen training, een voorbeeld van het eerste type (core) training, is één van de meest onderzochte en geïmplementeerde cognitieve trainingen in de klinische praktijk. Gedurende 25 individuele sessies wordt een kind blootgesteld aan verscheidene visueel spatiale en verbale werkgeheugen oefeningen die aangeboden worden via een computer. De moeilijkheidsgraad van de oefeningen past zich steeds aan op de prestaties van het kind. Bij aanvang van dit proefschrift in 2011 waren er verscheidene studies die veelbelovende effecten van deze training lieten zien op zowel getrainde domeinen (ook wel near transfer genoemd) als niet getrainde domeinen (ook wel far transfer genoemd). Er waren echter maar weinig studies die ook de effecten van de training onderzochten op het gebied van schools functioneren en er bleken ook enkele methodologische hiaten te zijn waardoor replicatie en uitbreiding van eerdere studies noodzakelijk was. Daarnaast was er ook vanuit de klinische praktijk een toenemende vraag voor implementeerbare interventies in een schoolse context voor kinderen met cognitieve problemen. Gestandaardiseerde en wetenschappelijk onderbouwde interventies die zowel de leerkracht als de leerling met cognitieve problemen kon ondersteunen waren echter bij aanvang van dit proefschrift niet of nauwelijks beschikbaar. Dit leidde tot de ontwikkeling van de Beter Bij de Les training, een gecombineerde strategieen vaardigheidstraining voor kinderen met zwakke executieve functies. De Beter Bij de Les training bestaat eveneens uit 25 individuele sessies waarbij op school, weliswaar buiten de klas, wordt getraind. De training bestaat uit een drietal kernelementen. Ten eerste krijgt het kind psychoeducatie aangeboden via een luisterboek waarbij het op een speelse en 161

164 Nederlandse samenvatting toegankelijke manier leert welke vaardigheden belangrijk zijn bij het goed kunnen uitvoeren van een taak, namelijk: gerichte aandacht, planning en initiatie, verdeelde aandacht en werkgeheugen, doel en taakgericht gedrag en metacognitie. Het kind leert daarbij in welke situaties het voor hem/haar moeilijk kan zijn om informatie te onthouden en krijgt daarbij strategieën aangereikt. Het tweede belangrijke element van de training betreft een drietal werkgeheugenoefeningen (visuele, verbale en gecombineerde oefening) die bij iedere sessie worden aangeboden. De moeilijkheidsgraad van de oefeningen wordt steeds aangepast op de prestaties van het kind. Tenslotte staat de generalisatie naar de klas centraal in deze training. Tijdens de sessie oefent het kind aan de hand van de psycho-educatie met schoolse taken en krijgt het daarna ook een ondersteunde kaart mee voor in de klas. Daarnaast ontvangt de leerkracht ook psycho-educatie gericht op executive functies in de klas en worden zij ook actief betrokken tijdens de training. Het huidige proefschrift was gericht op het bepalen van de effectiviteit van bovengenoemde tweetal cognitieve trainingen voor schoolgaande kinderen met ADHD. De nadruk lag daarbij op de vraag of de effecten van de training ook zouden generaliseren naar schoolse vaardigheden, zowel op het gebied van gedrag in de klas als leerprestaties. Tenslotte was het doel van dit proefschrift om ook meer grip te krijgen op de factoren die de effectiviteit van een cognitieve training mogelijk kunnen beïnvloeden zoals de werkingsmechanismen, individuele verschillen en specifieke kenmerken van een training. Generalisatie van effecten In hoofdstuk 2 worden de resultaten gepresenteerd van het gerandomiseerde onderzoek wat tot doel had om eerdere studies van de Cogmed werkgeheugen training bij kinderen met ADHD te repliceren en tevens uit te breiden. Zowel de korte als de lange termijn (6 maanden) effecten op neurocognitief functioneren, leerprestaties, gedrag in de klas, gedragsproblemen en kwaliteit van leven werden onderzocht. De kinderen in de controle groep ontvingen de actieve Beter Bij de Les 162

165 Nederlandse samenvatting training. In totaal werden 105 kinderen met een ADHD diagnose tussen de 8 en 12 jaar op basis van toeval toebedeeld aan de Cogmed groep of de Beter Bij de Les groep. Voor beide interventies werden de kinderen onder schooltijd, buiten de klas, individueel getraind door een orthopedagoog of kinderpsycholoog. De resultaten lieten zien dat kinderen in beide groepen vooruit gingen op aandachts-, werkgeheugen- en inhibitie taken direct na de training. Ouders en leerkrachten rapporteerden een vooruitgang in ADHD gerelateerd gedrag en ouders rapporteerden ook vooruitgang op een gedragsvragenlijst voor executief functioneren. Op de lange termijn waren er ook brede verbeteringen zichtbaar voor de kinderen in beide groepen, zowel voor de neurocognitieve maten als voor ouder en leerkracht rapportages van ADHD gerelateerd gedrag en executief functioneren. Voor beide interventies werden er geen verbeteringen gevonden voor de leerprestaties en vragenlijsten voor leervoorwaarden en kwaliteit van leven. Tenslotte werd er één superieur effect gevonden voor de kinderen die de Cogmed werkgeheugen training hadden gevolgd, zij presteerden beter op een visueel werkgeheugen taak. Dit superieure effect op visueel werkgeheugen werd ook door eerdere studies gevonden (Klingberg e.a., 2002; 2005; Gray e.a., 2012; Hovik e.a., 2013), mogelijk kan dit verklaard worden door het feit dat de meeste getrainde taken binnen de Cogmed training een beroep doen op het visueel werkgeheugen. De superieure effecten van Cogmed die door eerdere studies aangetoond werden op het gebied van aandacht (Klingberg e.a., 2002; 2005; Egeland e.a., 2013), verbaal werkgeheugen (Holmes e.a., 2010; Hovik e.a., 2013), academische maten (Green e.a., 2012; Egeland e.a., 2013), ouderrapportages van ADHD gedrag (Klingberg e.a., 2005; Beck e.a., 2010) en executief functioneren (Beck e.a., 2010) konden niet gerepliceerd worden. Samenvattend kan er geconcludeerd worden dat kinderen, ongeacht de interventie groep, zowel op de korte als de lange termijn vooruitgaan op meerdere neurocognitieve taken en dat ouders en leerkrachten ook verbeteringen waarnemen op het gebied van executief functioneren en ADHD gerelateerd gedrag. Echter moet er hier wel rekening gehouden worden met het feit dat er geen derde gerandomiseerde controle groep was die geen behandeling ontving. Daarnaast wisten ouders en leerkrachten dat kinderen in beide groepen 163

166 Nederlandse samenvatting een actieve behandeling ontvingen. Hierdoor kunnen potentiele storende factoren zoals test-hertest effecten of verwachtingseffecten van ouders en leerkrachten niet uitgesloten worden. Het feit dat deze twee inhoudelijke verschillende behandelingen tot gelijke resultaten leidden, suggereert dat niet specifieke behandelfactoren, zoals bijvoorbeeld positieve bekrachtiging en beloning, mogelijk een belangrijke rol spelen in het tot stand komen van de effecten. Predictoren en moderatoren van het behandeleffect De studie die gepresenteerd wordt in hoofdstuk 3 had als doel om te exploreren of sommige groepen kinderen mogelijk meer zouden kunnen profiteren van behandeling over het algemeen (predictoren) of meer profijt zouden kunnen hebben van de ene behandeling boven de andere (moderatoren). Er werd onderzocht of medicijngebruik (hoofzakelijk psychostimulantia), subtype van ADHD, comorbide stoornissen (leerstoornis of oppositionele gedragsstoornis) en initiële verbale- en visuele werkgeheugenvaardigheden invloed hadden op de neurocognitieve uitkomstmaten en leerprestaties uit bovengenoemd gerandomiseerd onderzoek. De resultaten lieten zien dat medicijngebruik en initiële verbale- en visuele werkgeheugenvaardigheden getrainde domeinen beïnvloedde. Ongeacht het type training (Cogmed of Beter Bij de Les), profiteerden kinderen met initiële beneden gemiddelde of gemiddelde verbale- en visuele werkgeheugenvaardigheden over de tijd heen meer op het gebied van aandacht en visueel werkgeheugen. Daarnaast bleek dat kinderen die medicijnen gebruikten tijdens de training en kinderen met initiële beneden gemiddelde of gemiddelde verbale werkgeheugenvaardigheden meer profijt hadden op het gebied van visueel werkgeheugen als ze de Cogmed werkgeheugen training hadden gevolgd. Subtype van ADHD en comorbide stoornissen beïnvloedden meerdere niet getrainde domeinen. Kinderen met het inattentieve subtype profiteerden tijdelijk meer van behandeling over het algemeen op het gebied van ouder en leerkracht gerapporteerde gedragsregulatie problemen. Bovendien profiteerden deze kinderen ook het meeste op de lange termijn op het gebied van leerkracht gerapporteerde gedragsregulatie en metacognitieve problemen als ze de Cogmed werkgeheugen training hadden gevolgd. Tenslotte werd 164

167 Nederlandse samenvatting er aangetoond dat, ongeacht het type behandeling, kinderen met het gecombineerde subtype en kinderen zonder comorbide stoornissen op de korte termijn nauwkeuriger woorden gingen lezen in vergelijking met kinderen met het inattentieve subtype en kinderen met comorbide stoornissen. Om onze kennis uit te breiden over de individuele verschillen van de Beter Bij de Les training in zowel de getrainde als niet getrainde domeinen werd er een tweede deelstudie uitgevoerd waarbij nog eens 116 kinderen met ADHD de training volgden. Op basis van de complete sample kinderen die Beter Bij Les hadden gevolgd (RCT en deelstudie 2 samen N=150), wordt in hoofdstuk 4 onderzocht welke demografische gegevens, klinische variabelen en initiële cognitieve vaardigheden de individuele behandelrespons na 6 maanden kon voorspellen. Deze behandelrespons werd gedefinieerd op basis van een klinisch significante vooruitgang in werkgeheugen. Er werden 32 nonresponders, 65 gedeeltelijke responders en 53 responders geïdentificeerd. Vervolgens werden de effecten van de training op neurocognitief functioneren, leerprestaties, gedrag in de klas, gedragsproblemen en kwaliteit van leven onderzocht voor de verschillende responsgroepen. Nonresponders kregen tevens de Cogmed werkgeheugen training aangeboden. De resultaten lieten zien dat initiële volgehouden aandacht, verbale werkgeheugen vaardigheden en leerkracht gerapporteerde metacognitie problemen de individuele behandelrespons voorspelde. Kinderen met zwakkere volgehouden aandacht en verbale werkgeheugen vaardigheden en minder leerkracht gerapporteerde metacognitieve problemen hadden meer kans om non-responder te zijn. Verder toonden de non-responders alleen vooruitgang op taken uit getrainde domeinen (verbaal en visueel werkgeheugen) en ouder rapportages van executief functioneren en gedragsproblemen. Daarentegen verbeterde de gedeeltelijke responders en de responders op de meeste niet getrainde domeinen. De responders lieten een grotere vooruitgang zien op het gebied van visueel werkgeheugen, leerkracht gerapporteerde metacognitieve vaardigheden, directe leervoorwaarden (d.w.z. concentratie, motivatie, werkhouding en taak oriëntatie) en de kwaliteit van leven schalen psychologisch welbevinden en schoolomgeving, gerapporteerd door ouders. Deze resultaten suggereren 165

168 Nederlandse samenvatting dat een klinisch significante vooruitgang in werkgeheugen een voorwaarde is om ook op niet getrainde domeinen vooruit te kunnen gaan. De nonresponders die uiteindelijk ook nog de Cogmed werkgeheugen training volgden (n=6), verbeterden na de training alleen op het gebied van visueel werkgeheugen. Ondanks dat dit een kleine groep betrof, impliceert dit dat deze kinderen met een zwakker cognitief profiel maar moeizaam kunnen profiteren van cognitieve trainingen en dat aangepaste en alternatieve interventies noodzakelijk zijn. Groeicurves van getrainde taken In hoofdstuk 5 richten we ons op de individuele verschillen in vooruitgang op de adaptieve werkgeheugen oefeningen van de Beter Bij de Les training. Enerzijds onderzochten we hoe deze individuele verschillen in groeicurves invloed hadden op niet getrainde werkgeheugen taken en academische vaardigheden direct na de training. Anderzijds onderzochten we of leeftijd, intelligentieniveau, externalizerende gedragsproblemen of een additionele leerstoornis de individuele verschillen in groeicurves konden voorspellen. Op basis van de data die werd verzameld in hoofdstuk 4 analyseerden we de vooruitgang van de getrainde verbale (woorden onthouden) en visuele (blokjes achteruit) werkgeheugen taak voor 154 kinderen in totaal. De resultaten lieten zien dat er voor beide werkgeheugentaken individuele verschillen waren in het beginniveau van de training maar ook individuele verschillen in de groeicurves, ofwel de vooruitgang op de taken. Leeftijd en intelligentieniveau voorspelden het niveau van beide werkgeheugentaken aan het begin van de training waarbij oudere en intelligentere kinderen een hoger startniveau hadden. Echter geen van de variabelen voorspelden de individuele verschillen in de groeicurves. Verder lieten de resultaten zien dat kinderen die meer vooruit gingen op de getrainde verbale werkgeheugen taak (steilere groeicurve) ook meer vooruit gingen op de niet getrainde verbale werkgeheugentaak die direct na de training werd afgenomen. De groeicurves van de getrainde visuele werkgeheugentaak taak hadden geen invloed op de niet getrainde visuele werkgeheugentaak. Tenslotte hadden de individuele verschillen in groeicurves geen significante invloed op de academische uitkomstmaten. Het forse percentage onverklaarde 166

169 Nederlandse samenvatting variantie dat werd gevonden, impliceert dat er nog vele andere factoren zijn die bijdragen aan de effecten van een cognitieve training die in vervolgonderzoek nog geëxploreerd kunnen worden. Conclusie en implicaties Het huidige onderzoek heeft laten zien dat cognitieve training goed implementeerbaar is binnen de onderwijscontext. De uitvalspercentages waren laag en de Beter Bij de Les training sloot goed aan bij de vraag uit de praktijk. Vele hulpverleners, leerkrachten, ouders en kinderen werkten enthousiast mee aan het onderzoek en waren ook positief over de toepasbaarheid van de Beter Bij de Les training in de onderwijscontext. Het onderzoek heeft laten zien dat kinderen vooruit gaan op verscheidene uitkomstmaten en dat zowel klinische als cognitieve individuele verschillen een belangrijke rol spelen in het bepalen van de effectiviteit van de training. Voor clinici is het van belang om cognitieve training te zien als een aanvullend behandelaanbod, rekening houdend met het feit dat het geen one size fits all behandeling is. De keuze of en welke cognitieve training geïndiceerd wordt, moet gebaseerd zijn op een weloverwogen beslissing. Naast het vooraf in kaart brengen van de cognitieve en functionele beperkingen van het kind, zal de clinicus moeten beoordelen of de omgeving aan alle randvoorwaarden kan voldoen om de training optimaal te laten verlopen en zullen ook de verwachtingen goed met ouders en leerkrachten besproken moeten worden. Het onderzoek heeft ook laten zien dat cognitieve training geen snelle en makkelijke oplossing is voor ADHD gerelateerde problematiek. Het laten generaliseren van de effecten naar een klassensituatie is een complex proces dat ook inspanningen vereist van de omgeving om de effecten goed tot zijn recht te laten komen. Ook bepaalde praktische zaken en randvoorwaarden zoals het plannen van een gunstige trainingstijd en het trainen in een relatief rustige ruimte speelden een belangrijke rol bij het laten slagen van de training. Gezien de invoering van de Wet Passend Onderwijs, waarbij scholen verantwoordelijk zijn voor het bieden van gepaste begeleiding voor kinderen 167

170 Nederlandse samenvatting die extra zorg nodig hebben, zal de vraag voor soortgelijke interventies als Beter Bij de Les alleen maar toenemen. Daarnaast vraagt de huidige transformatie en transitie van de jeugdzorg naar de gemeenten ook om een intensieve samenwerking tussen hulpverleners, ouders en leerkrachten. Toekomstige interventies zullen zich daarom ook meer moeten richten op het integreren van de rollen van alle personen die betrokken zijn bij het kind. Tenslotte brengt een individuele behandeling een aanzienlijke financiële investering met zich mee en gezien het grote aantal kinderen met executieve functie problemen zal ook de toepasbaarheid van een groepsinterventie overwogen moeten worden in toekomstig onderzoek. Dit proefschrift startte met een vrij simpele vraag namelijk: is cognitieve training effectief voor kinderen met ADHD? Al snel bleek het beantwoorden van deze vraag echter een stuk complexer dan verwacht. Toekomstig onderzoek zal rekening moeten houden met de vele factoren die de effecten van de training kunnen beïnvloeden zoals methodologische kwesties, specifieke en niet specifieke werkingsmechanismen, individuele verschillen en specifieke kenmerken van een training voordat cognitieve training aan de volledige criteria voldoet als evidence-based behandeling voor ADHD. 168

171 References

172 References Alloway, T.C. (2011). Improving working memory. Supporting student s learning. London: Sage. Alloway, T. P., Gathercole, S. E., Kirkwood, H., & Elliott, J. (2009). The cognitive and behavioral characteristics of children with low working memory. Child Development, 80, American Psychiatric Association [APA]. (2000). Diagnostic and statistical manual of mental disorders (4th ed.). Washington, DC: Author. American Psychiatric Association [APA]. (2013). Attention-deficit/hyperactivity disorder. Diagnostic and statistical manual of mental disorders (fifth ed., pp. 59). Washington, DC: Author. Ardito, R.B., & Rabellino, D. (2011). Therapeutic alliance and outcome of psychotherapy: historical excursus, measurements, and prospects for research. Frontiers in Psychology, 2, article 270. doi: /fpsyg Baddeley, A. (2000). The episodic buffer: a new component of working memory? Trends in cognitive sciences, 4, Baddeley, A. (2007). Working memory, thought, and action, 1st Edn. Oxford: Oxford University Press. doi: /acprof:oso/ Baddeley, A. D., & Hitch, G. (1974). Working memory. In G. A. Bower (Ed.), The psychology of learning and motivation (Vol. 8, pp ). New York: Academic Press. Barkley, R. A. (1997). Behavioral inhibition, sustained attention, and executive functions: Constructing a unifying theory of ADHD. Psychological Bulletin, 121, doi: / Barlow, D.H., Nock, M.K., & Hersen, M. (2009). Single Case Experimental Designs: Strategies for Studying Behavior Change (3rd Ed.). Boston, MA: Allyn and Bacon. Beck, S. J., Hanson, C. A., Puffenberger, S. S., Benninger, K. L., & Benninger, W. B. (2010). A controlled trial of working memory training for children and adolescents with ADHD. Journal of Clinical Child and Adolescent Psychology, 39, doi: / Berger, I., Dor, T., Nevo, Y., & Goldzweig, G. (2008). Attitudes toward attention-deficit hyperactivity disorder (ADHD) treatment: Parents and children s perspectives. Journal of Child Neurology, 23, doi: / Bergman-Nutley, S., & Klingberg, T. (2014). Effect of working memory training on working memory, arithmetic and following instructions. Psychological Research, 78, doi: / s Brus, B. T., & Voeten, M. J. M. (1973). Een Minuut Test. [One Minute Test] Nijmegen: Berkhout testmateriaal B.V. Buitelaar, J. K., Van der Gaag, R. J., Swaab-Barneveld, H., & Kuiper, M. (1995). Prediction of clinical response to methylphenidate in children with attention-deficit hyperactivity disorder. Journal of the American Academy of Child and Adolescent Psychiatry, 34, Bürki, C. N., Ludwig, C., Chicherio, C., & de Ribaupierre, A. (2014). Individual differences in cognitive plasticity: An investigation of training curves in younger and older adults. Psychological Research, 78, Castellanos, F. X., Sonuga-Barke, E. J., Milham, M. P., & Tannock, R. (2006). Characterizing cognition in ADHD: Beyond executive dysfunction. Trends in Cognitive Sciences, 10, doi: /j.tics Castellanos, F. X., & Tannock, R. (2002). Neuroscience of attention-deficit/hyperactivity disorder: The search for endophenotypes. Nature Reviews Neuroscience, 3, Chacko, A., Bedard, A. C., Marks, D. J., Feirsen, N., Uderman, J. Z., Chimiklis, A., et al. (2014). A randomized clinical trial of cogmed working memory training in school-age children with ADHD: A replication in a diverse sample using a control condition. Journal of Child Psychology and Psychiatry, and Allied Disciplines, 55, doi: /jcpp Chacko, A., Feirsen, N., Bedard, A. C., Marks, D., Uderman, J. Z., & Chimiklis, A. (2013). Cogmed working memory training for youth with ADHD: A closer examination of efficacy utilizing evidence-based criteria. Journal of Clinical Child and Adolescent Psychology, 42, doi: /

173 References Chacko, A., Kofler, M., & Jarrett, M. (2014). Improving outcomes for youth with ADHD: A conceptual framework for combined neurocognitive and skill-based treatment approaches. Clinical Child and Family Psychology Review, 17, doi: /s Chatoor, I., & Krupnick, J. (2001). The role of non-specific factors in treatment outcome of psychotherapy studies. European Child and Adolescent Psychiatry, 10, Corsi, P. M. (1972). Human memory and the medial temporal region of the brain. Diss.Abstr.Int. 34, 891B. Cortese, S., Ferrin, M., Brandeis, D., Buitelaar, J., Daley, D., Dittmann, R. W., et al. (2015). Cognitive training for attention-deficit/hyperactivity disorder: Meta-analysis of clinicaland neuropsychological outcomes from randomized controlled trials. Journal of the American Academy of Child and Adolescent Psychiatry, 54, doi: /j.jaac Dahlin, K. (2011). Effects of working memory training on reading in children with special needs. Reading and Writing, 24, doi: /s y Dahlin, K. (2013). Working memory training and the effect on mathematical achievement in children with attention deficits and special needs. Journal of Education and Learning, 2, doi: /jel.v2n1p118 Dawson, P. & Guare, R. (2014). Interventions to promote executive development in children and adolescents. In Goldstein, S., & Naglieri, J.A. (Eds.), Handbook of executive functioning (pp ). New York: Springer. Dehn, M. J. (2008). Working memory and academic learning. New Jersey: John Wiley & Sons. De Vos, T. (2010). Tempo test automatiseren. [Automating time test] Amsterdam: Boom test uitgevers. Diamond, A. (2005). Attention-deficit disorder (attention-deficit/ hyperactivity disorder without hyperactivity): A neurobiologically and behaviorally distinct disorder from attention-deficit/ hyperactivity disorder (with hyperactivity). Development and Psychopathology, 17, doi: /s Diamond, A., & Ling, D.S. (2015). Conclusions about interventions, programs, and approaches for improving executive functions that appear justified and those that, despite much hype, do not. Developmental Cognitive Neuroscience. doi: /j.dcn Dovis, S. (2014). Cognitive control and motivation in children with ADHD. How reinforcement interacts with the assessment and training of executive functioning (Doctoral dissertation). Amsterdam: University of Amsterdam. Retrieved from Dovis, S., Van der Oord, S., Wiers, R. W., & Prins, P. J. (2012). Can motivation normalize working memory and task persistence in children with attention-deficit/hyperactivity disorder? the effects of money and computer-gaming. Journal of Abnormal Child Psychology, 40, doi: /s Dovis, S., Van der Oord, S., Wiers, R. W., & Prins, P. J. (2015). Improving executive functioning in children with ADHD: training multiple executive functions within the context of a computer game. A randomized double-blind placebo controlled trial. Plos One, 10, doi: / journal.pone Duncan, B.L., Miller, S.D. & Sparks, J.A. (2004). The Heroic Client: a revolutionary way to improve effectiveness through client-directed outcome-informed therapy. San Fransisco: Jossey-Bass. Dunning, D.L., & Holmes, J. (2014). Does working memory training promote the use of strategies on untrained working memory tasks? Memory & Cognition, 42, doi: /s Egeland, J., Aarlien, A. K., & Saunes, B. K. (2013). Few effects of far transfer of working memory training in ADHD: A randomized controlled trial. PloS One, 8, doi: /journal.pone Gathercole, S. E. (2014). Commentary: Working memory training and ADHD - where does its potential lie? reflections on chacko et al. (2014). Journal of Child Psychology and Psychiatry, and Allied Disciplines, 55, doi: /jcpp Gathercole, S.E., Durling, E., Evans, M., Jeffcock, S. & Stone, S. (2008). Working memory abilities and children s performance in laboratory analogues of classroom activities. Applied Cognitive Psychology, 22, doi: /acp

174 References Gathercole, S. E., Pickering, S. J., Knight, C., & Stegmann, Z. (2004). Working memory skills and educational attainment: Evidence from national curriculum assessments at 7 and 14 years of age. Applied Cognitive Psychology, 18, doi: /acp.934 Geelhoed, J., & Reitsma, P. (1999). PI dictee. [Dictation] Lisse: Swets & Zeitlinger. Gerrits, B. J. L., van der Zwaag, W. D., Gerrits-Entken, & M.E.A., van Berkel, S.L. (2012). Handleiding voor coaches cogmed werkgeheugentraining [manual coaches Cogmed Working Memory Training]. Amsterdam: Pearson Assessment and Information. Gibson, B.S., Gondoli, D.M., Johnson, A.C., Steeger, C.M., Dobrzenski, B.A., & Morrissey, R.A. Component analysis of verbal versus spatial working memory training in adolescents with ADHD: A randomized, controlled trial. Child Neuropsychology, 17, doi: / Gillberg, C., Gillberg, I. C., Rasmussen, P., Kadesjo, B., Soderstrom, H., Rastam, M., et al. (2004). Co-existing disorders in ADHD - implications for diagnosis and intervention. European Child & Adolescent Psychiatry, 13 Suppl 1, I Graham, J., & Coghill, D. (2008). Adverse effects of pharmacotherapies for attention-deficit hyperactivity disorder: Epidemiology, prevention and management. CNS Drugs, 22, doi: / Gray, S. A., Chaban, P., Martinussen, R., Goldberg, R., Gotlieb, H., Kronitz, R., et al. (2012). Effects of a computerized working memory training program on working memory, attention, and academics in adolescents with severe LD and comorbid ADHD: A randomized controlled trial. Journal of Child Psychology and Psychiatry, and Allied Disciplines, 53, doi: /j x Green, C. T., Long, D. L., Green, D., Iosif, A. M., Dixon, J. F., Miller, M. R., et al. (2012). W i l l working memory training generalize to improve off-task behavior in children with attentiondeficit/hyperactivity disorder? Neurotherapeutics: The Journal of the American Society for Experimental NeuroTherapeutics, 9, doi: /s y Green, C. S., Strobach, T., & Schubert, T. (2014). On methodological standards in training and transfer experiments. Psychological Research, 78, Hinshaw, S. P. (2007). Moderators and mediators of treatment outcome for youth with ADHD: Understanding for whom and how interventions work. Journal of Pediatric Psychology, 32, doi: /jpepsy/jsl055 Holmes, J., & Gathercole, S. E. (2013). Taking working memory training from the laboratory into schools. Educational Psychology, 34, doi: / Holmes, J., Gathercole, S. E., & Dunning, D. L. (2009). Adaptive training leads to sustained enhancement of poor working memory in children. Developmental Science, 12, F9-15. doi: /j x Holmes, J., Gathercole, S. E., & Dunning, D. L. (2010). Poor working memory: Impact and interventions. In J. Holmes (Ed.), Advances in child development and behavior (pp. 1-43). Burlington: Academic Press. Holmes, J., Gathercole, S. E., Place, M., Dunning, D. L., Hilton, K. A., & Elliott, J. G. (2010). Working memory deficits can be overcome: Impacts of training and medication on working memory in children with ADHD. Applied Cognitive Psychology, 24, doi: /acp.1589 Hovik, K. T., Saunes, B. K., Aarlien, A. K., & Egeland, J. (2013). RCT of working memory training in ADHD: Long-term near-transfer effects. PloS One, 8, doi: /journal.pone Jacobson, N. S., & Truax, P. (1991). Clinical significance: A statistical approach to defining meaningful change in psychotherapy research. Journal of Consulting and Clinical Psychology, 59, Jaeggi, S. M., Buschkuehl, M., Jonidas, J., & Shah, P. (2011). Short- and long-term benefits of cognitive training. Proc. Nat. Acad. Sci. U.S.A., 108, doi: /pnas Jaeggi, S. M., Buschkuehl, M., Shah, P., & Jonides, J. (2014). The role of individual differences in cognitive training and transfer. Memory & Cognition, 42, doi: /s z Jensen, P. S., Arnold, L. E., Swanson, J. M., Vitiello, B., Abikoff, H. B., Greenhill, L. L., et al. (2007). 172

175 References 3-year follow-up of the NIMH MTA study. Journal of the American Academy of Child and Adolescent Psychiatry, 46, doi: /CHI.0b013e d48 Jolles, D. D., & Crone, E. A. (2012). Training the developing brain: A neurocognitive perspective. Frontiers in Human Neuroscience, 6, doi: /fnhum Karbach, J., & Unger, K. (2014). Executive control training from middle childhood to adolescence. Frontiers in Psychology, 5, Article 390. doi: /fpsyg Kasper, L. J., Alderson, R. M., & Hudec, K. L. (2012). Moderators of working memory d e f i c i t s in children with attention-deficit/hyperactivity disorder (ADHD): a meta-analytic review. Clinical Psychology Review, 32, Kline, R. B. (2011). Principle and Practice of Structural Equation Modeling. New York, NJ: The Guildford Press. Klingberg, T. (2010). Training and plasticity of working memory. Trends in Cognitive Sciences, 14, doi: /j.tics Klingberg, T., Fernell, E., Olesen, P. J., Johnson, M., Gustafsson, P., Dahlstrom, K., et al. (2005). Computerized training of working memory in children with ADHD-a randomized, controlled trial. Journal of the American Academy of Child and Adolescent Psychiatry, 44, doi: / Klingberg, T., Forssberg, H., & Westerberg H. (2002). Training of working memory in children with ADHD. Journal of Clinical and Experimental Neuropsychology, 24, doi: / jcen Kofler, M. J., Rapport, M. D., Bolden, J., Sarver, D. E., & Raiker, J. S. (2010). ADHD and working memory: The impact of central executive deficits and exceeding storage/rehearsal capacity on observed inattentive behavior. Journal of Abnormal Child Psychology, 38, Kofler, M. J., Rapport, M. D., Bolden, J., Sarver, D. E., Raiker, J. S., & Alderson, R.M. (2011). Working memory deficits and social problems in children with ADHD. Journal of Abnormal Child Psychology, 39, Könen, T., & Karbach, J. (2015). The benefits of looking at intraindividual dynamics in cognitive training data. Frontiers in Psychology, 6, Article 615. doi: /fpsyg Kraemer, H. C., Wilson, G. T., Fairburn, C. G., & Agras, W. S. (2002). Mediators and moderators of treatment effects in randomized clinical trials. Archives of General Psychiatry, 59, doi: /archpsyc Landelijke Stuurgroep Multidisciplinaire Richtlijnontwikkeling in de GGZ. (2005). Multidisciplinaire richtlijn ADHD bij kinderen en jeugdigen. Utrecht: Trimbos instituut. Le, H. H., Hodgkins, P., Postma, M. J., Kahle, J., Sikirica, V., Setyawan, J., et al. (2014). Economic impact of childhood/adolescent ADHD in a european setting: The netherlands as a reference case. European Child & Adolescent Psychiatry, 23, Little, T. D. (2013). Longitudinal Structural Equation Modeling. New York, NJ; The Guildford Press. Loe, I. M., & Feldman, H. M. (2007). Academic and educational outcomes of children with ADHD. Ambulatory Pediatrics : The Official Journal of the Ambulatory Pediatric Association, 7(1 Suppl), doi: /j.ambp Mawjee, K., Woltering, S., Lai, N., Gotlieb, H., Kronitiz, R., & Tannock, R. (2014). Working memory training in ADHD: controlling for engagement, motivation, and expectancy of improvement (pilot study). Journal of Attention Disorders, 10, doi: / Manley, T., Robertson, H., Anderson, V., & Nimmo-Smith, I. (2004). Test of everyday attention for children. Enschede: Harcourt Assessment. Martinussen, R., Hayden, J., Hogg-Johnson, S., & Tannock, R. (2005). A meta-analysis of working memory impairments in children with attention-deficit/hyperactivity disorder. Journal of the American Academy of Child and Adolescent Psychiatry, 44, Mazzone, L., Postorino, V., Reale, L., Guarnera, M., Mannino, V., Armando, M., et al. (2013). Selfesteem evaluation in children and adolescents suffering from ADHD. Clinical Practice and Epidemiology in Mental Health : CP & EMH, 9, McGrath, L. M., Pennington, B. F., Shanahan, M. A., Santerre-Lemmon, L. E., Barnard, H. D., 173

176 References Willcutt., E. G., et al. (2011). A multiple deficit model of reading disability and attention-deficit/ hyperactivity disability: Searching for shared cognitive deficits. Journal of Child Psychology and Psychiatry, 52, doi: /j x Melby-Lervag, M., & Hulme, C. (2013). Is working memory training effective? A meta-analytic review. Developmental Psychology, 49, doi: /a Meltzer, L. (2014). Teaching executive functioning processes: promoting metacognition, strategy use, and effort. In Goldstein, S., & Naglieri, J.A. (Eds.), Handbook of executive functioning (pp ). New York: Springer. Merrill, R. M., Lyon, J. L., Baker, R. K., & Gren, L. H. (2009). Attention deficit hyperactivity disorder and increased risk of injury. Advances in Medical Sciences, 54, Moreau, D. (2014). Making sense of discrepancies in working memory training experiments: a monte carlo simulation. Frontiers in Systems Neuroscience, 8, Article 161. doi: / fnsys Moreau, D., & Conway, A.R.A. (2014). The case for an ecological approach to cognitive training. Trends in Cognitive Sciences, 18, doi: /j.tics Morrison, A. B., & Chein, J. M. (2011). Does working memory training work? the promise and challenges of enhancing cognition by training working memory. Psychonomic Bulletin & Review, 18, doi: /s MTA cooperative group. (1999a). A 14-month randomized clinical trial of treatment strategies for attention-deficit/hyperactivity disorder. Multimodal treatment study of children with ADHD. Archives of General Psychiatry, 56, doi: /archpsyc MTA cooperative group. (1999b). Moderators and mediators of treatment response for children with attention-deficit/hyperactivity disorder. Archives of General Psychiatry, 56, Olesen, P.J, Westerberg, H., & Klingeberg, T. (2004). Increased prefrontal and parietal activity after training of working memory. Nature Neuroscience, 7, Orban, S., Rapport, M., Friedman, L. M., & Kofler, M. J. (2014). Executive Function/ Cognitive training for children with ADHD: Do results warrant the hype and cost? The ADHD Report, 22, doi: /adhd Otero, T.M., Barker, L.A., & Naglieri, J.A. (2014). Executive function treatment and intervention in schools. Applied Neuropsychology Child, 3, doi: / Ottersen, J., & Grill, K. M. (2015). Benefits of extending and adjusting the level of difficulty on computerized cognitive training for children with intellectual disabilities. Frontiers in Psychology, 6, doi: /fpsyg Owens, E. B., Hinshaw, S. P., Kraemer, H. C., Arnold, L. E., Abikoff, H. B., Cantwell, D. P., et al. (2003). Which treatment for whom for ADHD? moderators of treatment response in the MTA. Journal of Consulting and Clinical Psychology, 71, doi: / X Pastor, P. N., & Reuben, C. A. (2006). Identified attention-deficit/hyperactivity disorder and medically attended, nonfatal injuries: US school-age children, Ambulatory Pediatrics : The Official Journal of the Ambulatory Pediatric Association, 6, Pelham, W. E., Foster, E. M., & Robb, J. A. (2007). The economic impact of attention-deficit/ hyperactivity disorder in children and adolescents. Journal of Pediatric Psychology, 32, Polanczyk, G. V., Salum, G. A., Sugaya, L. S., Caye, A., & Rohde, L. A. (2015). Annual research review: A meta-analysis of the worldwide prevalence of mental disorders in children and adolescents. Journal of Child Psychology and Psychiatry, and Allied Disciplines, 56, Preacher, K.J., Wichman, A.L., MacCallum, R.C., & Briggs, N.E. (2008). Latent growth curve modeling. Sage Publications. Prins, P.J.M., Ollendick, T.H., Maric, M., & MacKinnon, D.P. (2015). Moderators and mediators in treatment outcome studies of childhood disorders: the what, why and how. New York: Oxford University Press. doi: /med:psych/ R Core Team (2015). R: A language and environment for statistical computing. Vienna, Austria: R Foundation for Statistical Computing. 174

177 References Raggi, V. L., & Chronis, A. M. (2006). Interventions to address the academic impairment of children and adolescents with ADHD. Clinical Child and Family Psychology Review, 9, doi: /s Raiker, J. S., Rapport, M. D., Kofler M. J., & Sarver, D. E. (2012). Objectively-measured impulsivity and attention-deficit/hyperactivity disorder (ADHD): testing competing predictions from the working memory and behavioral inhibition models of ADHD. Journal of Abnormal Child Psychology, 40, doi: /s Rapport, M. D., Alderson, R. M., Kofler, M. J., Sarver, D. E., Bolden, J., & Sims, V. (2008). Working memory deficits in boys with attention-deficit/hyperactivity disorder (ADHD): The contribution of central executive and subsystem processes. Journal of Abnormal Child Psychology, 36, Rapport, M. D., Orban, S. A., Kofler, M. J., & Friedman, L. M. (2013). Do programs designed to train working memory, other executive functions, and attention benefit children with ADHD? A meta-analytic review of cognitive, academic, and behavioral outcomes. Clinical Psychology Review, 33, doi: /j.cpr Ravens-Sieberer, U., Auquier, P., Erhart, M., Gosch, A., Rajmil, L., Bruil, J., et al. (2007). The kidscreen-27 quality of life measure for children and adolescent: Psychometric results from a cross-cultural survey in 13 european countries. Quality of Life Research, 16, doi: /s Rutledge, K. J., van den Bos, W., McClure, S. M., & Schweitzer, J. B. (2012). Training cognition in ADHD: Current findings, borrowed concepts, and future directions. Neurotherapeutics : The Journal of the American Society for Experimental NeuroTherapeutics, 9, doi: /s Scholte, E. M., & Van der Ploeg, J. D. (2009). Leervoorwaardentest. [Learning condition test] Houten: Bohn Stafleu van Loghum. Shah, P., Buschkuehl, M., Jaeggi, S. M., & Jonides, J. (2012). Cognitive training for children with ADHD: The importance of individual differences. Journal of Applied Research in Memory and Cognition, 1, doi: /s Shaw, M., Hodgkins, P., Caci, H., Young, S., Kahle, J., Woods, A. G., et al. (2012). A systematic review and analysis of long-term outcomes in attention deficit hyperactivity disorder: Effects of treatment and non-treatment. BMC Medicine, 10, Shinaver, C. S., Entwistle, P. C., & Söderqvist, S. (2014). Cogmed WM training: Reviewing the reviews. Applied Neuropsychology.Child, 3, doi: / Shipstead, Z., Hicks, K. L., & Engle, R. W. (2012). Cogmed working memory training: Does the evidence support the claims?journal of Applied Research in Memory and Cognition, 1, doi: /j.jarmac Shipstead, Z., Redick, T. S., & Engle, R. W. (2010). Does working memory training generalize? Psychologica Belgica, 50, doi: /pb Shipstead, Z., Redick, T. S., & Engle, R. W. (2012). Is working memory training effective? Psychological Bulletin, 138, doi: /a Smidts, D., & Huizinga, M. (2009). BRIEF executieve functies gedragsvragenlijst.amsterdam: Hogrefe Uitgevers B.V. [Dutch translation from: Gioia, G.A., Isquith, P.K., Guy, S.C., & Kenworthy, L. (2002) Behavior Rating Inventory of Executive Function. Odessa, FL: Psychological Assessment Resources] Söderqvist, S., & Bergman Nutley, S. (2015). Working memory training is associated with long term attainments in math and reading. Frontiers in Psychology, 6, Article doi: / fpsyg Söderqvist, S., Nutley, S. B., Ottersen, J., Grill, K. M., & Klingberg, T. (2012). Computerized training of non-verbal reasoning and working memory in children with intellectual disability. Frontiers in Human Neuroscience, 6, Article 271. doi: /fnhum Sonuga-Barke, E. J. S. (2002). Psychological heterogeneity in AD/HD a dual pathway model of behaviour and cognition. Behavioral and Brain Research, 130, Sonuga-Barke, E. J. (2003). The dual pathway model of AD/HD: An elaboration of neuro- 175

178 References developmental characteristics. Neuroscience and Biobehavioral Reviews, 27, doi: /j.neubiorev Sonuga-Barke, E. J., Brandeis, D., Cortese, S., Daley, D., Ferrin, M., Holtmann, M., et al. (2013). Nonpharmacological interventions for ADHD: Systematic review and meta-analyses of randomized controlled trials of dietary and psychological treatments. The American Journal of Psychiatry, 170, doi: /appi.ajp Sonuga-Barke, E., Brandeis, D., Holtmann, M., & Cortese, S. (2014). Computer-based cognitive training for ADHD: A review of current evidence. Child and Adolescent Psychiatric Clinics of North America, 23, doi: /j.chc Steenhuis, M. P., Serra, M., Minderaa, R. B., & Hartman, C. A. (2009). An internet version of the diagnostic interview schedule for children (DISC-IV): Correspondence of the ADHD section with the paper-and-pencil version. Psychological Assessment, 21, doi: / a Tamm, L., Epstein, J. N., Peugh, J.L., Nakonezny, P.A., & Hughes, C.W. (2013). Preliminary data suggesting the efficacy of attention training for school-aged children with ADHD. Developmental Cognitive Neuroscience, 4, doi: /j.dcn Taylor, E., Dopfner, M., Sergeant, J., Asherson, P., Banaschewski, T., Buitelaar, J., et al. (2004). European clinical guidelines for hyperkinetic disorder - first upgrade. European Child & Adolescent Psychiatry, 13 Suppl 1, I7-30. doi: /s x Titz, C., & Karbach, J. (2014). Working memory and executive functions: Effects of training on academic achievement. Psychological Research, 78, doi: /s Tjeenk-Kalff, A., & Krabbendam, L. (2006). Behavioural assessment of the dysexecutive syndrome for children. Amsterdam: Pearson Assessment and Information. [Dutch translation from: Emslie, H., Wilson, F. C., Burden, V., Nimmo-Smith, I., & Wilson, B.A. (2003). Behavioural assessment of the dysexecutive syndrome for children (BADS-C). Suffolk: Thames Valley Test Company] Turgay, A., Goodman, D. W., Asherson, P., Lasser, R. A., Babcock, T. F., Pucci, M. L., et al. (2012). Lifespan persistence of ADHD: The life transition model and its application. The Journal of Clinical Psychiatry, 73, Van der Donk, M., Tjeenk-Kalff, A., & Hiemstra-Beernink, A. (2015). Beter Bij de Les: handleiding. Training in executieve functies. [Paying Attention in Class: manual. Training in executive functions] Houten: LannooCampus Van der Donk, M., Hiemstra-Beernink, A., Tjeenk-Kalff, A., Van der Leij, A., & Lindauer, R. (2015). Cognitive training for children with ADHD: A randomized controlled trial of cogmed working memory training and paying attention in class. Frontiers in Psychology, 6, Article doi: /fpsyg Van der Oord, S. & Daley, D. (2015). Moderators and mediators of treatments for youth with ADHD. In Prins, P.J.M, Ollendick, T.H., Maric, M., & Mackinnon, D.P. (Eds.), Moderators and mediators in treatment outcome studies of childhood disorders: the what, why and how. New York: Oxford University Press. doi: /med:psych/ Van der Oord, S., Prins, P. J., Oosterlaan, J., & Emmelkamp, P. M. (2008a). Efficacy of methylphenidate, psychosocial treatments and their combination in school-aged children with ADHD: A meta-analysis. Clinical Psychology Review, 28, doi: /j. cpr Van der Oord, S., Prins, P. J., Oosterlaan, J., & Emmelkamp, P. M. (2008b). Treatment of attention deficit hyperactivity disorder in children. Predictors of treatment outcome. European Child & Adolescent Psychiatry, 17, Van der Oord, S., Van der Meulen, E. M., Prins, P. J., Oosterlaan, J., Buitelaar, J. K., & Emmelkamp, P. M. (2005). A psychometric evaluation of the social skills rating system in children with attention deficit hyperactivity disorder. Behaviour Research and Therapy, 43, Van Dongen-Boomsma, M., Vollebregt, M. A., Buitelaar, J. K., & Slaats-Willemse, D. (2014). Working memory training in young children with ADHD: A randomized placebo-controlled trial. Journal of Child Psychology and Psychiatry, and Allied Disciplines, 55, doi: / jcpp

179 References Verhulst, F. C., van der Ende, J., & Koot, H. M. (1996). Handleiding child behaviour checklist Rotterdam: Afdeling Kinder en Jeugdpsychiatrie, Sophia Kinderziekenhuis/ Academisch Ziekenhuis Rotterdam. [Dutch translation from: Achenbach, T. M., & Rescorla, L. A. (2001). Manual for the ASEBA School-Age Forms & Profiles. Burlington, VT: University of Vermont, Research Center for Children, Youth, & Families] Verhulst, F. C., van der Ende, J., & Koot, H. M. (1997). Handleiding Teacher s report form. Rotterdam: Afdeling Kinder en Jeugdpsychiatrie, Sophia Kinderziekenhuis/Academisch Ziekenhuis Rotterdam. [Dutch translation from: Achenbach, T. M., & Rescorla, L. A. (2001). Manual for the ASEBA School-Age Forms & Profiles. Burlington, VT: University of Vermont, Research Center for Children, Youth, & Families] Von Bastian, C. C., & Eschen, A. (2015). Does working memory have to be adaptive? Psychological Research, 80, doi: /s z Von Bastian, C. C., & Oberauer, K. (2013). Effects and mechanisms of working memory training: A review. Psychological Research, 78, doi: /s Warreyn, P., Raymaekers, R., & Roeyers, H. (2004). SCQ: Vragenlijst sociale communicatie. Destelbergen: Vormingsdienst SIG vzw. [Dutch translation from: Rutter, M., Bailey,B. & Lord, C. Social Communication Questionnaire. WPS Publish] Wechsler, D. (2005). Wechsler intelligence scale for children-iii-nl. Londen: Harcourt Test Publishers. Wechsler, D., & Naglieri, J. A. (2008). Wechsler nonverbal-nl. Amsterdam: Pearson Assessment and Information. Weiss, M. D., & Salpekar, J. (2010). Sleep problems in the child with attention-deficit hyperactivity disorder: Defining aetiology and appropriate treatments. CNS Drugs,24, Willcutt, E. G., Doyle, A. E., Nigg, J. T., Faraone, S. V., & Pennington, B. F. (2005). Validity of the executive function theory of attention-deficit/hyperactivity disorder: A meta-analytic review. Biological Psychiatry, 57, doi: /j.biopsych Willcutt, E. G., Nigg, J. T., Pennington, B. F., Solanto, M. V., Rohde, L. A., Tannock, R., et al. (2012). Validity of DSM-IV attention deficit/hyperactivity disorder symptom dimensions and subtypes. Journal of Abnormal Psychology, 121, Wolraich, M., Brown, L., Brown, R. T., DuPaul, G., et al. (2011). Subcommittee on Attention-Deficit/ Hyperactivity Disorder, Steering Committee on Quality Improvement and Management, ADHD: Clinical practice guideline for the diagnosis,evaluation, and treatment of attentiondeficit/hyperactivity disorder in children and adolescents. Pediatrics, 128, Zelazo, P. D., & Müller, U. (2011). Executive function in typical and atypical development. In U. Goswani (Ed.), The wiley-blackwell handbook of childhood cognitive development. (second ed., pp ) Blackwell Publishing. Zijlstra, H., Kingma, A., Swaab, H., & Brouwer, W. (2010). NEPSY-II-nl. Amsterdam: Pearson Assessment and Information. [Dutch translation from: Korkman, M., Kirk, U., & Kemp, S. (2007). NEPSY-II. Harcourt Assessment] 177

180

181 Dankwoord

182 Dankwoord Na vijf jaar is het proefschrift dan eindelijk af! Het was een waar proces dat heel wat bloed, zweet, tranen en koppen koffie heeft gekost maar zonder de steun van fijne collega s, vrienden en familie was dit een onmogelijke opgave geweest. Graag spreek ik mijn dank uit voor allen die hebben bijgedragen om dit proces succesvol af te sluiten. Ten eerste gaat mijn dank uit naar mijn promotoren em. prof. dr. Aryan van der Leij en prof. dr. Ramón Lindauer. Aryan, dank voor alle inspiratievolle momenten waarop jij jouw overweldigende dosis aan kennis en ervaring met mij deelde. Met jouw helikopter view wist je steeds de onmisbare structuur aan te brengen en ik voel me vereerd dat ik een van jouw laatste promovendi mocht zijn. Ramón, dank dat jij mij de afgelopen jaren de ruimte en het vertrouwen gaf om mijn eigen weg te volgen binnen het onderzoek. Ik waardeer het ten zeerste dat jij altijd open stond voor nieuwe ideeën en dat je ook altijd met snelle en flexibele oplossingen kwam als er weer eens een probleem opgelost moest worden. Ook wil ik mijn copromotoren dr. Anne- Claire Hiemstra-Beernink en dr. Ariane Tjeenk-Kalff bedanken voor hun nimmer aflatende professionele en persoonlijke steun van de afgelopen jaren. Voor mij vormden jullie hét voorbeeld van hoe wetenschappelijk onderzoek en de klinische praktijk elkaar kunnen verrijken. Ik heb ontzettend veel mogen leren van jullie kritische doch enthousiaste reacties op al mijn stukken en jullie expertise was dan ook een onmisbare factor voor dit proefschrift. Jullie wisten mij ook op de juiste momenten dat noodzakelijke zetje in de rug te geven waardoor ik er altijd weer vol goede moed voor kon gaan. Dank voor alles lieve Anne-Claire en Ariane, ik hoop dat we in de toekomst samen nog aan vele mooie projecten mogen werken! Dit proefschrift had ook niet tot stand kunnen komen zonder de bijdrage van Jan Geelhoed en Jehanne Vieijra. Met veel plezier en genoegen kijk ik terug op de stuurgroep bijeenkomsten waarin jullie vol enthousiasme alle kennis en ervaring op het gebied van onderwijs en kinderpsychiatrie deelden. Dank voor al jullie professionele en inspirerende inzichten! Ook jullie persoonlijke betrokkenheid door de jaren heen waardeer ik enorm, het zorgde er voor dat ik me helemaal thuis voelde bij SO en Z. 180

183 Dankwoord En dan mijn toppers van onderzoeksassistenten Carola en Marieke; wat een super team waren jullie! Jullie waren de drijvende kracht achter het project. Aan een half woord of zelfs een kleine blik hadden we vaak al genoeg. Carola, jouw jaloersmakende dosis aan energie en enthousiasme heeft een onvergetelijke indruk gemaakt op mij. Ik kijk met veel plezier terug op onze samenwerking en ik hoop dat onze paden elkaar nog gaan kruisen in de toekomst. Marieke, jij was een van de eerste trainers die meewerkte aan het project maar al snel bleek dat jij ook als onderzoeksassistent een hele waardevolle bijdrage kon leveren aan het project. Dank voor al het werk dat jij verzet hebt de afgelopen jaren maar vooral ook bedankt dat jij er altijd voor me was en ook nu op deze belangrijk dag naast mij wil staan als paranimf. Al mijn lieve en fijne (oud) collega onderzoekers Sanne, Shelley, Irma, Vivian, Judith, Inger, Mariëlle, Caroline, Maj, Els, Rosanne, Jasper, Eva, Chaïm en Else: dank voor al jullie input en fijne samenwerking. Naast alle leerzame momenten heb ik ook erg genoten van alle gezellige lunchwandelingen, teamuitjes, borrels en etentjes. Suzan en Susan, wat moet een promovendus zonder jullie! Voor ieder noodgeval hadden jullie altijd wel een oplossing en geen enkele vraag was teveel. Dank voor jullie zorgen en gezelligheid; jullie maken het leven van een promovendus echt een stuk gemakkelijker. Speciale dank ook voor mijn oud kamergootjes Lidewij en Esther. Ontelbare kopjes thee moeten er gespendeerd zijn aan het gezamenlijk overdenken van een juiste keuze voor analyse, heldere verwoording van resultaten maar ook persoonlijke dilemma s. Dank dat jullie er altijd voor me waren lieve dames! En dan ten slotte lieve collega onderzoeker Julia, zonder jou was het een onmogelijke opgave geweest om het proefschrift af te schrijven. Als trouwe en opmerkzame kamergenoot stond jij altijd voor me klaar, ik hoefde maar even te zuchten en je kwam al achter je bureau vandaan gerold om mij te helpen of me even mee te nemen voor koffieloopje naar de AH. Jij leerde mij niet alleen de fijne kneepjes van het vak als onderzoeker, je stond ook altijd klaar voor de noodzakelijke morele steun waardoor ook een waardevolle vriendschap is ontstaan. Ik ben bijzonder blij en dankbaar dat jij als paranimf dit proces met mij wil afsluiten. 181

184 Dankwoord Mijn (oud) collega s van de Speciale Onderwijs & Zorg poli Laura, Monique, Julie, Cathelijne, Maartje, Heleen, Manoushka, Sarah en Stephanie, heel veel dank voor jullie gezellige en inspirerende samenwerking van de afgelopen jaren! Jullie klinische inzichten waren onmisbaar voor het onderzoek maar ook jullie persoonlijke betrokkenheid maakten het altijd een feestje om naar het IJsbaanpad en later ook de Biesbosch te komen. Ik ben enorm trots en dankbaar dat ik deel uit mocht maken van jullie hechte club! Ook speciale dank aan Marjolijn Glotzbach, Diane Veugelers en vele andere collega s van de ADHD poli die ouders, scholen en kinderen enthousiast maakten om deel te nemen aan ons onderzoek. Naast alle directe collega s hebben ook andere collega onderzoekers en psychologen op uiteenlopende manieren bijgedragen aan dit proefschrift. Dank aan alle collega VU-Bascule onderzoekers voor alle inspiratievolle gezamenlijke researchseminars. Ook speciale dank aan collega UVA onderzoeker en co-auteur Sietske van Viersen voor de bijdrage aan het allerlaatste artikel, succes nog met het afronden van jouw eigen promotie! Bijzonder veel dank gaat ook uit naar alle orthopedagogen en ontwikkelingspsychologen die door weer en wind scholen in de regio Amsterdam afreisden om daar de kinderen te behandelen en hielpen met de dataverzameling. Jullie bijdrage en kritische feedback was een onmisbare factor voor dit project. Uiteraard had ook zonder de inzet van alle ouders, kinderen, scholen en leerkrachten dit proefschrift niet tot stand kunnen komen. Dank voor jullie enthousiaste deelnamen aan de projecten. De leden van de promotiecommissie, prof. dr. Pier Prins, prof. dr. Reinout Wiers, prof. dr. Arne Popma, prof. dr. Lydia Krabbendam en dr. Dorine Slaats-Willemse wil ik hartelijk danken voor het lezen en beoordelen van het manuscript. Daarnaast ook dank aan het Ministerie van Onderwijs, Cultuur & Wetenschap die de subsidie verschafte voor dit onderzoek in het kader van het programma Onderwijs Bewijs. 182

185 Dankwoord Naast alle fijne collega psychologen, - onderzoekers, (co)promotoren en bereidwillige deelnemers had ik de steun van mijn lieve familie en vrienden niet kunnen missen. Mijn lieve vriendinnetjes Elles, Kirsten, Meike, Caroline, Anouk, Loes en Yvon; jullie onuitputtelijke dosis aan warmte, humor en steun in de vorm van borrels, etentjes, (skype) gesprekken en vakanties waren onmisbare factoren voor de afgelopen jaren! Ik kijk er enorm naar uit om weer voor en met jullie te koken, samen festivals af te struinen of broodjes spé te eten op het Keizer Karel plein. Ook speciale dank voor mijn lieve (schoon)familie Manon, Harm, Bram, Maaike, Hans, Mieke en Sanne: zonder jullie onvoorwaardelijke en warme support was dit proefschrift een onmogelijke opgave geworden. Ook mijn lieve nichtjes Pleun, Fien en Tijsje verdienen een grote knuffel; jullie lieve berichtjes, kunstwerken en skype gesprekjes verrichtte altijd wonderen voor mijn humeur. Tenslotte zijn er nog twee bijzondere mensen waarvoor mijn dankbaarheid eigenlijk niet in woorden samen zijn te vatten. Lieve mam, papa en jij leerden mij altijd om op mijn gevoel te vertrouwen en dat vallen en opstaan erbij hoort in het leven: wijze levenslessen die mij ook door dit project heen gesleept hebben. Wat is het toch ontzettend fijn om altijd zo n warm en onvoorwaardelijk thuis te hebben, ik weet zeker dat pap super trots toekijkt vandaag! En dan rest alleen nog mijn allerliefste Job: zonder jou had ik de stap naar een promotietraject nooit durven maken en zonder jou was het ook niet gelukt om het af te maken. Jouw humor, warmte, nuchtere kijk op het leven en onuitputtelijke optimisme sleept mij overal doorheen. En of we nu in Nijmegen, Brabant, Amsterdam of Singapore zijn; met jou voel ik me overal thuis of liever gezegd jij bent mijn thuis. Wat of waar het ook mag zijn: ik kijk uit naar ons volgende avontuur! 183

186

187 Curriculum vitae

188 Curriculum vitae Marthe van der Donk werd geboren op 18 oktober 1986 te Oss. Ze groeide op in Schijndel en behaalde zowel haar Havo (2003) als VWO (2005) diploma aan het Elde College te Schijndel. Daarna startte zij met de opleiding Psychologie aan de Radboud Universiteit in Nijmegen. Na het schrijven van haar scriptie De invloed van borstvoeding en co-sleeping op slaap bij baby s en haar klinische stage bij de afdeling Medische Psychologie in het Canisius Wilhelmina Ziekenhuis, studeerde zij in 2010 af met een master in Ontwikkelingspsychologie. Zowel tijdens als na haar studie werkte zij mee aan verscheidene onderzoeksprojecten bij Karakter, kinder- en jeugdpsychiatrisch centrum te Nijmegen, waarbij ook haar interesse werd gewekt om een eigen promotietraject te starten. In 2011 startte zij als promovendus bij de afdeling Kinder- en Jeugdpsychiatrie van het Academisch Medisch Centrum (AMC) in Amsterdam in samenwerking met de Bascule, academisch centrum voor Kinder- en Jeugdpsychiatrie. Dit promotietraject werd begeleid door em. prof. dr. Aryan van der Leij, prof. dr. Ramón Lindauer, dr. Anne-Claire Hiemstra-Beernink en dr. Ariane Tjeenk-Kalff. Momenteel werkt zij in Sinagpore mee aan verscheidende wetenschappelijke projecten van Danone Nutricia Research. 186

189 Publications