Characterization of Executive Dysfunction in Real World Tasks: Analysis of Behaviours Performed During the Completion of the Multiple Errands Test

Transcription

1 Characterization of Executive Dysfunction in Real World Tasks: Analysis of Behaviours Performed During the Completion of the Multiple Errands Test by Sidrah Arshad A thesis submitted in conformity with the requirements for the degree of Masters of Science Graduate Department of Rehabilitation Science University of Toronto Copyright by Sidrah Arshad (2011)

2 Characterization of executive dysfunction in real world tasks: Analysis of behaviours performed during completion of the Multiple Errands Test Abstract Sidrah Arshad Degree of Masters of Science Graduate Department of Rehabilitation Science» University of Toronto 2011 This study furthers our understanding of the impact of executive dysfunction on everyday activities in stroke survivors. A classification system was developed to analyze a wide range of behaviours performed by 14 stroke survivors and 12 matched controls on the Baycrest Multiple Errands Test, a task requiring participants to buy specific items and collect certain information on the main floor of the hospital. The event recorder was used to code the occurrences and frequencies of behaviours performed by participants. Results demonstrated that participants with stroke performed significantly more task specific relevant inefficient behaviours (p <.05) and non-task specific irrelevant behaviours (p <.10) compared to controls. This study indicates the importance of performing a detailed analysis of behaviours performed to better understand the impact of ED in everyday life. ii

3 Acknowledgments I would like to wholeheartedly thank my supervisor, Dr. Deirdre Dawson without whom this thesis would not have been possible. I am especially grateful of her for having me as her student and giving me the opportunity to pursue my graduate studies, attend conferences and meetings, and co-supervise the research project of two Masters of Occupational Therapy students at the University of Toronto. Her advice, expertise, and guidance throughout my research will always be remembered and cherished. I would also like to thank my program advisory committee members, Drs. Nicole Anderson and Helene Polatajko for their valuable and helpful feedback and continued support throughout this study. I also owe my deepest gratitude to the various funding bodies who have provided me with financial support: The Heart and Stroke Foundation Centre for Stroke Recovery, Finkler Graduate Student Fellowship, Ontario Research Coalition, Jack & Rita Catherall Funds and the University of Toronto. The findings in this research would not have been possible without access to participant videos and sensitive information from an existing database, for which I am thankful to the McDonnell Foundation for their grant to Dr. D. Stuss supporting this work and to Dr. Deirdre Dawson for allowing me to work with this information extensively. I would like to thank my colleagues who have supported me along the way and to the faculty and staff of the Graduate Department of Rehabilitation Science and Baycrest. On a final note, I would like to offer my regards to my parents, sisters and brother who have greatly supported me throughout my studies, and especially to my husband Abdul Samad Ahmed, for the encouragement to pursue this degree and for his technical expertise involved in the research. iii

4 Table of Contents Abstract... ii Acknowledgments... iii Table of Contents... iv List of Tables... vi List of Appendices... vii Chapter 1 Introduction... 1 Chapter 2 Literature Review... 3 Introduction Description, Definitions, and Neuroanatomy of Executive Functions Executive Dysfunction: Prevalence and Impact on Everyday Life Theories of Executive Functions Theory of Goal Neglect Adaptive Coding Model Supervisory Attentional System (SAS) Fractionation of the Supervisory System Transcending the Default Mode Executive Knowledge (Structured Event Complexes) Assessment of Executive Functions Traditional Measurements of Executive Functions Critique of Traditional Measurements of Executive Functions: Ecological Validity Real World Measurements of Executive Functions Ecological Validity of Read-World Measurements of Executive Functions Conclusion Chapter 3 Describing the Methodology: Event Recording Background Event Recorder: Behaviour Tracker Procedure for Using Behaviour Tracker Creating the codes Structuring the Codes Coding Behaviors Reliability Other Application of the Codes and Conclusion Chapter 4 Characterization of executive dysfunction in real world tasks: Analysis of behaviours performed during completion of the Multiple Errands Test Abstract Introduction Materials and Methods Participants The BMET Coding procedure Results Behaviour classification Related results Further investigation of results 'Stroke only' vs. 'control only' behaviours iv

5 Discussion Further investigation of results 'Stroke only' vs. 'controls only' behaviours Future Directions Study limitations Conclusions Chapter 5 Discussion Theories of executive functions The importance of real world assessments and behaviour analysis Improving the BMET Limitations Future directions Summary & conclusions References Appendix A: Behaviour Tracker Modes (a) Configuration Mode (b) Record Mode (c) Editor Mode (d) Viewer Mode Appendix B: BMET Participant Package Appendix C: Meta Appendix D: Metb Appendix E: Metc Appendix F: Metd Appendix G: List of Behaviours Observed During the BMET Appendix H: Behaviour Classification v

6 List of Tables Table 2.1 Traditional Measurements of EF Table 2.2 Real World Measurements of EF Table 4.1 Participant Characteristics Table 4.2 Inter-rater Reliability Table 4.3 Behaviour Classification Table 4.4 Classification of participants' behaviours on the BMET. Differences in means, SD, range and p values between stroke-ed, stroke and control groups for each behaviour category Table 4.5 Specific behaviours findings. Differences in mean, SD, range and p values between stroke-ed, stroke and control groups on two behaviours Table 4.6 'Stroke only' vs. 'control only' behaviours vi

7 List of Appendices Appendix A: Behaviour Tracker Modes Appendix B: BMET Participant Package Appendix C: Meta Appendix D: Metb Appendix E: Metc Appendix F: Metd Appendix G: List of Behaviours Observed During the BMET Appendix H: Behaviour Classification vii

8 1 Chapter 1 Introduction The main goal of this thesis was to further our understanding of the impact of executive dysfunction on everyday performance in people following stroke. This is important because executive dysfunction affects many aspects of day-to-day activities such as preparing a meal and taking public transportation to more challenging tasks involving decision making such as managing a budget. Individuals with stroke may do relatively well on traditional neuropsychological tests, yet still manifest executive impairments (Burgess et al., 2006; Chan, Shum, Toulopoulou, & Chen, 2008; Manchester, Priestley, & Howard, 2004). This in part seems to be due to the highly structured nature of traditional neuropsychological tests. Also, since the ecological validity (the degree to which test performance reflects real world performance (Chaytor & Schmitter-Edgecombe, 2003)), of these tests is not strong, poor performance on the tests tells us relatively little about how people will perform in their daily life (Burgess et al., 2006). For this reason, the Multiple Errands Test (MET) was created. The MET involves participants performing real-life tasks such as shopping and collecting information and has good ecological validity (Dawson et al., 2009). It allows the examiners to observe participants' abilities to plan, organize and perform tasks in an efficient manner. Examining the underlying real world behaviours that are particularly problematic for people with executive impairments as they attempt to achieve tasks on the MET will provide a better understanding of the impact of executive dysfunctions on everyday activities. In turn, this will enable clinicians to work more effectively with people with executive dysfunction and implement effective interventions, which will enhance their overall quality of life. This thesis is organized in a manuscript format. Chapter 2 provides the background literature to understand executive functioning in the context of everyday behaviours. Relevant theories of executive functions, which help explain these processes, are discussed. A large section of the chapter is dedicated to the assessments of executive function, including traditional and naturalistic, real world measures. These are reviewed on the basis of the processes hypothesized to be measured by the test and the ecological validity of each is also discussed.

9 2 Chapter 3 details the methodology used in the study. A description of how the software Behaviour Tracker was used to more comprehensively document observable behaviours during the MET is provided. This approach to analyzing behaviours in the MET is unique and Chapter 3 details the entire process. Chapter 4 is written in a format to be submitted for publication to Neurorehabilitation and Neural Repair. The chapter consists of the overall research component of the thesis and its findings. The final chapter (Chapter 5) is a general discussion that brings together the entire thesis and integrates the findings reported in Chapter 4 with the background literature provided in Chapter 2.

10 3 Chapter 2 Literature Review Introduction This chapter reviews the literature regarding executive dysfunction within the stroke population. A review of the literature was performed using PubMed and PsycINFO between 1990 and 2011 with keywords related to executive function. These consisted of executive function, executive dysfunction, neuropsychological test, cognition, prefrontal cortex, activities of daily living (ADLs), assessment, stroke, and acquired brain injury. This chapter reviews the definitions of executive functions and the relevant theories of executive function. A real world environment is necessary when assessing executive function. Hence, ecological validity of both traditional and real world measurements of executive functions are also discussed. 2.1 Description, Definitions, and Neuroanatomy of Executive Functions Executive functions (EF) are those higher-order cognitive processes that enable problem solving, information processing, decision making and the formulation of goals in daily life (Jurado & Rosselli, 2007; Levine, Turner, & Stuss, 2008). In general, the term EF refers to a group of cognitive abilities which include: attention, planning, reasoning, monitoring, complex problem solving, verbal reasoning, decision making, inhibition of irrelevant information, social functioning and the ability to deal with novel situations (Alexander & Stuss, 2003; Bryan & Luszcz, 2000; Keil & Kaszniak, 2002; Levine et al., 2008). Together, these skills allow one to plan, initiate and complete an action, while monitoring one s own behaviour and the environment. Most theorists emphasize the distinction between routine and non-routine behaviours when attempting to describe the role of EF (Gilbert & Burgess, 2008). Routine processing refers to automatic, well rehearsed operations that an individual is able to use in familiar situations (Gilbert & Burgess, 2008), while non-routine processing is important in novel and varying situations requiring sustained attention (e.g. distractions) or where stimulus-response relationships are unclear (Gilbert & Burgess, 2008). The term EF has become synonymous with non-routine processing (Gilbert & Burgess, 2008) and EF are considered to exert higher-level

11 4 modulation on lower-level routine processing in different situations. This allows an individual to behave flexibly while taking into account environmental information and past consequences in familiar and unfamiliar situations. Behavioural and neuroimaging studies support the notion that EF are governed, at least in part, by different structures in the frontal lobes (Gilbert & Burgess, 2008). This leads to a common problem where many researchers use the term frontal functions when describing EF even if they are not examining the anatomy in particular (Stuss & Alexander, 2000). In addition, terms such as dysexecutive syndrome (Stuss, 2007) and dysexecutive control (Stuss & Alexander, 2000) are also used interchangeably. However, this thesis will only use the term EF as described by Stuss (2006) when discussing these high-order processes. There is considerable debate over whether EF are discrete, fractionable processes governed by different regions in the frontal lobes or if they are a single, unitary function (Jurado & Rosselli, 2007; Stuss & Alexander, 2000). Stuss (2007) argues in favour of the former based on empirical evidence and divides EF into four different functional categories which are characterized by a grouping of similar behaviours and anatomical structure (Cicerone, Levin, Malec, Stuss, & Whyte, 2006). These include: (1) executive cognitive functions, (2) behavioural-emotional selfregulatory functions, (3) energization regulating functions, and (4) metacognitive processes (Stuss, 2007). Others try to define executive functioning from a unitary perspective in which all areas of executive functioning can be explained by one single underlying ability such as behavioural inhibition (Barkley, 1997; Duncan & Miller, 2002; Jurado & Rosselli, 2007). 2.2 Executive Dysfunction: Prevalence and Impact on Everyday Life Executive dysfunction is one of the most critical and prevalent problems in the acquired brain injury (ABI) population, which include people with stroke, traumatic brain injury (TBI) and other forms of brain injury (Levine et al., 2008). Executive dysfunction can also be observed in people with frontal dementia, multiple sclerosis, Alzheimer s disease, Parkinson s disease, Huntington s disease and various other psychiatric disorders including schizophrenia and depression (Alexander & Stuss, 2003; Levine et al., 2008; Stuss, 2007). Similarly, conditions such as depression, anxiety and sleep deprivation, which all interact with the frontal lobes, can also lead to deficits in executive functioning (Alexander & Stuss, 2003).

12 5 Executive dysfunction can have a devastating effect on everyday living, including the ability to perform daily tasks at home and at work, and to maintain an independent life (Godbout, Grenier, Braun, & Gagnon, 2005; Knight, Alderman, & Burgess, 2002). People with stroke and TBI have a difficult time regulating their behaviours in response to the changes occurring in their environments (Minassian, Perry, Carlson, Pelham, & DeFilippis, 2003). Some of the common cognitive deficits associated with executive dysfunction include: lack of judgment, inability to concentrate, and difficulties in organization and intellectual abilities (Alexander & Stuss, 2003; Gillen, 2008). Specific behavioural problems include lack of flexibility, impulsivity, distractibility and poor self-control (Alexander & Stuss, 2003). Some patients suffer from an inability to self-monitor their behaviours despite appropriate feedback (Minassian et al., 2003), while others have difficulties in sustaining and reinitiating past behaviours (Lezak, 2004). Most executive impairments are mediated by the frontal lobes and are regulated by the lateral prefrontal areas (Alexander & Stuss, 2003; Stuss, 2007). However, diffuse damage to posterior regions and subcortical structures connected to the frontal cortex can cause similar deficits (Elliot, 2003; Levine et al., 2008). Similarly, damage to cortical connections to basal ganglia, cerebellar and thalamic areas may also result in executive impairments (Alexander & Stuss, 2003). Further, it is important to note that executive impairments do not only occur as a result of extensive cortical damage and may even occur in the absence of cortical lesions due to the nature of connectivity between frontal lobes and other regions in the brain (Alexander & Stuss, 2003). 2.3 Theories of Executive Functions Theories related to EF provide a better understanding of the underlying systems involved in executive functioning and offer guidance on the appropriate assessments that can be used with individuals with executive dysfunction. Various theories have been proposed in order to better understand EF. Each of them is useful, however none alone entirely explain these higher-order cognitive processes. Moreover, although each of the theories is different from the other, they all deal with the frontal lobes, especially the prefrontal cortex (Turner & Levine, 2004). Further, all of the theories have been proposed following studies of adults with frontal damage (Chan, Shum, Toulopoulou, & Chen, 2008; Turner & Levine, 2004). The following section will provide a review of some of the more widely employed and discussed theories of EF.

13 Theory of Goal Neglect Duncan and colleagues emphasize the importance of goals in human behaviour (Duncan, 1986, 1995; Duncan & Owen, 2000; Duncan et al., 2000). They note that human behaviour is goaloriented and is directed by goal lists or sub-goals that are created in response to environmental and internal demands (Burgess & Robertson, 2002). Goal lists are formulated and stored in mind by individuals so they can behave optimally in response to external stimuli (Chan et al., 2008). It is assumed that actions are directed by goal lists, which are a sequence of task requirements that must be accomplished in order to achieve a desired goal (Turner & Levine, 2004). These lists provide a road map of what actions and operations are needed when a current situation does not match the goal state. In such situations, these goal lists are pursued and a series of actions are activated to resolve the discrepancy. When goals are changed or new goals are set, a new goal list is selected and specific actions are carried out that will lead towards goal attainment (Chan et al., 2008). An important characteristic of goal-directed behaviour is that it seeks new actions to achieve task completion if previously elected actions have failed (Burgess & Robertson, 2002). According to Duncan, Emslie, Williams, Johnson and Freer (1996), patients with damage to the frontal lobe are usually disorganized and this can be illustrated by the fact that these patients are unable to construct and use goals and goal lists in an effective manner. Duncan and associates (1996) use the term goal neglect when describing this phenomenon in patients with damage to the frontal lobe. These patients are able to understand and remember their goals but they seem to have difficulty in maintaining the goals in working memory such that their actions become random and they exhibit neglect of the intended goal. Duncan and colleagues (1996) found goal neglect to be common in patients with frontal lesions in situations that were novel and those that had multiple simultaneous requirements. Duncan's theory of goal neglect emphasizes the importance of the prefrontal cortex (PFC) in goal formulation, goal selection and goal monitoring (Turner & Levine, 2004). Duncan (1986) argues that this is because the main function of the PFC is to organize and control actions in accordance with desired goals. He also argues that it is common to observe a discrepancy between desired goals and actions following frontal lobe lesions (Duncan, 1986). Duncan proposes that the PFC has neural flexibility, which allows it to be recruited for different tasks and enables global

14 7 functioning of the PFC. However, the specific neurons, regions and structures of the PFC that explain these processes have not been defined Adaptive Coding Model The underlying idea behind the adaptive coding model (Duncan, 2001; Duncan and Miller, 2002) is that since frontal regions are involved in many cognitive tasks, it is important to understand the PFC as a global, adaptive coder. It suggested that the PFC controls many processes ranging from working memory and attention to cognitive control. Duncan and Miller (2002) argued that the neurons of the PFC are highly adaptable in nature and can meet various task demands. They also argue that the PFC has a role in selecting and integrating information relevant to the current situation. Hence, the frontal lobes are able to take on actions as required, without having specific regions mediate specific cognitive demands. This adaptive ability of the frontal lobes has been demonstrated by functional imaging research. Duncan & Owen (2000) compared the areas of the PFC involved in five different executive cognitive demands. They found similar activation patterns regardless of the cognitive demands studied, as well as overlap of the areas activated for each of the executive cognitive demands (Duncan & Owen, 2000). In contrast, Cabeza and Nyberg (2000) demonstrated different regions in the PFC to be related to different functions such as attention, working memory and language. Also, Stuss (2006) questions the global, adaptive ability of the PFC as the theory is unable to explain how the PFC regions select and abandon information. The theory also does not address how the PFC integrates sensory, memory and task-relevant information and coordinates subcortical and other regions (Wood and Grafman, 2003) Supervisory Attentional System (SAS) The SAS theory, developed by Norman and Shallice (1986) is one of the most influential theories of frontal lobe functions. Norman and Shallice posited that the SAS is comprised of two systems: contention scheduling and the supervisory system. These systems are important in initiating, regulating and monitoring human actions and behaviours. The role of contention scheduling is to control routine motor behaviours and cognitive operations in familiar situations and prioritize the sequence of these behaviours (Burgess & Robertson, 2002; Chan et al., 2008; Turner & Levine, 2004). The supervisory system is important in regulating non-routine and

15 8 novel behaviours where planning and decision making are involved (Chan et al., 2008). The supervisory system is thought to be located in the PFC although it is unclear which regions in the PFC are involved in the processes controlled by the supervisory system (Wood & Grafman, 2003). The location of the contention scheduler is also undefined (Wood & Grafman, 2003) though Normal and Shallice (1986) argue that the contention scheduler may be located in the basal ganglia and premotor cortex since both of these areas function as output targets for the PFC. That being said, the main difference between these two systems is in their ability to modulate routine and non-routine behaviours. Research suggests that the PFC is indeed much more implicated in non-routine than routine behaviours (Shallice, 2002). However, Wood & Grafman (2003) argue that neuropsychological research has demonstrated that knowledge about routine behaviour is also impaired following lesions in the PFC (Allain, Le Gall, Etcharry-Brouyx, Aubin, and Emilie, 1999; Sirigu et al., 1996). They also argue that novel tasks activate anterior PFC, while familiar tasks activate medial and posterior regions of the PFC (Koechilin, Corrado, Pietrini, & Grafman, 2000). In addition, they note that it is unclear how the model represents the integration of sensory input with memory information and the neuropsychological properties of the PFC neurons (Wood & Grafman, 2003). Regardless of the lack of agreement in relation to the specific neuroanatomical regions involved in the two systems, the SAS theory puts forth the concept of multitasking performance in daily life (Burgess, 2000; Burgess, Veitch, Costello, & Shallice, 2000) and the theory posits that an individual may have limited ability to respond to multiple stimuli Fractionation of the Supervisory System Stuss, Shallice, Alexander and Picton (1995) wanted to investigate whether there are different areas in the PFC that mediate different cognitive processes. They used Norman and Shallice s (1986) supervisory system as a starting point to examine if it could be further fractionated and defined (Stuss et al., 1995; Stuss & Alexander, 2007). They recruited patients with single focal frontal lesions to determine whether specific regions were important for different, specific cognitive processes. The results led Stuss and his colleagues to classify four different properties of PFC function: energization, behavioural self-regulation, metacognition, and EF (which include task setting and monitoring) (Stuss & Alexander, 2007). Energization is the process that

16 9 allows an individual to initiate, concentrate and maintain a response, and lesions in the superior medial regions of the PFC have shown to cause impairments in this ability (Stuss, 2006; Stuss & Alexander, 2007). Damage to the left dorsolateral PFC has been associated with problems in task setting, which refers to the ability to establish a stimulus-response relationship, while right lateral lesions are related to impairments in monitoring, which is the ability to ensure quality by checking performance and modifying actions and behaviours (Stuss, 2006; Stuss & Alexander, 2007). It is important to note that although fractionation has been demonstrated among these processes, this does not imply that they are independent. Instead, these processes work together with other networks of the frontal and posterior regions in response to the complexity and duration of the context (Stuss & Alexander, 2007; Vuilleumier & Driver, 2007). Specifically, these processes work independently in simple automated tasks, but with increased task demands, different frontal regions are involved to the point where all frontal regions may be employed (Stuss & Alexander, 2007). In contrast to Duncan s adaptive coding model where the PFC has unified control of the EF with little specificity, Stuss and colleagues' investigation suggests fractionated supervisory executive control Transcending the Default Mode Mesulam (2002) emphasizes a central role of the PFC in conquering the hypothetical default mode, which is a state directed by inflexible stimulus-response relationships and is unresponsive to context and experience. In the default mode, actions are triggered via automatic reactions and fulfill immediate satisfaction (Mesulam, 2002). Default actions are carried out automatically without consideration of alternative responses and are hard-wired in nature (Mesulam, 2002). In this state, the conscious is focused on here-and-now despite contextual feedback (Mesulam, 2002; Turner & Levine, 2004). Mesulam (2002) explains that frontal lobe damage results in the implementation of the default mode, however the main role of the PFC and EF is to suppress and minimize the effect of this mode. This is carried out via five executive processes: (1) working memory, which is the ability to actively hold and manipulate relevant information, (2) inhibition of distractibility, which is the ability to ignore and suppress distractor stimuli or events, (3) novelty seeking, which is the tendency to pursue novel and uncertain situations, (4) conditional mapping of emotional

17 10 significance, which is the ability to integrate emotion with action and experience, and, (5) encoding of context and perspective, which refers to the ability to process the influence of the environment, background and feedback from others (Mesulam, 2002). With the help of these executive processes, the PFC is able to overcome the default mode to allow more context-driven responses to occur (Mesulam, 2002; Turner & Levine, 2004) Executive Knowledge (Structured Event Complexes) Grafman (1995) explains the function of the PFC using neural representations of knowledge. According to this framework, higher-order cognition is controlled by knowledge domains, which are organized in a hierarchical manner in the PFC (Levine & Turner, 2004). A knowledge unit refers to a set of events or actions that are linked sequentially (Grafmam, 1995; Turner & Levine, 2004). A series of knowledge units linked together temporally are structured event complexes (SECs), which are encoded in sequence and represent morals, social customs, beliefs, event themes and other related features (Turner & Levine, 2004; Wood & Grafman, 2003). These SECs are stored independently and are retrieved in an episodic form. The SECs come into action when an individual needs to perform a goal and remain activated until the goal is achieved. This activation pattern of the SECs is consistent with the firing patterns of the PFC neurons (Wood & Grafman, 2003). This framework predicts that different aspects of SECs are stored in different regions of the PFC. For instance, nonsocial features would be stored in the dorsolateral PFC whereas social aspects of the event would be stored in the ventromedial PFC and would have strong connections with posterior regions. This notion is also supported by research suggesting that damage to the ventromedial PFC leads to impairments in social behaviour, while dorsolateral PFC damage leads to problems in reflective behaviour (Grafman et al., 1996). The framework predicts that damage to the PFC would lead to partial or incomplete retrieval of SECs which would in turn lead to observable impairments in everyday life. Wood and Grafman (2003) are critical of the theories mentioned above because they assume that these theories take a processing approach, which entails that cognition in the PFC can be explained on the basis of performance, without specifying how the information is represented in the PFC in the first place. In contrast, their approach, which is representational in nature, is more concerned with the form in which information is stored in the brain and the localization of

18 11 different characteristics of various stimuli. Although this framework is able to explain the what in terms of neurophysiology, organization and connectivity patterns of the PFC, it fails to explain how these processes are governed by the PFC (Turner & Levine, 2004). Each of the theories and models mentioned above are valuable and it is important to understand them before moving on to the next section on the assessments of EF, however it is interesting to note that many of the traditional measures were developed before any of the theories were created and some of the recent measures may also be lacking in a theoretical framework. 2.4 Assessment of Executive Functions The EF have conventionally been evaluated using traditional neuropsychological measurements. Although, neuropsychological assessment of executive functioning is critical (Marcotte, Scott, Kamat, & Heaton, 2010), it is also essential to understand the everyday impact executive dysfunction can have on an individual (Manchester, Priestley, & Howard, 2004). As a result, a number of researchers have developed real-life assessments and performance-based measurements which resemble the challenges and situations faced by people in everyday life. This section provides an overview of both traditional and naturalistic assessments of EF. One of the foci is how these measures describe and assess elements of performance Traditional Measurements of Executive Functions In this thesis, the term traditional measures of EF refers to the routine pencil-and/or-paper tests, such as the Wisconsin Card Sorting Test, that are typically administered in a clinical or laboratory-type setting. According to Hughes and Graham (2002), one of the most common problems with traditional measures of EF is the difficulty in differentiating between automatic and controlled actions. When an individual attempts a novel task, the performance slowly changes from being controlled to being automatic as the person grasps the task and draws on past experience to complete it. However, because traditional measures are usually structured and defined in nature, it is difficult to differentiate when the performance becomes automatic. A related problem with traditional measures is that of novelty of stimuli (Jurado & Rosselli, 2007). Most traditional measures of EF assess the ability to deal with new problems. However, the problems are no longer novel after the first administration of the test and this may result in a practice effect (Salthouse, Atkinson, & Berish, 2003). Nonetheless, many measures of EF are

19 12 traditional neuropsychological tests, which are highly structured in nature and are used because they are thought to be sensitive to frontal lobe damage (Miyake, Friedman, Emerson, Witzki, & Howerter, 2000). Three of the most commonly used are the Wisconsin Card Sorting Test (WCST), the Trail Making Test (TMT), and the Stroop Color and Word Test (Stroop) (Rabin, Barr, & Burton, 2005). The following section provides a brief description of these tests and the processes hypothesized to be measured. The psychometrics of all of these tests are wellestablished (Strauss, Sherman, & Spreen, 2006). This is followed by a critique of the traditional measures of EF on the basis of ecological validity. Table 2.1 Traditional Measurements of EF The Wisconsin Card Sorting Test The Trail Making Test The Stroop The Wisconsin Card Sorting Test (WCST) The WCST is one of the most popular frontal lobe tests of executive functioning. In this test, the participants are presented with four stimulus cards, the first has a red triangle, the second has two green stars, the third has three yellow crosses and the fourth has four blue circles on them (Strauss et al., 2006). The participants are then given 64 or 128 cards one by one, each of which has one to four coloured shapes on it. Each card has designs similar to the stimulus cards varying in terms of shape itself, the color of the shapes and the number of shapes depicted. The cards are shown in a set order that is unknown by the participant (Keil & Kaszniak, 2002). Participants are asked to match the cards to one of the four stimulus cards and determine the rule underlying the pattern without any help from the examiner or the environment (Bryan & Luszcz, 2000). When the participants match a card, they are informed whether it is right or wrong. However, they are not informed when the examiner changes the underlying pattern. The participants task is to use the information provided as feedback to correctly identify the underlying pattern for as many of the cards as possible. This requires the participants to form a cognitive set (Minassian et al., 2003), and to be flexible and adjust to the changing patterns, and use previous patterns to guide future responses. Performance on the WCST is scored in a number of ways, however executive control is assessed by the number of categories completed, which is the number of sequences of 10 consecutive correct patterns achieved, and the number of perseverative errors performed (Strauss et al.,

20 ). Perseverative errors are those that occur when participants continue to sort the cards according to the principle of a previously incorrect sort (Bryan & Luszcz, 2000) and do not change their responses even when the responses continue to be incorrect. These perseverative errors allow the examiner to measure the patients ability to monitor and pay close attention to the feedback provided by the examiner. The WCST has been validated as a measure of several components of EF (Keil & Kaszniak, 2002). However, its functional specificity may not be too strong since other processes need to be intact in order for successful performance to occur on the WCST (Keil & Kaszniak, 2002). For instance, the tasks on the WCST require numerous non-executive cognitive processes such as basic visual processing, numerical ability, rule induction ability, speed processing, and any significant deficit in one or more of these processes may affect WCST performance (Strauss et al., 2006). This suggests that impaired task performance does not necessarily imply executive dysfunction. In addition, many researchers and compendia of neuropsychological assessments note that the WCST cannot be used on its own as a predictor of frontal focal lesions (Anderson, Bigler, & Blatter, 1995; Demakis, 2003; Henry & Crawford, 2004a; Lezak, Howieson, & Loring, 2004; Strauss et al., 2006) since impaired performance on this test can be due to a variety of reasons (Stuss et al., 2000). Moreover, different EF contribute differently to a variety of complex executive tasks and simply relying on the WCST as a measure of executive functioning is not adequate (Miyake, Emerson, & Friedman, 2000; Miyake et al., 2000). The WCST appears to be sensitive to frontal damage (Keil & Kaszniak, 2002), however, in a recent review Nyhus and Barceló (2009) revealed that many researchers have demonstrated that damage to other regions such as temporal (Barceló, Escera, Corral, & Periañez, 2006; Giovagnoli, 2001), parietal (Gonzalez-Hernandez et al., 2002; 2003; Lie, Specht, Marshall, & Fink, 2006; Rogers, Andrews, Grasby, Brooks, & Robbins, 2000), subcortical (Monchi, Petrides, Petre, Worsley, & Dagher, 2001; Mukhopadhyay et al., 2008; Rogers et al., 2000) as well as hippocampal regions (Giovagnoli, 2001; Igararshi et al., 2002; Nagahma et al., 1997) may affect WCST performance. Consequently, it should be viewed as a measure that requires a large distributed neural network (Stuss et al., 2000). It is interesting to note that when the WCST was developed by Esta Berg in 1948, only healthy participants were used, however it has now become the leading sorting task associated with frontal lobe damage.

21 The Trails Making Test (TMT) The TMT is another popular and frequently used neuropsychological assessment. It consists of two parts. In Part A, the participants are required to sequentially connect the numbers 1-25 that are placed randomly on a piece of paper (Jurado & Rosselli, 2007; Strauss et al., 2006). In Part B, participants are required to alternate between numbers and letters, again working sequentially (1 to A to 2 to B to 3 to C and so on) in order to create another trail (Jurado & Rosselli, 2007). The time it takes to complete each part is recorded which includes time spent by the examiner indicating any errors made by the participants and the time it takes participants to correct these errors. Total time to complete Part B is considered to be a measure of executive functioning (Chaytor, Schmitter-Edgecombe, & Burr, 2006). This is because the tasks are thought to require specific cognitive abilities such as speed of processing, monitoring, inhibition and scanning abilities (Chaytor et al., 2006; Jurado & Rosselli, 2007; Strauss et al., 2006). Part A is presumed to measure visual search and motor skills, while Part B appears to measure higher level cognitive abilities such as mental flexibility and divided attention (Bowie & Harvey, 2006; Strauss et al., 2006). Some disagree with the notion that the TMT, in particular the nature of switching of Part B to be a sensitive measure of frontal lobe dysfunction (Stuss et al., 2002). Furthermore, both parts A and B of the TMT measure different constructs of executive functioning, one researcher may use Part B as an assessment of mental flexibility while another might categorize the test as a measure of attention and may use it only to test for perceptual-motor speed (Chaytor & Schmitter- Edgecombe, 2003). In addition, Lezak et al. (2004) noted the ambiguity in recording the time taken by the examiner to point out mistakes and speed taken by the participant in correcting them, which may lead to reduced reliability of the test. Strauss and colleagues (2006) mention that errors on the TMT tend to be fairly uncommon in individuals with moderate to severe head injury, which further questions the reliability of error scores. On a related note, practice effects are also present over short retest intervals on the TMT (Strauss et al., 2006), which may hinder its use if the examiner wants to repeat the assessment The Stroop Colour-Word Interference Test The Stroop is another measure used to assess EF because of its sensitivity to determine proneness to interference (Bryan & Luszcz, 2000). The test consists of three trials, each with a

22 15 time limit of 45 seconds (Strauss et al., 2006). In the first trial, participants are presented with a Word Page consisting of a list of 100 color words (red, green, blue) written in black ink. The participants are asked to read the color words (Chaytor et al., 2006; Strauss et al., 2006). In the second trial, a Color Page with 100 Xs in either red, green or blue ink are given and participants are asked to read the color of the word X is written in (Chaytor et al., 2006; Strauss et al., 2006). The third trial includes a Color-Word Page with 100 words from the first trial written in colors from the second trial (e.g., word 'red' is printed in green ink) and the participants are asked to name the color the word is written in (Chaytor et al., 2006; Strauss et al., 2006). The test produces three scores: (1) word-reading score from the first trial, (2) color-naming score from the second trial and (3) color-word score from the third trial (Strauss et al., 2006). The interference score is also determined for the third trial (Strauss et al., 2006). The examiner measures interference by recording the time it takes the participant to read the color-word minus one of the other scores from the other trials (Bryan & Luszcz, 2000). This interference score indicates the time it takes to suppress or inhibit reading a word and the time it takes to name a color, which is known as the Stroop effect (Strauss et al., 2006). This is considered a measure of executive functioning for the Stroop. Inhibition, concentration, selective attention and cognitive flexibility are other executive processes thought to be tapped by the Stroop (Bryan & Luszcz, 2000; Jurado & Rosselli, 2007; Strauss et al., 2006). Participants are slower at reading the colorword in the third trial because of the inability to ignore and inhibit a habitual response of reading the word rather than the name of the color used to write it. Strauss and colleagues (2006) note that since the three trials in the Stroop are organized in a congruent manner, it may reduce the involvement of working memory and allow the participants to employ one strategy for the entire trial. This may make it easier for the participants to keep one goal in mind as task demands stay the same in each trial. Another difficulty associated with the Stroop is the fact that alternate versions of this test do not reveal the same Stroop effect (Shilling, Chetwynd, & Rabbitt, 2002).This would make it hard for clinicians to compare results if they had been using different versions of the test. In addition, although the Stroop appears to be sensitive to the frontal lobe damage, many advise that other neural systems are involved during task performance and it is important to use multiple assessments of EF as the Stroop only taps into certain aspects of executive abilities (Boone et al., 1998; Keil & Kaszniak, 2002; Pineda & Merchan, 2003). Furthermore, the interference score, which is determined using the third trial,

23 16 yields only marginal/acceptable reliability, hence this score should be supplemented with other data (Strauss et al., 2006). These commonly used traditional measures are extremely vital because their contribution to research is enormous. However, it is important to assess their ecological validity as outlined in the next section. Critique of Traditional Measurements of Executive Functions: Ecological Validity Ecological validity refers to whether the findings obtained in a testing environment can be generalized to those occurring in a natural, real world setting such as home, work and the community (Chaytor & Schmitter-Edgecombe, 2003). In other words, it refers to whether performance on the test is related to performance in daily life (Dawson et al., 2009). Burgess and colleagues (2006) presented a critique of traditional tests of EF and highlighted many of the key points associated with the lack of ecological validity in these tests. First of all, they adopted Kvavilashvili and Ellis s (2004) definition of ecological validity which refers to both the representativeness of the test to a situation encountered outside of the testing arena and the generalisability of test results to predict similar problems in related circumstances of everyday life. They noted that some of the traditional measures were never created to test for significant cognitive deficits and were a result of a variety of experimental investigations (Burgess et al., 2006). As a result, performance on these tests tells us very little about how people deal with the same cognitive deficits in their everyday life. Another key argument presented by Burgess and associates (2006) is that the methods in which traditional tests are implemented have very little in common with situations encountered in everyday life. This poses a challenge to both the idea of representativeness and generalisability of the test and its results as it is not clear if what is being measured is related to performance in the real world and whether the results obtained can explain performance in other situations (Burgess, Alderman, Evans, Emslie, & Wilson, 1998; Chan et al., 2008; Manchester et al., 2004). Since the ecological validity of these tests is not strong, poor performance on these tests does not predict how people will perform in their daily life. For example, the situation encountered in the WCST of sorting cards and others are unlike everyday circumstances and are rarely if ever applied in real world tasks (Burgess et al., 2006).

24 17 The third argument is that traditional tests such as the WCST and Stroop are basic, short and simplified tasks (Burgess et al., 2006) that are to be completed one at a time. These tasks are also highly structured and defined in nature. As a result, they do not resemble the spontaneous and uncontrolled nature of real-life situations. The nature of the test environment also has an impact on the participant s performance. In many assessments, the environment is kept quiet with no distractions if possible and the examiner controls the initiation and completion of tasks (Chaytor & Schmitter-Edgecombe, 2003). In addition, it is the examiner that provides the instructions to complete the tasks and is usually supportive regardless of the outcome of the test (Chaytor & Schmitter-Edgecombe, 2003). These types of restrictions are highly unlikely to occur in a real world environment where situations are distracting and hardly encouraging. Therefore, an environment consisting of these restrictions will tell us little about participants performance in an everyday situation (Chaytor & Schmitter-Edgecombe, 2003). Hence, there are a whole multitude of executive processes that these tests are unable to measure. Kingstone and colleagues (2005) whose research is grounded in the field of attention take this a step further and argue that there are two main assumptions with traditional laboratory-based research which present a great problem in the clinical world. The first is that it is assumed that the processes and conditions in the lab are similar if not the same as the ones occurring in the real world. This is comparable to what Burgess and associates (2006) have argued which highlights the lack of representativeness of traditional measures. The second assumption is that to attain maximum results, one should minimize variability in a situation. However, there are many variables occurring in naturalistic situations and it is important to take into account all of them in order to understand and make inferences. Chan and associates (2008) expand further to identify the reason behind the weak generalisability of traditional tests to everyday circumstances. Since these assessments were developed for basic experimental brain research (Burgess et al., 2006), they are only successful in measuring at the impairment level (e.g. problems in attention) and fail to grasp the complicated nature of responses that are required to carry out the many multistep tasks in daily life (Chan et al., 2008; Lewis, Babbage, & Leathem, 2011). This may explain the reason why many patients with ABI are able to do relatively well on these traditional neuropsychological tests but still manifest executive impairments (Stuss, Floden, Alexander, Levine, & Katz, 2001).

25 18 Spooner and Pachana (2006) also note that there is a general problem in ecological validity research in which certain traditional tests are believed to measure specific constructs such as working memory even if they do not do so. For instance, the varying ways of defining executive functioning (see section 2.1 in this chapter) make it harder to categorize certain neuropsychological tests (Chaytor & Schmitter-Edgecombe, 2003). This also leads to difficulty in selecting an appropriate test to measure a specific construct. For example, certain tests are labelled with a specific construct that they measure only on the basis of face validity and may have not been tested using construct validity. Even if the test is measuring the specified construct, this may not be enough to explain the clinical findings at the individual level (Burgess et al., 2006) because these inferences are based on theoretical constructs and are unable to explain failure in performance. Ecological validity is important when studying real world behaviours and traditional measures are lacking in this. As a result, it is essential to take a closer look at the naturalistic, real world assessments in the next section of the chapter Real World Measurements of Executive Functions As mentioned above, traditional neuropsychological tests can measure isolated cognitive and executive processes but are less effective in predicting everyday life performance following executive dysfunction (Burgess et al., 2006). Daily life performance often requires multitasking and the ability to generate strategies to deal with novel situations (Manchester et al., 2004). For this reason, many have suggested the need for testing in real world environments using naturalistic, real world assessments (Burgess et al., 2006, Godbout et al., 2005; Goverover et al., 2005). This section of the chapter provides a review of the naturalistic assessments and questionnaires published in peer-reviewed journals, which are specific to EF and have been validated against other neuropsychological measures of EF. Specific keywords such as: naturalistic and real world were used in order to find these measures. The questionnaires included measure the impact of executive dysfunction on leading an independent life, while the performance-based assessments measure one or more everyday activities such as cooking or shopping that require executive abilities. The psychometrics of these measures have not been reviewed, rather the following section discusses the executive processes hypothesized to be tapped. Assessments that only

26 19 measure nonexecutive components like the State-Trait Anxiety Inventory (Spielberger, Gorsuch, & Lushene, 1970), and those that use virtual reality or simulation such as the Hotel Test (Manly, Hawkins, Evans, Woldt, & Robertson, 2002), Executive Secretarial Task (Lamberts, Evans, & Spikman, 2010), Six Elements Test (Shallice & Burgess, 1991), Behavioural Assessment of the Dysexecutive Syndrome (BADS) (Wilson, Alderman, Burgess, Emslie. & Evans, 1996) and Iowa Gambling Task (Bechara, Damásio, Damásio, & Anderson, 1994) are excluded. The review of the assessments is followed by an overview of the ecological validity of real world measures. Table 2.2 Real World Measurements of EF Questionnaires Performance-based Assessments Questionnaires The Behaviour Rating Inventory of Executive Function - Adult Version The Dysexecutive Questionnaire The Profiles of the Executive Control System The Kitchen Task Assessment The Rabideau Kitchen Evaluation-Revised The Cooking Task The Executive Function Performance Test The Executive Function Route-Finding Task The Instrumental Activities of Daily Living Profile The Multiple Errands Test Behaviour Rating Inventory of Executive Function - Adult Version (BRIEF-A) The BRIEF-A, a 75-item questionnaire for adults based on the original BRIEF developed for children and adolescents, measures EF in daily life over the previous month (Roth, Isquith, & Gioia, 2005). The BRIEF-A includes two versions: self-report and an informant report (Roth et al., 2005) in which each item is scored on a scale of 1 to 5 and higher scores reflect greater difficulty experienced by the patient. The informant report can be used by itself if the patient is unable to complete the self-report. The BRIEF-A yields an overall score called the global executive composite, which consists of two separate indexes called the Behavioural Regulation Index (BRI) and Metacognition Index (MI). The BRI is composed of four scales: (1) Inhibit, which refers to the ability to resist an impulse, (2) Shift, which refers to the ability to be flexible and switch attention, (3) Emotional Control, refers to modulation of emotional responses and (4) Self-monitor, which refers to checking one s own actions during and after goal attainment. The MI Index is comprised of five scales: (1) Initiate, which refers to the ability to begin the task, (2) Working Memory, which refers to the process of maintaining relevant information in mind in

27 20 order to complete a task, (3) Plan, refers to the ability to set goals and develop a strategy to complete the task, (4) Task Monitor, which refers to checking and ensuring the task is performed in an organized manner and leads to completion, and (5) Organization of Materials, refers to the ability to establish and maintain all the necessary materials required to achieve the task (Roth et al., 2005). The scores are calculated for each scale, indices and for the global summary composite. T-scores are based on comparisons to a normative sample composed of 1050 self reports and 1200 informant reports.(roth et al., 2005). Rabin and colleagues (2006) compared the BRIEF-A with various neuropsychological tests of EF in patients with amnestic mild cognitive impairment and significant cognitive complaints. They found a moderate inverse correlation between the selfreport BRI and the Weschler Memory Scale-III (WMS-III) Visual Reproduction II (Rabin et al., 2006). They also found a significant correlation between the adjusted Geriatric Depression Scale and both the MI and BRI for self-report version only (Rabin et al., 2006). However they failed to report any strong relations with neuropsychological measures of EF. Rabin et al. (2006) suggested that this finding may be due to the fact that the BRIEF-A is measuring different aspects of EF than those tapped by performance-based neuropsychological tests. In contrast, Garlinghouse, Roth, Isquith, Flashman and Saykin (2010) compared the subjective, self-report of working memory scale of BRIEF-A in patients with schizophrenia with the Digit Span subtest of the Wechsler Adult Intelligence Scale-III. They found that the patient group reported poorer subjective working memory and performed worse on the Digit Span Backwards than the comparison group (Garlinghouse et al., 2010). The authors, however, emphasize the differences between the working memory demands being measured by the BRIEF-A and the Digit Span. The BRIEF-A asks whether the patient had problems with different behaviours that require working memory over the past month, while the Digit Span measures the ability to maintain and manipulate digits in working memory (Garlinghouse et al., 2010). In addition, the difference between the subjective nature of the self-report versus the objective features of the Digit Span may also have an impact on the degree of correlation Dysexecutive Questionnaire (DEX) The DEX provides an analysis of the impact of executive dysfunction in the real world (Gillen, 2008). It is a 20-item questionnaire that measures the range of impairments related to

28 21 dysexecutive syndrome (Wilson et al., 1998) such as problems in abstract thinking, decision making and planning, confabulation, temporal sequencing, lack of insight, disinhibition and perseveration (Malloy & Grace, 2005; Strauss et al., 2006). It also assesses personality-related changes such as impulsivity, aggression, euphoria, apathy, lack of insight, distractibility and unconcern for social rules (Malloy & Grace, 2005; Strauss et al., 2006). The questions on the DEX tap into four areas: emotional or personality changes, motivational changes, behavioural changes and cognitive changes (Strauss et al., 2006; Wilson et al., 1998). Each question is scored on a 5-point Likert type scale (0 = never and 5 = very often) of problem severity. The DEX is presented in two versions: one is completed by the participant and the other by a relative, caregiver or a clinician who has close contact with the participant (Strauss et al., 2006; Wilson et al., 1998). Wilson et al. (1996) conducted a factor analysis on other s ratings on the DEX by caregivers and found three factors related to the symptoms reported by caregivers. These factors included behavioural, cognitive and emotional components (Wilson et al., 1996). In addition, Burgess and colleagues (1998) also performed a factor analysis and stated five symptoms as reported by the caregivers namely: inhibition, intentionality, executive memory, positive and negative affect. They found that the first three factors were well correlated with executive tasks, whereas the latter two factors had much weaker correlations (Burgess et al., 1998). They also noted that compared to other tests, the DEX was able reflect the patients lack of insight into their problems as most patients rated themselves as having less severe, executive impairments than their caregivers (Burgess et al., 1998). Bogod, Mateer & MacDonald (2003) compared the DEX (other and self ratings) and Self- Awareness of Deficits Interview (SADI) as independent measures of self-awareness with measures of EF and IQ in participants with TBI and failed to support previous findings that the DEX discrepancy score is associated with executive functioning as reported by Burgess et al. (1998) and Wilson et al. (1996). They found only a marginal relationship between the DEX and the SADI. Moreover, in comparison to the DEX, only the SADI had better correlations with measures of EF. These conflicting results may in part depend on the rater as Bennett and colleagues (2005) found that professionals provided more accurate assessment of executive dysfunction in comparison to the ratings provided by caregivers. Nonetheless, the DEX is one of the few rating scales able to measure self and others perceptions of executive behaviours.

29 Profiles of the Executive Control System (PRO-EX) The PRO-EX is not a questionnaire but a rating scale that assesses executive functioning in everyday situations (Braswell et al., 1992) on the basis of observation of activities over a time period. The rating scale is completed by a caregiver, significant other or clinician and measures current level of functioning in seven different component areas: goal selection, planning and sequencing, initiation, execution, timesense, awareness of deficits, and self-monitoring (Braswell et al., 1992). The PRO-EX assesses how impairments in these component areas impact everyday activities (Proctor, Wilson, Sanchez, & Wesley, 2000). Goal selection refers to the ability to create goals and have a sense of intention during goal setting. Planning and sequencing refer to organization skills needed to form written or oral plans. Initiation refers to the ability to begin an action independently or with physical prompts. Execution deals with the ability to carry out a multistep action. Timesense refers to the ability to complete an action in a specific time period and monitor time efficiently. Awareness of deficits assesses awareness of deficits post-injury and the need to use compensatory strategies. Self-monitoring refers to the ability to evaluate and modify behaviours as needed. These component areas are rated from 1-7, while self-monitoring is rated on a scale from 1-6 with the highest possible score of 48 that can be achieved on the PRO-EX (Braswell et al., 1992). Proctor and colleagues (2000) investigated the relationship between EF and working memory in eight adolescents with closed head injury. They used the PRO-EX to measure executive dysfunction and the recognition memory task (RMT) to assess working memory and found a strong positive correlation between the two measures. A positive relationship was found between goal selection, planning and sequencing, awareness of deficits, self-monitoring and RMT scores. They also found moderate positive correlations between initiation, execution, timesense and RMT scores (Proctor et al., 2000). The PRO-EX was able to differentiate between the patient group and the control group, however the RMT was not. The results of the RMT showed that some patients were able to perform as well as matched controls. In contrast, when comparing the patient group with the controls on the PRO-EX, the results indicated that the controls functioned significantly different in their daily lives than did the patient group (Proctor et al., 2000). This is an important finding because it indicates that the PRO-EX, which is a naturalistic questionnaire is able to tap into and differentiate between executive abilities. The PRO-EX has recently been

30 23 suggested to be used with older adults with late-life mood and anxiety disorders that demonstrate executive dysfunction (Mohlman, 2005). Performance-based Assessments Kitchen Task Assessment (KTA) The KTA is a performance-based assessment of cognition and EF while performing a specific instrumental activity of daily living (IADL) (Baum & Edwards, 1993). It involves a simple cooking task in which participants are required to prepare a cooked pudding (Baum & Edwards, 1993). The KTA has three purposes: to determine which of the six EF components (mentioned below) is affecting performance; to determine the individual s ability for independent functioning; and to determine the level of assistance required to complete the task (Baum et al., 1993). The KTA uses a structured cueing and scoring system to assess six executive components: (1) initiation, which assesses if the individual is able to start the task, (2) organization, which assesses if the individual is able to gather the necessary items needed to perform the task, (3) performance of all steps, which assesses if the individual is able to execute all the necessary steps to complete the task, (4) sequencing, which assesses the ability of the individual to arrange the steps in a chronological manner, (5) judgment and safety, which assesses the individual s ability to complete the task safely, and (6) task completion, which measures the ability of the individual to know when the task is finished (Baum & Edwards, 1993). The participants are scored from 0 to 3 on the basis of the number and type of cues needed to successfully complete the task. A total score of 18 points suggests the need for total assistance. According to Baum and Edwards (1993), the information collected on this everyday task and the different aspects of behaviour can be used by clinicians in training caregivers as well as the participants. One limitation of the KTA is that it only measures performance on one task, which is not enough to judge a participant s overall everyday performance (Baum et al., 2008). In addition, the six cognitive components do not represent a full range of EF. For example, the test does not include planning, even though it is important in kitchen performance (Josman & Birnboim, 2001). Moreover, Josman and Birnboim (2001) note that the sequencing component does not include an external criterion, such as following a list of steps provided in which the ability to arrange and follow steps can be tested (Josman & Birnboim, 2001).

31 The Rabideau Kitchen Evaluation-Revised (RKE-R) Like the KTA, the RKE-R is a performance-based assessment, which requires preparation of a meal. The difference between the two tests however, is that the KTA is more focused on the execution of the cooking task, whereas the RKE-R primarily assesses the planning aspect of EF (Josman & Birnboim, 2001). In this test, participants are required to prepare a cold sandwich with two fillings and a hot instant beverage (Neistadt, 1992). The RKE-R assesses the participants status over time, their functional ability and the information gathered to determine the amount of assistance needed to complete these tasks and treatment plans (Neistadt, 1992). The two tasks involve 40 detailed component steps listed in the order they are usually performed. The examiner evaluates these steps as part of a scoring system and measures the level of cueing needed by the participants. Each step is scored between 0 and 3 for a maximum total of 120 points, where a score of zero signifies no assistance while a score of three indicates direct intervention needed to complete the step. One limitation of the RKE-R is that the scoring system is very long (Josman & Birnboim, 2001). This is because the scoring system includes a combination of performance components as well as performance steps. This makes the scoring method more complicated as some of the performance steps are routine steps executed during task completion, whereas performance components refer to more complex cognitive abilities, which are performed after understanding written instructions (Josman & Birnboim, 2001). Moreover, like the KTA, the results on the RKE-R cannot be generalized to the individual s overall EF as specific cognitive components such as the ability to know when the task is complete is not assessed (Josman & Birnboim, 2001). A recent study by Yantz, Johnson-Greene, Higginson and Emmerson (2010) compared the results of the RKE-R with various other neuropsychological measures in participants with stroke. They found significant correlations between RKE-R performance and almost all neuropsychological measures such as the MMSE, the Brief Test of Attention, the Hopkins Verbal Learning Test-Revised (HVLT-R) and the Rey Complex Figure Test. The HVLT-R Delayed Recall score had the strongest relationships with the RKE-R performance. Yantz and colleagues (2010) performed a post-hoc groupwise analysis and found that patients who had

32 25 more than five errors on the RKE-R had significantly worse performance on measures related to attention, learning and memory, visuospatial organization and cognitive estimation The Cooking Task The Cooking Task is another naturalistic, performance-based assessment that relies on kitchen performance to assess underlying executive impairments. The information gathered from the test can be used to better understand the actual impairments and discharge planning (Chevignard et al., 2008). The test requires participants to make two separate dishes: a baked, chocolate cake and an omelette for two people (Chevignard et al., 2008). This cooking assessment is different from the ones mentioned earlier as the environment where the test takes place also includes distracting utensils and ingredients to mimic what is normally present in a kitchen and ensures an ill-structured environment (Chevignard et al., 2008). In addition, the test assesses multitasking abilities of participants as they attempt to complete the assessment. The Cooking Task is able to differentiate between brain injured and control participants (Chevignard et al., 2008). Chevignard and colleagues (2008) demonstrated that brain injured participants made significantly more errors and performed dangerous behaviours compared to controls. Moreover, more than half of the brain injured group was not able to complete the test (Chevignard et al., 2008). They performed a regression analysis and found that the best predictor of the total number of errors in the Cooking Task was the scores of the Six Elements Test, the number of perseverative errors in the WCST, and verbal fluency. Also, the cognition sub-score of the DEX was significantly associated with the total number of errors in the cooking task (Chevignard et al., 2008). This test is unique as it involves a thorough classification and quantification of errors which takes into account personal, contextual and environmentally-related errors (Chevignard et al., 2008). In addition, the qualitative and descriptive analysis of errors provides specific information that can be used in rehabilitation planning (Chevignard et al., 2008) Executive Function Performance Test (EFPT) The EFPT is another performance-based measure that assesses executive dysfunction by incorporating four real world tasks including preparing a meal, using the telephone, paying bills and managing one s medication (Baum et al., 2008). The EFPT uses the meal preparation task of

33 26 the KTA but replaces the task of preparing pudding with preparing oatmeal. The test utilizes the same cueing and scoring system and assesses the same five EF components as the KTA (Baum et al., 2008). The EFPT results in three scores: (1) the executive function component score, (2) the task score and (3) the total score (Baum et al., 2008). This information reveals which EF components are impaired, whether the participant is able to live independently and helps the caregivers and family members of the participant to understand what kind of support and/or reinforcement is needed to gain optimal performance at home (Baum et al., 2008). The EFPT has several advantages. First of all, it is easily administered after brief training (Baum et al., 2008; Wolf, Stift, Conor, Baum, & The Cognitive Rehabilitation Research Group, 2010). Second, the authors suggest that it is able to isolate the specific executive process during the completion of four real world activities (Baum et al., 2008; Wolf et al., 2010). Last but not least, the EFPT allows the examiner to objectively measure the participant s activities rather than relying on self-reports (Baum et al., 2008; Wolf et al., 2010). The EFPT has shown to discriminate between healthy controls and participants with mild and moderate stroke (Baum et al., 2008) and is able to detect executive abilities as early as one week following stroke (Wolf et al., 2010). The EFPT has been used on other clinical populations such as multiple sclerosis (Goverover et al., 2005) and schizophrenia and was able to differentiate between individuals that differed on the degree of pathological signs and phases of the disease (Katz, Tadmor, Felzen, & Hartman-Maeir, 2007). One major limitation of the EFPT was found in the study with schizophrenic participants (Katz et al., 2007). In this study, the initiation component of EF was not able to differentiate between the groups (Katz et al., 2007). This is because the test includes structured activities and instructions from the examiner, which limit the ability to measure initiation Executive Function Route-Finding Task (EFRT) The EFRT is a naturalistic, performance-based assessment that measures executive abilities related to route-finding or wayfinding (Boyd & Sautter, 1993). In this test, participants are asked to find an unfamiliar place within a facility. Performance is rated on six different abilities: task understanding, information seeking, retaining directions, error detection, error correction and ontask behaviour (Boyd & Sautter, 1993). Task understanding assesses both the ability to understand instructions and the ability to grasp the nature of an open-ended task with very few

34 27 guidelines provided. Information seeking involves the plans and strategies adopted by the participants, the type of information sought and how the information is searched for. Retaining directions assesses how the participants are able to retain important information received, such as paraphrasing or note taking (Boyd & Sautter, 1993). Error detection measures if the participants are able to note any discrepancy between directions and performance or surroundings (Boyd & Sautter, 1993). Error correction assesses the level of independence demonstrated in troubleshooting and adjusting behaviour. Finally, on-task behaviour measures the extent to which the participants are focused on completing the task in the presence of various distractions. These executive abilities are rated using a Likert-type scale ranging from 1 to 4, where one represents extensive dependence on the examiner and four indicates independent completion of the test (Boyd & Sautter, 1993). Webber and Charlton (2001) used the EFRT in older adults to study the nature of difficulty in wayfinding and concurrent validity of the EFRT. In their study, participants were asked to find their way from a specified location to one of the two rooms in their residence. One room was a familiar room (e.g. the dining room) while the other room was an unfamiliar room (e.g. the cleaner s room) (Webber & Charlton, 2001). The results demonstrated a significant correlation between the EFRT when finding an unfamiliar location and Rivermead Behavioural Memory Test (RBMT) scores, which reflected deficits in everyday memory functioning (Webber & Charlton, 2001). A lower, but significant relationship was also found between EFRT when finding a familiar location and the RMBT scores (Webber & Charlton, 2001). However, no significant relationship was found between the EFRT and the Wechsler Adult Intelligence Scale- Revised and the MMSE scores. Webber and Charlton (2001) also reported that approximately one-third of the participants had some difficulty in finding their way to a familiar room, while half of the participants had difficulty finding the unfamiliar room. The key wayfinding difficulties that were observed in this study were: the inability to maintain attention to the task, checking and correcting errors, asking for help and remembering where to go (Webber & Charlton, 2001). These results are consistent with what was demonstrated by Boyd and Sautter (1993) in young adults with head injury. Spikman, Deelman and van Zomeren (2000) investigated the presence and nature of executive impairments after closed head injury in 51 participants compared to 45 healthy controls as they completed various tests of EF, planning and attention. They also wanted to study whether these

35 28 tests were able to differentiate between patient and control groups. Spikman and colleagues (2000) found that of all the EF, planning and attention tests 1 they investigated, only the EFRT showed a significant difference between the two groups. Spikman et al. (2000) concluded that problems in executive functioning can be observed only in tasks that mimic everyday situations and allow the participants to generate a strategy on their own. They also reported that the patient group often needed more cues than the healthy control group to continue working on the task. In addition, the patient group sought information less adequately and had more difficulty with detecting and correcting errors in comparison to the control group (Spikman et al., 2000) The Instrumental Activities of Daily Living (IADL) Profile The IADL Profile extends the ADL Profile (Dutil, Bottari, Vanier, & Gaudreault, 2005; Dutil, Forget, Vanier, & Gaudreault, 1990) which measures independence in 17 ADL and IADL tasks ranging from grooming to making a budget. Due to the challenges faced by clinicians during task analysis on the ADL profile, which resulted in low inter-rater reliability, the authors decided to refine the tasks involved and improve the scoring system (Bottari, Dassa, Rainville & Dutil, 2009a; 2009b; 2010). This led to the development of the IADL Profile, which consists of eight tasks, six of which are associated with the overall goal of receiving unexpected guests for a meal (including putting on outdoor clothing, going to the grocery store, shopping for grocery items, preparing a hot meal for guests, having a meal with guests, cleaning up after the meal) and two of which are single complex tasks (including obtaining information and making a budget). These tasks are measured via direct observation by occupational therapists in the participants home and community environments and performance is analyzed according to four operations related to EF: (1) goal formulation, which refers to the ability to find a solution to solve a problem situation, (2) planning, which refers to the ability to generate a strategic plan of action after reviewing all the alternatives, (3) carrying out the task, which refers to the ability to initiate the plan and adapt to novel and ambiguous situations, and (4) attainment of the initial goal, which refers to the ability to verify that the task was executed as planned and make the necessary adjustments (Bottari et al., 2009a; 2010) 1 The Modified Tinker Toy Test, the Ecological Planning Task, the Spatial Learning Task with Self Set subgoals, the Tower of London Test, the Reaction Time Task, the Modified PASAT, the Stroop test and the Trail Making Test.

36 29 The IADL Profile measures both the type of difficulties encountered and the type and amount of assistance needed to complete the tasks (Bottari et al., 2009b). The first six tasks are assessed on all four operations of EF, while the final two tasks are measured using only three operations for a total of 30 items (Bottari et al., 2009b). This is because the goal formulation operation is not rated as it is the examiner who prepares these goals (Bottari et al., 2009b). Each operation is scored using a five-level ordinal scale (dependence, verbal and physical assistance, verbal or physical assistance, independence with difficulty, independence without difficulty) (Bottari et al., 2009a; 2009b). Bottari and colleagues (2009a) examined the relationship between three EF measures (The Stroop, Tower of London test and WMS-III) and three indices of TBI severity (Glasgow Coma Scale (GCS), Post-traumatic Amnesia (PTA) and coma length) with the IADL Profile in 100 patients with moderate/severe TBI. They reported all three indices of TBI to be significantly correlated with the IADL Profile, with the highest correlation with coma length. In terms of the EF measures, the IADL Profile was modestly correlated with the Working Memory Index (WMI) of the WMS-III and the Tower of London test (Bottari et al., 2009a). The study however, failed to observe any significant relationship with the interference score of the Stroop (Bottari et al., 2009a). The authors argue this may have resulted because of the differences in the two tasks: the structured inhibition task of the Stroop and the uncontrolled nature of the real world environment where multiple distractors are present and the IADL Profile is administered (Bottari et al., 2009a). In addition, Bottari et al. (2009a) also noted that they did not find any significant relation between gender and any of the IADL Profile scores suggesting that the test is applicable to both men and women even though the test mostly centers around a meal preparation task. The authors point out several advantages of the IADL Profile compared to other real world measures of EF. According to Bottari and colleagues (2010), the EFPT and the Multiple Errands Test (MET) (discussed below) give more structured instructions compared to the instructions provided in the IADL Profile, which simply states that the participants have to prepare a meal for an unexpected guest. They are given $20 for their expenses, however they need to make their own plans in terms of the meal they wish to prepare for their guest, the ingredients they will need, which store they would need to go to purchase the items and how they would get there. The authors argue that even though the goals of these real world measures are fairly similar, the instructions given in the EFPT and the MET eliminate the likelihood of observing the

37 30 participant s ability to formulate goals and plans since the participants are asked to follow instructions. In contrast, the unstructured nature of the instructions provided in the IADL Profile allows a more detailed assessment of the participant s ability to perform in everyday tasks. In addition, Bottari et al. (2010) argue that the MET assesses executive dysfunction on the basis of the number and type of errors observed, however the IADL Profile measures errors, difficulties as well as task related abilities to provide important information for treatment planning and interventions The Multiple Errands Test (MET) The MET is a naturalistic, performance-based assessment of EF which allows the examiner to observe participants doing real-life tasks such as shopping and collecting information. It was created by Shallice and Burgess in 1991, however since then, simplified hospital (Knight et al., 2002) and shopping mall (Alderman, Burgess, Knight, & Henman, 2003) versions have been created with a defined number of tasks and scoring system. The MET involves observing clients moving around a real world environment (e.g. shopping mall or a hospital complex) to purchase specific items (e.g. buy local stamps) and collect specific pieces of information (e.g. closing hours of the library). The test consists of four sets of tasks (12 subtasks in total) which are undertaken within the constraints of a set of rules (e.g. you may not spend more than $7.50) and allows the examiner to observe errors made and strategies used. The test allows participants to structure, plan, monitor and execute tasks efficiently. It also places multitasking demands on the participants which is a unique feature of the MET (Burgess, 2000; Brugess et al., 2000; Shallice & Burgess, 1991). Several studies have reported that the MET is able to discriminate between participants with and without brain injury (Alderman et al., 2003; Dawson et al., 2009; Rand, Rukan, Weiss, & Katz, 2008), and initial results have demonstrated that patients who did relatively well on traditional neuropsychological tests of EF performed worse on the MET as compared with controls (Shallice & Burgess, 1991). Significant correlations have been demonstrated with the DEX (other ratings) (Dawson et al., 2009), the Assessment of Motor and Process Skills the Sickness of Impact Profile (Dawson et al., 2005a) and the Zoo Map subtest of the BADS (Rand et al., 2008) suggesting that the MET measures several aspects of executive functioning.

38 31 Andre, Anderson, Stuss and Dawson (2009) used a unique approach by assessing the strategies used by participants with TBI and stroke, and matched healthy controls as they attempted to complete the Baycrest version of the MET (BMET). The strategies were classified as the use of print, personal or environmental resources and money management. They found that the total number of strategies observed was not significantly different between cases and controls (Andre et al., 2009). They also reported that participants with stroke used more environmental resources compared to the participants with TBI (Andre et al., 2009). These findings are important because they put forth the idea that compared to traditional measures of EF, different aspect of performance on the MET (e.g. strategies used, errors committed and naturalistic behaviours performed) can be further analyzed to understand the impact of executive dysfunction in real world situations. Alderman and associates (2003) proposed an interesting way to analyze errors in the braininjured and control participants which were committed during the completion of the MET. They argued that it is not enough to measure the number of errors committed by patients and controls. Instead, it is important to identify the qualitative differences in performance in both groups (Alderman et al., 2003). This is because they found that participants in both groups committed similar types of error such that both groups can be categorized as either rule breakers (for breaking rules) or task failers (for failing to complete assigned tasks). They devised a more sensitive scoring method in which errors were weighted based on their normality (Shallice and Evans, 1978). This way, normal errors or errors performed by 95 percent of controls were given a score of one; errors committed by five percent of controls were given a score of two, and errors unique to the brain-injured group were given a score of three (Alderman et al., 2003). This proved to be a better scoring method and accounted for significant differences between the groups. Dawson et al. (2009) also used the weighted scoring method and reported significant differences between participants with stroke and their matched controls on the MET. They found that patients with stroke committed more rule breaks and performed significantly worse on the number of tasks completed, and patients with TBI scored significantly worse on the weighted error score and a trend towards worse performance on tasks omitted and time to completion. Similarly, Rand and associates (2008) also employed this method and found that participants with stroke were more likely to make mistakes, which resulted in rule breaking and inability to multitask during the completion of the MET.

39 32 Alderman and colleagues (2003) furthered the analysis by looking more closely into the executive impairments in rule breakers vs. task failers by evaluating the responses of caregivers on the DEX questionnaire. They compared five symptoms in the DEX questionnaire, namely inhibition, intentionality, executive memory, positive affect and negative affect (Burgess et al., 1998) with the two groups and found that rule breakers exhibited more executive memory symptoms related to confabulation, inability in temporal sequencing and perseveration (Alderman et al., 2003). In contrast, task failures showed more symptoms of negative affect such as apathy and lack of emotion. In the context of the MET, the relation between rule breakers and executive memory symptoms can be explained by the fact that these participants were either more likely to fail to carry out the instructions they received or were unable to understand the instructions (Alderman et al, 2003). In other words, rule breakers had problems in monitoring their behaviours and as a result were not able to follow the rules correctly. In contrast, the relationship between task failers and negative affect can be described on the basis of lack of initiation in this group of participants; that they were unable to complete the task because they failed to initiate those in the first place (Alderman et al., 2003). This section provided an overview to the naturalistic assessments of EF. A great deal of variety exists in these assessments, from questionnaires assessing everyday activities to tests measuring the ability to prepare a meal and follow a shopping list. These measures may provide a great deal of information in terms of the impact of executive dysfunction on everyday life, hence it is important to discuss their ecological validity. Ecological Validity of Read-World Measurements of Executive Functions There are two approaches that need to be considered when it comes to ecological validity: verisimilitude and veridicality. Verisimilitude refers to the degree of similarity in cognitive demands between the testing arena and everyday environment (Chaytor & Schmitter- Edgecombe, 2003; Franzen and Wilhelm, 1996). Thus, to achieve verisimilitude, tests must include tasks that resemble situations in everyday life and there must be considerable relationship between the complexity of the test and the behaviour that is being tested (Marcotte, Scott, Kamat, & Heaton, 2010). This relationship allows the test to closely approximate the participant s ability to perform those tasks in daily life and infer more conclusive results (Spooner & Pachana, 2006). Chaytor and Schmitter-Edgecombe (2003) suggest that this

40 33 approach requires tests of EF to have more face validity than traditional measures and to be able to simulate cognitive tasks in daily life. As a result, these tests are more likely to identify challenges that people have in completing real world tasks, however may not discriminate braininjured population from controls or identify the etiology of brain injury. This view shifts away dramatically from the traditional focus where tests have been used continuously to diagnose brain damage. The idea behind the verisimilitude approach is that performance on these tests would improve with increase in functional skills even if the brain damage remains (Chaytor & Schmitter-Edgecombe, 2003). Some standardized neuropsychological assessments have developed while keeping in mind the verisimilitude approach to ecological validity. These tests include the Test of Everyday Attention (Robertson, Ward, Ridgeway, & Nimmo-Smith, 1996), the RBMT (Wilson, Cockburn, & Baddeley, 1985) and the BADS (Wilson et al., 1996). These tests attempt to measure everyday skills in attention (e.g. searching a telephone directory), memory (e.g. remembering the location of an item), and executive functioning (e.g. problem solving), respectively. Although this has not been clinically proven, it can be argued that the MET demonstrates verisimilitude as the setting encountered in the MET is more naturalistic and life-like. In addition, an individual is more likely to go shopping for various items and collect specific pieces of information in their everyday life. Moreover, the MET allows the individual the freedom to plan and execute tasks at will with very little restriction on the time and manner chosen by the participant to complete the tasks. Veridicality refers to the degree to which results on an assessment are related to the scores on other measures of everyday functioning (Franzen & Wilhelm, 1996). This approach involves the use of statistical methods to understand the relationship between neuropsychological tests and measures of real world functioning such as employment status, clinical and behavioural observations. Tests like the EFPT, which the occupational therapists administer can be tested for veridicality by determining the correlation of results with other measures of functioning such as clinician s ratings to build treatment plans and guide intervention. The EFPT already demonstrates significant correlations with standardized measures which assess working memory, verbal fluency and attention (Baum et al., 2008). This test can also be tested for verisimilitude because it uses real world tasks that are necessary to support independent living such as heating up a light meal, managing medications, using the telephone and paying bills (Baum et al., 2008).

41 34 The assessments described in this chapter have been developed relatively recently and demonstrate a greater degree of ecological validity than do traditional measures of EF. Yet, there are still a number of outstanding issues. Most of the tests that demonstrate ecological validity have not attained widespread use because they have only been used by limited number of researchers or across a few neurological groups (Marcotte, et al., 2010), and have limited psychometrics and lack of theoretical base, time to administer and ease of administration. In addition, since many tests have been created by researchers in their own facility, they have only been employed in one laboratory which poses a challenge to their use in general (Marcotte, et al., 2010). Another obstacle in developing ecologically valid measure is that it needs to be challenging enough so that it results in a normal distribution of function across individuals (Marcotte, et al., 2010). This is important to avoid scenarios where everyone either achieves a perfect score or fails the test. However, this reveals another critical problem for the assessments. If the difficulty level of the assessment is too high, some tests may transform from a test-like measure to a game-like measure and lose the real world flavour it is supposed to possess (Marcotte, et al., 2010). This may particularly be the case with measures that are using virtual reality to simulate everyday situations, where the equipment and the environment may feel like an arcade game (Marcotte, et al., 2010). Despite these challenges, Marcotte and Grant (2010) suggest that it is essential to develop and implement new EF assessments that demonstrate better ecological. This is because we still lack the ability to fully understand the impact of executive dysfunction on everyday activities. As Burgess and colleagues (2006) indicate, it would be wise to develop measures whose design begins with observations of real world behaviours rather than looking at limited behaviours tested in the lab and inferring findings to real world scenarios. Marcotte and Grant (2010) also suggest the use of discreet technologies such as video cameras to improve measurement of behaviours occurring in natural settings. This will allow researchers to measure performance under common demands with naturally occurring distractions. Conclusion This chapter reviewed the literature on executive dysfunction in stroke population and provided the background on the definitions, relevant theories, and traditional and naturalistic assessments

42 35 of EF. It is important to have an understanding of these topics before moving to the next chapters, which discuss my master s research.

43 36 Background Chapter 3 Describing the Methodology: Event Recording One difficulty in understanding the impact of executive dysfunction on behaviours in everyday life is that environments have differential effects on behaviour. This is one reason why most assessments occur in very constrained circumstances, as discussed in Chapter 2. Unfortunately, as mentioned in Chapter 2, it is being recognized that the information gained form such assessments provides limited information regarding performance in more naturalistic settings (Burgess et al., 2006). Thus, to understand the impact of executive dysfunction fully, it is important to assess performance in natural environments. This study (described in Chapter 4) employed an event recorder to study the behaviours performed on the Baycrest version of the MET (BMET) which is a naturalistic assessment that assesses participant's ability to purchase specific items and obtain certain pieces of information within the restriction of a set of rules (Dawson et al., 2009) to better understand the impact of executive dysfunction on the performance of everyday life tasks in a natural setting. An event recorder is a device on which the user can record specific information about an event. There are a variety of different types of event recorders, ranging from recorders used in the assessments of cardiac patients, which monitor a patient's heart rhythm, flight data recorders, which record data on the operations of aircraft controls and performance and are involved in the investigation of airplane accidents, to computer software that enables behaviour analysis. This chapter focuses on describing the use of the latter. Researchers use event recorders to observe and record multiple, mutually occurring, or overlapping behaviours and events as they naturally occur. Event recorders can be used to record behaviours from videotapes as well as from naturalistic observations. They allow coding of the occurrence, frequency and duration of behaviours and events. These behaviours can then be viewed and analyzed using a variety of graphic, descriptive, aggregate and statistical representations. A literature search was conducted on PsycINFO using the following keywords: event recorder, event recording, event recorder software and behaviour. It was found that several researchers

44 37 have used event recorders in studies involving infants and children to measure infant feeding behaviour (Taylor, Lujan, & Vázquez-Geffroy, 2000), mother-child conflict behaviours (Huang, Teti, Caughy, Feldstein, & Genevro, 2007), in-home injuries (Morrongiello, Ondejko, & Littlejohn, 2004a; 2004b), modification of classroom behaviours (Sibley, Abbot, and Cooper, 1969) and volunteer tutors teaching performance when working with children with developmental disabilities (Tindall & van der Mars, 2005). This methodology has also been used to assess job environments of adolescents (Ruggiero & Steinberg, 1981), office seating behaviours in adults (Dowell, Yuan, & Green, 2001) and aggressive behaviours in young adults (Warden, Grasso, & Luyben, 2009). However, to the best our knowledge event recording has not been used in relation to naturalistic assessment of executive dysfunction following stroke. This technology was selected for use in this study because it would allow for a more detailed exploration of multiple behaviours occurring concurrently. 3.1 Event Recorder: Behaviour Tracker This study employed the event recorder called the Behaviour Tracker (Behaviour Tracker, 2003), chosen for its low cost, compatibility with several Windows-based operating systems, and satisfactory technical support to codify the behaviours of stroke participants and their matched controls as they attempted to complete the Baycrest version of the MET (BMET). This software consists of four different modes (see Appendix A for screenshots of the four modes described below) and allows coding of the occurrence, frequency and duration of behaviours. (1) The configuration mode allows for the creation of template files with predefined keys on the keyboard that can be used to specify each behaviour to be coded. For example, when participant goes to the gift shop, this behaviour is named as P in GS, assigned the letter A on the keyboard and specified to have its duration coded. (2) The record mode allows tracking and recording of the behaviours named in the configuration mode. It consists of start, stop and pause buttons. This mode also permits modifiers to be added, that is a descriptor of the behaviour to describe its uniqueness. For example, when the participant purchases Coke, this behaviour is named as 'P takes Coke' and it can be modified to describe whether the participant purchases a can or a bottle of Coke. (3) The editor mode allows viewing and modification of recorded sessions. It displays both the original data collected and the edited data side by side to allow editing of the frequency, duration and deletion of behaviours collected, as well as modification of

45 38 the descriptors added. For example, if one incidence of the behaviour participant picks up card was missed in the original recorded session, then another occurrence of this behaviour can be added using this mode. (4) The viewer mode allows the data to be viewed in several formats such as graphs, raw data, aggregate and detailed. It also permits exportation as a spreadsheet for further analysis. 3.2 Procedure for Using Behaviour Tracker Creating the codes The codes for each configuration file were created by starting with the inefficiencies and partial task failures identified in the score sheets employed by Dawson et al. (2009). These formed the framework to which more behaviours were added. Additional behaviours were added in the following ways: 1. Several strategies identified by Andre, Anderson, Stuss and Dawson (2009) were incorporated, for example: 'participant looks at map', 'participant looks at task sheet', 'participant checks watch', 'participant looks at signage and surroundings', 'participant looks at candy rack for the price of Mars bar', 'participant self-talks' and 'participant asks staff for help'. 2. The theories of executive function were reviewed to assist in the identification of key behaviours to be recorded. For instance, it was important to code overlapping and concurrently occurring behaviours such as 'participant checking watch while walking' or 'participant stopping and marking task sheet' to note dual-task behaviours that were expressed during the BMET. This also allowed us to look into the stopping and walking behaviours of the participants because those with executive dysfunction may demonstrate problems with dual-tasking and have to stop in order to perform certain behaviours such as marking task sheet or map. 3. Stuss et al. s (1995) fractionation theory of attentional functions also played an important role in shaping the final configuration files. Stuss and his colleagues (1995) identify task setting and monitoring processes which may be affected following damage to different areas of the frontal lobes. Task setting and monitoring behaviours specific to the BMET that were incorporated included behaviours specific to each task such as: 'participant buys

46 39 four stamps' and 'participant puts stamp on urgent letter collected at information desk to mail'. It also included those that were performed at any time the test such as: 'participant checks off task sheet' and 'participant checks watch'. 4. Other behaviours were added after reading written descriptions of participants performance prepared by the test administrators and watching some videotapes of participants with stroke and controls to capture those behaviours not yet captured by the coding system so that an exhaustive list of codes existed to document behaviours common to all participants as well as those that were unique to a specific participant Structuring the Codes In order to document all the behaviours performed, four separate configuration files were created. The first three files included codes that were specific to certain locations and tasks. The fourth file included codes that occurred at any time during the administration of the BMET. The files were organized this way since a participant can only be in one location at one point in time. This meant that only two of the four configuration files were needed to be coded for any event; for example, if the participant was in the gift shop (GS), only the configuration file that had all the behaviours occurring at the GS and the one that consisted of behaviours occurring at any time during the BMET had to be accessed. In addition, as one configuration file contained a maximum of 37 events (which corresponded to the 26 letters, space bar and 10 number keys on the keyboard) and the number of behaviours needed for this study exceeded that amount, it was deemed best to separate them in a way which made coding much easier and more accurate. This organization also made the logistics of coding easier. It was possible to play a participant s BMET video on one side of the computer monitor and have four separate files open next to one another on the monitor; Behaviour Tracker allowed simultaneous access to multiple files with one click of the mouse. The files were named Meta, Metb, Metc and Metd. Meta focused on the tasks that were carried out in the GS and at the lotto booth (see Appendix C). These included: (1) buy a birthday card, (2) buy four stamps, (3) write down the price of Mars bar, and (4) write down the opening time of the GS on Friday. Metb included the following tasks performed in the cafeteria, at the Information Desk (ID) and at the mailbox, respectively: (5) buy Coke, (6) collect something for the examiner and do what is necessary, and (7) mail something to Dr. Deirdre Dawson (see

47 40 Appendix D). Metc included the tasks performed near the phone, the library and the parrot cage (PC), namely: (8) telephone Katherine and tell her your name, your location and the time, (9) write down the closing time of the library, (10), meet examiner 10 minutes after starting the test at the PC and tell time, and (11) tell the examiner when the test is finished (see Appendix E). The task of writing down the number of entrances/exits on the main floor was not given a separate code and was incorporated as part of the behaviours coded in the final configuration file. Metd consisted of behaviours that occurred at any time during the entire test (see Appendix F). Appendix G lists all the behaviours and events coded using the Behaviour Tracker. All behaviours were coded as either frequency or duration events. Frequency behaviours were coded in terms of how often they occurred and were typically of very brief duration, for example, 'participant looks at task sheet'. The modifier was used to further describe the behaviour, for example, 'while walking'. Duration behaviours were coded in terms of how long the behaviour lasted, for example, 'participant looks at map'. The software also allowed frequency counts of duration behaviours. As shown in Appendix C, frequency behaviours were represented using a lightning icon (blue arrow) and duration behaviours were displayed using a stop-watch icon (red arrow) buttons on the Behaviour Tracker Coding Behaviors The process of coding participants BMET videos using the Behaviour Tracker is explained below. Each participant s video was first watched to ensure a general understanding of it before proceeding to the coding stage. This process took approximately half an hour to an hour and a half depending on the length of the video and the time it took to take notes. During this stage, written descriptions of participants' test performance by test administrators were also read. This was followed by the coding procedure. During coding, all four configuration files were open with the video being played on one side of the monitor. It was ensured that the start and stop times for each configuration file and the video were the same to have accurate coding of how long it took each participant to complete the BMET. Also, notes were taken if a behaviour or a modifier was missed during coding. After coding was completed, the files were saved using the BMET assigned identification and the configuration file name. The editor mode was then used to make the necessary corrections to the coded files, based on the notes taken, and if necessary, the video was watched again and followed using the codes in the editor mode. Once the editing and

48 41 reviewing was completed, the viewer mode was used to export both raw and aggregate data to a spreadsheet for each of the four configuration files. These spreadsheets were then merged into one file for further analysis. In general, it took three to four hours to code each video completely. The process of becoming familiar with the codes took several hours of training as some participants were very quick and a number of behaviours and events occurred simultaneously. Also, every BMET video was different, which impeded keeping to a single pattern of coding. For example, some participants were only able to complete half of the test and did not perform certain behaviours; on the other hand, other participants made an attempt to complete every task but were not able to finish them. Training in coding consisted of coding one video several times and comparing results. This also helped improve the accuracy of coding and following a video at the same time Reliability To ensure within-rater reliability, one BMET video was randomly selected and re-coded and the responses were identical. Reliability was further established by watching two BMET videos of participants with one of the co-investigators and comparing results. In both instances, the results were fairly similar. Four separate videos were also watched by a second rater and the results were compared with the coding conducted by the first author (SA). The second rater, an upper level MScOT student, had been trained on coding the BMET videos. Intraclass correlation coefficients were calculated to evaluate inter-rater reliability (see section on Reliability in Chapter 4 for inter-rater agreement for further details). 3.3 Other Application of the Codes and Conclusion To the best of our knowledge, this is the first attempt to conduct a detailed analysis of BMET behaviours using an event recorder. The coding described was developed for the naturalistic observation of executive dysfunction in participants with stroke and matched controls as they worked on the BMET in order to provide a detailed analysis of the behaviours performed. This coding system or some modification of it is also being used to assess executive dysfunction in participants with traumatic brain injury and their matched controls in another related study.

49 42 Chapter 4 Characterization of executive dysfunction in real world tasks: Analysis of behaviours performed during completion of the Multiple Errands Test Abstract (In preparation for submission to Neurorehabilitation and Neural Repair) Authors: Arshad, S., Anderson, N., Polatajko, H., & Dawson, D. The purpose of this study was to understand the impact of executive dysfunction on everyday activities in stroke participants. A classification system was developed to analyze behaviours performed by 14 stroke participants and 12 healthy control participants matched for age, education and gender on the Baycrest Multiple Errands Test (BMET), a task requiring participants to purchase different items and gather certain information within the main floor of a hospital. The study employed an event recorder to code the occurrences and frequencies of behaviours as participants attempted to complete tasks. It was found that participants with stroke performed significantly more task specific relevant inefficient behaviours (p <.05) and non-task specific irrelevant behaviours (p <.10) than controls. In addition, participants with stroke were significantly more likely to ask staff for directions to a location, and significantly less likely to go to the 0.99 card rack first and use the map while walking in comparison to controls (p <.05). These differences between stroke participants and controls indicate that future research should account for a wide range of behaviours occurring in a test situation and highlight the importance of assessment in a naturalistic setting. Introduction Executive dysfunction (ED) is thought to have a significant impact on an individual's ability to perform everyday activities independently (Godbout, Grenier, Braun, & Gagnon, 2005; Royall et al., 2007). This can occur following a variety of conditions leading to frontal lobe damage such as after traumatic brain injury and stroke (Levine, Turner, & Stuss, 2008). Executive functions are higher-order cognitive abilities that involve attention, planning, inhibition, reasoning, decision making and problem solving (Alexander & Stuss, 2003; Bryan & Luszcz, 2000; Keil & Kaszniak, 2002; Levine, Turner, & Stuss, 2008). They help an individual to formulate and complete goal-directed behaviours and to make decisions in novel and complex situations in life

50 43 (Cicerone, Levin, Malec, Stuss, & Whyte, 2006). This complexity related to executive functions makes it particularly difficult to assess. Recently, researchers have critiqued traditional clinical and laboratory-based assessments of executive functions. They argue that these assessments were not developed to assess ED and were instead a result of basic experimental brain research for psychological investigations (Burgess et al., 2006). These assessments measure function at the impairment level (e.g., problems in attention) and are relatively poor to predict the impact ED has on everyday performance and during completion of complex real world tasks (Alderman, Burgess, Knight, & Henman, 2003; Burgess et al., 2006; Chan, Shum, Toulopoulou, & Chen, 2008; Keil & Kaszniak, 2002; Lewis, Babbage, & Leathem, 2011). In addition, Burgess et al. argue that traditional assessments are usually highly structured in nature and are not representative of the situations encountered in the real world. For example, it is difficult to infer how sorting cards in the Wisconsin card sorting test is related to everyday situations and what circumstances in daily life would require the abilities measured by the Wisconsin card sorting test (Burgess et al., 2006). According to Burgess et al., assessing executive functions in real world settings would provide a more accurate representation of the participant's impairments. In response to the growing need for performance-based naturalistic assessments, a number of tests have been developed to assess ED in the real world behaviour. The Multiple Errands Test (MET) is one such measure. Administered in a real world setting (e.g. hospital complex, shopping mall), it requires participants to complete everyday tasks (e.g. buy a birthday card) and collect specific pieces of information (e.g. closing time of library) within the constraints of a set of rules (e.g. you should not enter hospital treatment areas). Thus, one is able to learn about the impact of ED in everyday life. This test was originally developed by Shallice and Burgess (1991) who demonstrated that patients who performed relatively well on traditional neuropsychological assessments measuring language, memory, perception and executive functions performed worse than healthy controls on the MET in terms of errors committed, inefficiencies and abnormal social behaviours. Shallice and Burgess (1991) argued that assessments such as the MET that require participants to plan and perform multiple tasks over extended periods of time, without consistent feedback from the examiner, reveal the impact of ED better than do traditional neuropsychological tests of ED. In relation to the International Classification of Functioning, Disability and Health s (ICF) framework of human functioning (World Health Organization,

51 ), the MET measures functional performance in a real world environment and assesses at the activity (previously known as disability) level (Chan et al., 2008). According to Chan et al. (2008), the MET assesses planning and strategy allocation abilities. To date, research on the MET has focused on understanding performance errors made by participants in terms of (a) inefficiencies, where a more productive strategy could have been used, (b) rule breaks, where a particular rule is broken, (c) interpretation failures, where task instructions are misunderstood, (d) task failures, where any of the 12 tasks are not fully achieved and (e) task omissions, where a particular task is excluded. Knight, Alderman and Burgess (2002) reported that participants with severe acquired brain injury (traumatic brain injury, stroke, tumors) committed significantly more rule breaks, had more errors, and completed significantly fewer number of tasks than controls. In addition, Alderman and colleagues (2003) developed a shopping mall version of the MET and found that brain-injured participants made three times more total errors, broke more rules and were more likely to fail to complete tasks in comparison to controls. They also developed a weighted scoring method to further analyze the results (see Alderman et al. for more details). They reported two different patterns of failure on the MET in the brain injured population: rule breakers and task failers. Rule breakers demonstrated problems in monitoring their behaviours and were unable to understand and follow instructions, which resulted in breaking task rules. In contrast, participants who were characterized as task failers were unable to complete tasks because they failed to initiate tasks in the first place. Similarly, Rand, Basha-Abu Rukran, Weiss, and Katz (2008), using a virtual version of the MET as well as the MET in a real mall, found that participants with stroke made many types of errors in planning, problem solving and multitasking, and were unaware of their errors and made more social mistakes relative to controls. Dawson et al. (2009) developed a Baycrest version of the MET (BMET) and also used the weighted scoring method. They reported that participants with stroke committed more rule breaks and performed significantly worse on the number of tasks completed. They also found that participants with TBI scored significantly worse on the weighted error score and trended towards worse performance on tasks omitted and time to completion. The abovementioned studies provide an understanding of the performance errors in terms of inefficiencies, interpretation failures, rule breaks, task omissions and problems in completing tasks on the MET in the acquired brain injury population. Like the MET, other naturalistic

52 45 assessments of ED have also emphasized on studying performance errors such as the Cooking Task (Chevignard et al., 2000) and the Rabideau Kitchen Evaluation-Revised (Neistadt, 1992), both of which rely on kitchen performance. Understanding how well an individual performs on a particular assessment is important and this is a traditional approach to scoring assessments of cognition. However, we argue that to fully understand the impact of ED on participants' performance in everyday situations, it is necessary to further examine the behaviours performed while completing an assessment. This includes behaviours that are task specific as well as those that the participants perform in relation to various environmental and contextual constraints. This raises questions as to which behaviours should be scored, how to score these, and also how to classify them. The ICF also emphasizes the importance of studying the environmental and personal factors that may impact activity and participation in an individual (Vrankrijker, 2003). To the best of our knowledge, thus far, no one has taken a theoretical approach to identifying and categorizing behaviours, as we were unable to find relevant classification in the literature. Hence, the main purpose of this exploratory study was to perform an in-depth analysis of observable behaviours during the course of the MET. This would help investigate whether a wide range of behaviours would also be important for successful test performance and allow better discrimination between participants with stroke and healthy matched controls. The main objective was to identify the behaviours performed by participants with stroke and healthy controls (matched for age, education, gender) and determine differences between the two groups in relation to their executive function. Materials and Methods Participants This study was a secondary analysis of data collected for a previous study on BMET performance (Dawson et al., 2009). Data consisted of videotapes of community dwelling adults with stroke and healthy controls: 14 stroke survivors and 12 healthy controls matched for age, gender and education (see Table 4.1 for a comparison of these variables). The participants with stroke were divided into two groups on the basis of documented ED: 11 participants with stroke (aged 47-77; mean=61.8; SD= ±11.5) and 3 participants with stroke-ed (aged 33-73; mean=48.7; SD= ±21.4; see Table 4.1 for participant characteristics). The stroke-ed group

53 46 consisted of participants that demonstrated impairments in executive function, which was defined as 1.5 standard deviations or more below age-corrected norms on two or more of the following neuropsychological tests: FAS verbal fluency test (Benton, Hamsher, & Sivan, 1994; Gladsjo, Shuman, Miller, & Heaton, 1999), Digits Backward (Wechsler, 1985), Trails B (Reitan & Wolfson, 1985), and the Wisconsin card sorting test (Heaton, Chelune, Talley, Kay, & Curtiss, 1993). The participants in stroke-ed group were younger, more highly educated and had had their strokes more recently compared to the participants in stroke group (p < 0.05). However, there were no significant differences among the three groups on the basis of age, gender, education, self-reported familiarity with the 1 st floor of Baycrest ratings, the number of times visiting Baycrest, and the number of rules remembered with and without cues (p > 0.20). Table 4.1 Participant Characteristics Total Participants with Stroke (n=14) Participants with Stroke (n=14) Stroke-ED (n=3) Stroke (n=11) Total Controls (n=12) Age (y) 59.0 ± 14.2 (33-80) 48.7 ± 21.4 (33-73) 61.8 ± 11.5 (47-77) 56.9 ± 16.5 (27-81) Education (y) 15.1 ± 3.3 (7-19) 18 ± 1.0 (17-19) 14.3 ± 3.2 (7-18) 15.7 ± 3.5 (10-23) Number (males:females) 8:6 3:0 5:6 7:5 Years post-stroke 8.6 ± 6.0 ( ) 2.7 ± 2.1 ( ) 10.6 ± 5.9 ( ) n/a Familiarity with 1 st floor (rating scale 1-10) 4.0 ± 3.3 (1-10) 5.7 ± 4.0 (1-8) 3.5 ± 3.1 (1-10) 3.3 ± 1.4 (1-6) Number of times been to Baycrest 1.7 ± 3.6 (0-10) 0.7 ± 1.2 (0-2) 2.0 ± 2.0 (0-10) 1.6 ± 1.4 (0-5) Number of rules remembered without cue 9.6 ± 2.4 (4-11) 9.0 ± 3.5 (5-11) 9.8 ± 2.2 (4-11) 10.4 ± 1.2 (7-11) Number of rules remembered with cue 10.1 ± 1.7 (6-11) 9.7 ± 2.3 (7-11) 10.3 ± 1.7 (6-11) 10.8 ± 0.6 (9-11) NOTE: Values are Mean ± SD (range) or as otherwise indicated Participants with stroke were recruited through local community agencies or from a list of participants who had given consent to take part in future studies. They were included in the study if they met the following inclusion criteria: (a) a minimum of 3 months post-injury, (b) were at least 18 years of age or older, (c) were able to read, understand and speak English and (d) were able to walk independently for at least half an hour. During the screening process, those who scored above the cut-off of 16 on the Centre for Epidemiological Studies Depression Scale (Radloff, 1977) were not included in the study. Also, those with conditions such as seizures or leukemia were excluded from the study. Severity of stroke was difficult to determine for every participant because of the time that had elapsed since the events. We were unable to obtain

54 47 health records and medical information on 5 out of 14 stroke survivors. Participants in both stroke and stroke-ed groups had good language abilities. However, two participants in the stroke group and two in the stroke-ed group had hemiparesis and walked with the help of a cane. The twelve controls were recruited through friends and family members of the participants and from the Baycrest volunteer pool and matched individually for gender, age (±5 years) and education (±5 years). They met the same inclusion criteria as participants with stroke with the exception of post-injury criterion and had to have a Mini-Mental Status Examination (MMSE) score within the normative range based on age and education. All control participants also went through neuropsychological assessment consisting of tests of attention, executive function, memory, visuo-perception and visuo-constructional abilities and controls were required to be within 2 standard deviations of age and education norms on each of these neuropsychological tests (see Dawson et al., 2009 for more details on the neuropsychological assessment). The study was conducted in accordance with human ethics standards and received ethics approval from the joint Baycrest/University of Toronto Scientific and Ethics Review Committee. All of the participants provided informed, written consent to participate in the study. The BMET The BMET, a version of the MET, was developed for use on the first floor of the Baycrest Centre in Toronto, Canada. In this test, participants are required to complete 12 everyday tasks while observing a set of 8 rules (see Appendix B and Dawson et al., 2009 for a list of tasks and rules). The test required participants to access different areas of the first floor while using a map, namely the gift shop, mailbox, information desk, cafeteria, parrot cage and library. Participants were give a clipboard with the map of the first floor of Baycrest and task sheet listing the 12 tasks, which included buying and collecting items such as stamps, obtaining information such as opening time of the gift shop, mailing something, meeting the examiner at a specific time and location and telling the examiner when they have completed the test. Participants were also provided with a watch, a pen, a bag to store their collected items, and a ten dollar bill for their purchases. A pre-test session was conducted with each participant to familiarize them with the tasks, rules, and expectations, and also to answer any questions they had before starting the test. During this

55 48 session, participants were asked to memorize the 8 rules (three of which have two subparts for a total of 11 rules). Participants were asked to freely recall each rule and were cued if they could not do this. Both of these scores are shown in table 4.1. The test took about 60 minutes and was videotaped and scripted to allow for scoring. Coding procedure Creating codes The 26 BMET videotapes of were viewed and coded by the first author (SA) using event recording software Behaviour Tracker version 1.5 (Behaviour Tracker, 2003). Behaviour Tracker is an inexpensive software compatible to Windows-based operating systems, which allows coding of the occurrence and frequency of multiple, concurrently occurring behaviours performed by participants with stroke, stroke-ed and controls as they worked on the BMET (see Chapter 3 for more detail on event recording). A list of behaviours was identified and 66 separate codes for the Behaviour Tracker were created to incorporate these behaviours. Behaviours were identified in the following manner: 1. Incorporating those previously documented in the score sheets of the BMET used by Dawson et al. (2009); 2. By watching a subset of the videos (SA & DD) and determining additional behaviours not previously coded, and by reading test administrators written descriptions of participants test performance (SA). Several strategies recognized by Andre, Anderson, Stuss and Dawson (2009) (who studied the same sample of participants) were also utilized. Various rule breaks originally part of Dawson et al.'s (2009) study were also included such as 'participant talks to the examiner' as well as examples of task specific rule breaks such as 'participant goes to the second floor'. Interpretation failures that were part of Dawson et al.'s (2009) work were also included such as 'participant does not tell time at the parrot cage'. The Behaviour Tracker also allowed identification of dualtasking behaviours such as walking and looking at task sheet. This is important because those with ED may have more problems with dual-tasking and have to stop to complete certain behaviours.

56 49 Coding behaviours To ensure all behaviours were captured, all videos were watched three times by the first author (SA). Each participant's BMET written description was read and the video was watched prior to coding to gain familiarity. The video was watched a second time and coding was performed. One computer was used to code the video with both the video and the Behaviour Tracker software running parallel to one another. Finally, each video was watched a third time to ensure completeness and accuracy of coding. Reliability of coding Inter-rater reliability was evaluated by having a second rater code four of the 26 participant videos using the Behaviour Tracker. The Intraclass Correlation Coefficients (ICCs) were calculated using two-way random effects models, which makes the assumption that raters and participants are random factors from a larger pool. Out of a total of 66 codes on the Behaviour Tracker, the ICCs for 64 codes were substantial (> 0.6). Two codes had the ICC < 0.2, these corresponded to two separate behaviours (see table 4.2 for inter-rater reliability results). Table 4.2 Inter-rater Reliability Number of Codes ICC Code Details 50 codes codes codes participant touches card - examiner talks to participant Results Behaviour classification Once all the videotapes were coded, behaviours were classified in the following way. As a first step, behaviours were divided into two groups: (1) task specific behaviours, which referred to those that were related to the completion of the 12 tasks on the BMET (for example 'participant buys birthday card'), and (2) non-task specific behaviours, which referred to the behaviours that occurred at any time during the test, but were not required or related to completing tasks on the BMET, such as 'participant checks watch'. We organized the behaviours into these two groups to analyze both task related as well as non-task related behaviours. We proposed that this would provide an overall understanding of the impact of ED on an individual.

57 50 All the coded behaviours were then classified into one of the three categories within each group: (1) relevant efficient behaviours, (2) relevant inefficient behaviours and (3) irrelevant behaviours. Relevant behaviours were operationalized as those that were bearing upon and pertinent to achieving a task. Irrelevant behaviours were those that were not applicable and/or pertinent to achieving a task, which included apparent habitual behaviours such as 'licking a selfadhesive stamp', as well as rule breaks such as 'talking to the examiner' and distracters such as 'watching television in the patient area'. Efficient behaviours were those that were performed effectively and in the best possible manner to yield the most desirable result and inefficient behaviours referred to those that were not performed effectively and/or did not yield the most desirable result. Table 4.3 provides an example of how certain behaviours were classified into their respective categories. An example of a task specific relevant, efficient behaviour is 'writes down opening time of gift shop on Friday'. This classification was made as this is a required task on the BMET. An example of a task specific relevant, inefficient behaviour is 'asks for less than four stamps'. One of the tasks on the BMET was to buy four local stamps in which it was found that some participants asked for less than four stamps. This classification was made because this behaviour would not yield the most desirable result since the participant asked for less than the amount required. An example of task specific, irrelevant behaviour is 'reads message inside the card'. This classification was made because this behaviour was not required to successfully complete the task. It was categorized as an apparent habitual behaviour since people generally read messages when they purchase cards. In order to complete the birthday card task efficiently, the participant would have to go to the gift shop, go directly to the 0.99 card rack, find a birthday card, and (if possible) check the back of the card to make sure the price is right and purchase the card. However, reading the message, going through different cards and switching between card racks would result in loss of time, when one of the rules provided on the task sheet stated that they are to take as little time as possible to complete the exercise.

58 51 Table 4.3 Behaviour Classification Task Specific Behaviours Non-task Specific Behaviours Efficient Writes down opening time of gift shop on Friday Relevant Inefficient Asks for less than four stamps Irrelevant Reads message inside card Asks staff for help Asks non staff for help Asks examiner for help We mention the example of 'asking for help' to highlight non-task specific behaviours. We classified 'asking staff for help' as relevant efficient. Staff members included hospital staff such as nurses, doctors, as well as cashiers and customer service representatives at the gift shop and information desk. It would be relevant and efficient to ask staff members for help because they would provide the most accurate and appropriate aid as they would be most aware of the hospital. On the contrary, we classified 'asking non staff personnel (e.g. book vendors, charity representatives) for help' as relevant inefficient because they may or may not know accurate information that the participants were seeking. We classified 'asking help from the examiner' as irrelevant because the rule list provided to the participants stated that speaking to the examiner is not allowed unless it is part of the exercise. A complete list of all the behaviours and their classification is provided in Appendix H. This includes behaviours specific to the 12 tasks that were part of the MET as well as non-task specific behaviours. Related results Descriptive analyses were conducted for each of the categories (task specific and non-task specific: relevant efficient, relevant inefficient and irrelevant behaviours) to study the differences in types and frequencies of behaviours for each of the three groups of participants. Differences between mean behaviours of participants in each of the three groups (stroke-ed, stroke, controls) for each of the six categories were analyzed using one-way analysis of variance (ANOVA). The Tukey method was conducted as a post-hoc test to further analyze which groups were significantly different than each other (Kafadar, 2003). The level of significance was set at 0.10 due to the exploratory nature of the study. The effect size was calculated using eta squared, a common effect size estimate for ANOVA in which 0.01 is a small effect, 0.09 is a medium effect and 0.25 is a large effect (Levine & Hullett, 2002). The descriptive data were closely

59 52 examined for behaviours that had the most variation based on descriptive data (i.e. mean, SD). All data were analyzed using the Version 17.0 of the SPSS. The ANOVAs were conducted to compare the mean differences between stroke, stroke-ed and control groups for all of the six behaviour categories. The main findings can be seen in Table 4.4. In the task specific category, there was no difference in the number of relevant efficient behaviours performed by stroke and stroke-ed groups compared to controls. The stroke group demonstrated more irrelevant behaviours than controls, while the control group displayed more irrelevant behaviours than the stroke-ed group. However, these differences were not statistically significant and the effect size was also small for these two behaviour categories. The stroke group performed more relevant inefficient behaviours than stroke-ed and control group, the ANOVA analysis was significant F(2, 23) = 4.18, p < 0.03, the Tukey's test revealed that the stroke group did this significantly more often than did the controls at the 0.05 level and the effect size was large for this difference. Table 4.4 Classification of participants' behaviours on the BMET. Differences in means, SD, range and p values between stroke-ed, stroke and control groups for each behaviour category Task Specific Behaviours Non-task Specific Behaviours Total Relevant Efficient Total Relevant Inefficient Stroke-ED (n=3) Stroke (n=11) Controls (n=12) p η ± 4.04 (17-25) ± 5.83 (10-27) ± 4.47 (14-28) ± 2.08 (2-6) 9.18 ± 5.53 (5-20) 4.42 ± 2.78 (2-11) * Total Irrelevant 1.33 ± 0.58 (1-2) 2.09 ± 2.34 (0-7) 1.58 ± 1.73 (0-6) Total Relevant Efficient Total Relevant Inefficient ± (22-50) ± (12-92) ± 9.89 (23-58) ± (20-49) ± (15-69) ± (12-52) Total Irrelevant 8.33 ± 5.69 (2-13) 4.45 ± 4.13 (0-13) 3.08 ± 2.47 (0-8) * NOTE: Values are mean ± SD (range) Significant at the p 0.10 for ANOVA analysis η 2 is the effect size In the non-task specific category, there was no significant difference between the number of relevant efficient behaviours performed by the stroke and stroke-ed groups compared to their matched controls and the effect size was small. There were also no significant difference in the number of relevant inefficient behaviours performed by stroke and stroke-ed participants compare to controls and the effect size was small. The stroke-ed group performed more irrelevant behaviours than the stroke and control groups, the ANOVA was significant F(2, 23) =

60 , p < 0.10, the Tukey's test demonstrated that the stroke-ed group did this significantly more often than the stroke group at the 0.05 level and the effect size was medium to large. Further investigation of results Three task specific and non-task specific behaviours that had the most variation were also analyzed to further understand their significance when comparing the three groups (see Table 4.5 below for findings on specific behaviour analyses). These included: 'participant asks staff for directions to a location', 'participants looks at/marks the map while walking' and 'participants goes to the 0.99 card rack first'. The first two were compared on the basis of frequency of the behaviour per participant in each group while the last compared the number of participants in each group who performed the behaviour. One non-task specific behaviour was 'participant asks staff for direction to a location' and the ANOVA was significant F(2, 23) = 4.11, p < 0.04, with the Tukey's test showing that the stroke group did this significantly more often than did the controls at the 0.05 level and the effect size demonstrated a large difference. Another non-task specific behaviour was compared to understand the importance of dual-tasking abilities and was labelled relevant efficient. This behaviour was 'walking and looking at/marking map'. The ANOVA analysis was significant F(2, 23) = 3.69, p < 0.03, the Tukey's test demonstrated that the stroke group did this significantly less often than did the controls at the 0.05 level and the effect size revealed a large difference. Lastly, the number of participants who went to '0.99 card rack first' were also compared. This behaviour was a task specific relevant efficient behaviour related to buying a birthday card task. The ANOVA analysis was significant F(2, 23) = 4.28, p < 0.02, the Tukey's test revealed that the stroke group did this significantly less often than the controls at the 0.05 level and the effect size was large for this difference. Table 4.5 Specific behaviours findings. Differences in mean frequency, SD, range and p values between stroke-ed, stroke and control groups on two behaviours Stroke-ED Specific Behaviours (n=3) Participant asks staff for directions to a location Participant walking and looks at/marks map NOTE: Values are mean ± SD (range) Significant at the p 0.10 for ANOVA analysis η 2 is the effect size Stroke (n=11) Controls (n=12) p η ± 2.31 (0-4) 3.27 ± 2.10 (0-7) 1.17 ± 1.40 (0-3) * ± 2.31 (0-4) 0.73 ± 1.10 (0-3) 3.75 ± 3.67 (0-11) *

61 54 It is interesting to note that compared to 9 out of 12 controls, only 2 out of 11 stroke participants and 1 out of 3 participants in the stroke-ed group went to the 0.99 card rack first. Further, it was observed that compared to 6 of the 12 control participants, 10 out of 11 stroke participants and 1 of the 3 participants in the stroke-ed group asked staff for directions to a location. Also, it was found that compared to 9 out of 12 controls, only 4 out of 11 stroke participants and 1 participant in the stroke-ed group were likely to attend to the map provided while walking during the test. 'Stroke only' vs. 'control only' behaviours Alderman et al. examined errors that were only observed in the brain injured group and those that were committed only by the control group. This led to a more sensitive analysis of the errors made on the MET. We also closely examined our data to isolate behaviours that were performed only by participants with stroke, stroke-ed and those executed only by controls. We grouped behaviours performed by participants in stroke and stroke-ed groups under 'stroke only' behaviours. Table 4.6 lists stroke only and controls only behaviours. These behaviours will be commented on in the discussion section. Table 4.6 'Stroke only' vs. 'control only' behaviours Behaviours demonstrated only by participants in the stroke group Asks for less than four stamps Asks for stamps at lotto booth Puts stamp on urgent letter collected at information desk to mail Licks self-adhesive stamp Asks non-staff member for Mars bar price Uses personal phone to call Katherine Waits for Katherine to call back at payphone Takes batteries from information desk Behaviours demonstrated only by participants in the stroke-ed group Asks for less than four stamps Behaviours demonstrated only by participants in the control group Buys Coke bottle instead of can Picks up phone at information desk but does not use it Tells examiner test is finished but continues to work on it Discussion The goal of this study was to investigate in more detail a wide range of behaviours performed by participants with stroke (with and without neuropsychologically defined ED) and controls as they worked on the BMET. To the best of our knowledge, this study was the first to employ an event recorder to code behaviours to understand the impact of ED on everyday activities.

62 55 Researchers working with the MET have analyzed the errors committed by the participants (tasks omitted, rules broken, tasks failed) as key markers of executive dysfunction. There have also been some preliminary analyses of strategies used (Andre et al., 2009). We proposed that to fully understand the impact of ED in everyday life, it is important to study the behaviours observed during the completion of the MET, including those directly related to task completion (e.g. 'participant buy 4 stamps'), as well as other behaviours that were less closely linked to a specific task (e.g. 'participant performs casual self-talk') on the BMET. Our goal was to perform an in-depth analysis of this fuller constellation of behaviours and to further classify them. As we were unable to find relevant classification in the literature, we developed one based on the notion that behaviours are undertaken as steps towards goal attainment (in this instance, completing the tasks on the MET without breaking any rules). We first separated the behaviours into task specific and non-task specific categories in which task specific behaviours were related to the completion of the 12 tasks on the BMET and non-task specific behaviours, which occurred any time during the test and were not related to a specific task. We further categorized the behaviours as either relevant efficient, relevant inefficient or irrelevant. Relevant efficient behaviours were those pertinent to achieving a task and yielded the most desirable result while relevant inefficient behaviours were those that did not yield the most desirable result. In contrast, irrelevant behaviours were those that were not related to achieving a task and included apparent habitual behaviours, rule breaks and distracters. The main findings of the study are as follows: the stroke group performed significantly more task specific relevant inefficient behaviours than stroke-ed and control groups; participants in stroke group performed significantly more non-task specific irrelevant behaviours than stroke-ed and control groups; there were no significant differences in the number of task specific relevant efficient and irrelevant behaviours performed between participants in stroke, stroke-ed and control groups; and there were no significant differences between the three groups on the number of non-task specific relevant efficient and relevant inefficient behaviours performed. Each of these findings is discussed below. There were no significant differences between the three groups on the number of relevant efficient task specific and non-task specific behaviours. Compared to the previous studies on the MET by Knight et al. and Alderman et al. who recruited participants with stroke that were either inpatients or outpatients at their rehabilitation centers, this study included participants who were

63 56 living in the community. Smaller differences may have been obtained in the current study compared to those of Knight et al. and Alderman et al. because the current individuals in stroke and stroke-ed groups were well adapted to their communities. Our stroke group was on average almost 10 years post-stroke and had been community-dwellers further adding to the likelihood that they would have well-developed community living skills. Although the stroke-ed group had had their stroke much more recently, they were both younger and had higher education. Each of these factors might have benefited their performance. There were no significant differences on task specific irrelevant behaviours and non-task specific relevant inefficient behaviours between the stroke, stroke-ed and control groups. This may be because we had a limited number of behaviours in these two categories that would discriminate: there were eight behaviours in the task specific irrelevant category and five behaviours in the non-task specific relevant inefficient category. Also, three out of eight task specific irrelevant behaviours were committed by one participant only (i.e. participant with stroke). Frequency of task specific relevant inefficient behaviours and non-task specific irrelevant behaviours differed between stroke, stroke-ed and control groups. Participants with stroke performed on average almost two times the number of relevant inefficient behaviours than controls did. For example, to complete the birthday card task, several of these participants purchased a card from the rack that contained regular priced cards as opposed to the ones from the 0.99 card rack. This allowed them to complete the task, however not efficiently. Another task on the BMET required participants "to collect something from the information desk and do what is necessary". As shown in Appendix B, the task sheet provided the name of the examiner at the bottom of the page labelled by an asterisk. When participants with stroke went to the information desk, a number of them did not ask using the name provided and instead simply asked if there was something available for the examiner. Also, one participant with stroke asked if there was something available to collect for himself. Some were eventually able to realize that they need to ask for something using the specific name of the examiner provided, however this indicated a lack of planning when initiating these tasks. Lastly, there were significant differences between the three groups on the number of non-task specific irrelevant behaviours, which included apparent habitual behaviours, rule breaks and distracters. Post hoc analysis revealed that the stroke-ed group performed more non-task

64 57 specific irrelevant behaviours than the stroke group. It appeared that participants in the stroke- ED group did not seek alternative ways to perform tasks compared to controls and may have not developed compensatory strategies like the stroke group, which may explain the high number of non-task specific irrelevant behaviours. This finding may also indicate problems with inhibition and the inability to monitor one's actions in participants in the stroke-ed group. It was found that participant in the stroke-ed group were also distracted by the environment and would stop to look at paintings and what was on the television in the patient waiting area. Furthermore, both stroke and stroke-ed groups had a high frequency of task-related as well as casual conversations with the examiner. It is natural and automatic to want to speak with the examiner, especially when one had questions. However, this was a rule break and it highlighted the inability of these participants to prevent themselves from breaking rules and altering their behaviours to fulfill the requirements of the test at hand. This lack of self-control, and the inability to self-monitor and inhibit oneself has been well documented as problems following ED (Alexander & Stuss, 2003; Minassian, Perry, Carlson, Pelham, & DeFilippis, 2003). Further investigation of results Frequency of use of three specific behaviours differed between the stroke, stroke-ed and control groups: going to the 0.99 card rack first, asking staff for directions to a location, and walking and looking/marking map. Going to the 99 card rack first can only be understood in the group of behaviours required to complete the birthday card task. If the participant went to the 0.99 card rack first, we categorized this as a task specific relevant efficient behaviour. There may be several reasons for this. First and foremost, the participants are only allowed to spend $7.50 to complete all tasks. This meant the fastest and most effective way to complete this task would be to go straight to the 0.99 card rack and choose a birthday card to purchase. Secondly, the participants are told that they are to finish the entire test in as little time as possible without rushing excessively. If participants go straight to the 0.99 card rack, they do not have to spend time comparing prices and looking for the cheapest card. Finally, since all participants in all three groups completed the birthday card task, this meant that it would not be enough to only look at task completion when assessing this task as it would not tell us enough about how the participants came about completing it. We found that most controls went straight to the 0.99 card rack, however very few participants in stroke group and only one of the three participants in stroke-ed group did this. They instead went to the other card rack which had regular priced

65 58 cards and picked up several cards and compared prices before choosing one to purchase, which meant they spent more time in achieving this task. This is still a correct way to complete the task without breaking any rules, however, it is not the most effective way. Some participants in stroke and stroke-ed groups did eventually go to the 0.99 card rack after asking for a less expensive card or when they noticed the rack themselves after having compared prices and cards at the other card rack. The 0.99 card rack was located on the far right while the other card rack was located closer to the entrance and came into the view first when participants entered the gift shop. It may be the case that participants in stroke and stroke-ed groups were less likely to consider alternatives and once they saw the other card rack, they did not go through the trouble of looking around the store for other options. Almost all of the participants in the stroke group and all of stroke-ed and control participants asked staff for task-related help, however compared to controls, participants with stroke more frequently asked staff for directions to a location. All the participants were provided with a map (see Appendix B) which showed all the locations on the first floor of Baycrest and where required items could be purchased and information could be obtained. It is interesting to note that participants with stroke relied on asking for help more often than consulting the map (see below). This may be a strategy that had proved beneficial to them in the past, which may have made them more likely to use it. At the same time, this behaviour may also be indicating a tendency towards disinhibition (Alderman et al., 2003) in which it is difficult for these participants to inhibit themselves from using the same strategy and consider alternatives. Asking for help has been closely examined by previous studies that used the MET. Both Knight et al. (2002) and Alderman et al. (2003) reported that acquired brain injured participants used and relied on this strategy more than their controls did. Alderman et al. (2003) also found that participants classified as task failers benefited more from asking for help than did participants classified as rule breakers. However, we did not observe such a pattern in our participants with stroke and stroke-ed. When it came to looking at/marking the map while walking, participants in stroke and stroke-ed groups did this much less frequently compared to their controls. Knight et al. (2002) also reported that their control group looked at the map more often. Looking at/marking the map while walking would be beneficial for completing the entire BMET because it not only helps with navigation to different locations, it also save time since two different behaviours are being

66 59 performed at once. Accordingly, this behaviour suggests different dual-tasking abilities in the three groups with the control group demonstrating the strongest dual-tasking abilities. As mentioned above, participants in stroke and stroke-ed groups relied heavily on asking the staff for directions to a location, and this strategy may have led them to not bother looking at the map. In addition, participants were provided the map on a clipboard but the map was placed behind the task sheet and the participants had to manually flip over the task sheet in order to look at the map. This may be another reason why stroke and stroke-ed participants were less likely to consult the map. However, it is important to note that a few participants with stroke and stroke- ED had hemiparesis and walked with the help of a cane (i.e. 2 participants in stroke group and 2 participants in stroke-ed group) and this may have hampered their ability to flip over to the map and mark it while walking. 'Stroke only' vs. 'controls only' behaviours Alderman et al. separated errors committed by brain injured participants from those made by controls and looked at them individually. We also wanted to see if there were certain stroke only and controls only behaviours in our sample. It was found that eight different behaviours were performed by participants in stroke group only, one behaviour by a participant in stroke-ed group only, while three behaviours were performed by controls only. On the basis of our classification, three out of the eight behaviours performed by participants in stroke group were irrelevant: licking self-adhesive stamp, waiting for Katherine to call back at payphone and taking batteries from the information desk. The other five were classified as relevant inefficient behaviours and all three behaviours performed by the controls were also relevant inefficient. Performing relevant inefficient behaviours did help the participants in stroke and stroke-ed groups achieve tasks, however they were not performed in the most efficient way. The three irrelevant behaviours are particularly interesting as they highlight problems in inhibition and lack of flexibility in these participants. Both licking a self-adhesive stamp and waiting for Katherine to call back at payphone were very unusual behaviours. At the same time, taking batteries from the information desk was also an atypical behaviour. During debriefing the participant said that she took the batteries because she was trying to complete the task which required her to collect something from the information desk and do what is necessary.

67 60 Bottari and Dawson (2011) reported that controls also make errors that may be qualitatively similar to those observed in the brain injured population, especially those with ED. In this study, we found that controls also performed behaviours that seemed out of the ordinary. For example, 'picking up the information desk phone and not using it', and 'telling the examiner that they have completed the test but then continuing to work on it', both suggest to a certain extent a lack of planning and task setting, impulsivity and distractibility, all of which are characteristics of ED (Alexander & Stuss, 2003). In addition, a Coke bottle is usually more expensive than a can and purchasing it would leave less money left to buy other items required in the test (for example birthday card, stamps). This may highlight a lack of monitored spending, which is another common problem related to the loss of monitoring abilities that may occur following ED. Recently, Bottari and Dawson (2011) analyzed whether clinicians were able to correctly attribute if specific isolated errors committed on the BMET were made by neurological participants or healthy controls. They found that only 55.6% of errors were attributed to the correct population (Bottari & Dawson, 2011). This speaks volume to the notion that a detailed analysis of the behaviours committed during the natural course of complex behaviour in an everyday setting is essential to make inferences about a participant's well being. Future Directions Future research using this behaviour classification may determine that participants in the stroke and stroke-ed groups would commit fewer task specific and non-task specific relevant efficient behaviours and more task specific and non-task specific relevant inefficient an irrelevant behaviours than controls. Some of these differences, however were not observed in this study possibly due to the limitations mentioned below. Study limitations The study is not without limitations. First of all, the study included a sample size that was both small and convenient and has previously been used for at least two other studies. For this reason, similar limitations can be found throughout these studies. Participants with stroke were several years post-injury and had been living in the community for many years. As a result, these participants may have adapted to their environments, which may have also impacted their performance on the BMET. The second limitation of this study is that lesion and stroke-severity data were not available for all participants with stroke, which meant that some may have had

68 61 strokes too mild to exhibit everyday problems. Another limitation of this study is that the quality of the audio and videotapes was average. It was sometimes difficult to understand what the participants were saying especially when they were speaking with a staff or a non-staff member. This is because of the background noise that was also captured in the videos. In addition, sometimes the videographer would film the participants from behind, which hindered our ability to code certain behaviours such as 'checking watch' or 'looking at task sheet or map'. Conclusions This exploratory study informed us that it is important to examine a wide range of behaviours, in addition to assessing performance on a priori behaviours, in order to better understand the impact of ED. Accordingly, our study included behaviours that represented inefficiencies, interpretation failures, rule breaks as well as other behaviours, which were not previously documented. To the best of our knowledge, this study was the first to employ an event recorder in order to document these behaviours. We presented a methodology to characterize these behaviours using our classification. We found that participants in stroke group performed significantly more task relevant inefficient behaviours than stroke-ed and control groups, while the stroke-ed group performed significantly more non-task specific irrelevant behaviours than the stroke-ed and control groups. In addition, we found that participants with stroke were significantly more likely to ask staff for directions to a location, and significantly less likely to go to the 0.99 card rack first and use the map while walking in comparison to controls. These results can be taken a step further in improving and establishing a behaviour classification system to better characterize ED and its impact on everyday activities. Moreover, these results can also play an important role in refining the BMET as a performance-based naturalistic assessment of ED.

69 62 Chapter 5 Discussion The main purpose of this study was to further our understanding of the impact of executive dysfunction (ED) on everyday activities in people with stroke. This is important because ED not only affects many aspects of daily life such as preparing a meal or shopping for groceries, it can also have devastating effects on people's ability to achieve successful community re-integration and social wellbeing (Grafman et al., 1996; Green, Kern, Braff, & Mintz, 2000). As described in Chapter 2, there are few ecologically valid assessments of ED and our knowledge of how ED impacts performance of everyday activities is very limited (Burgess et al., 2006). The Multiple Errands Test (MET) (Shallice & Burgess, 1991) is one such assessment that captures the situations of everyday life and provides an opportunity to examine participants' performance as they purchase certain items and collect specific information in a naturalistic setting like a hospital or a shopping mall. This thesis was undertaken to further examine the behaviours performed by people with stroke and matched controls as they completed the Baycrest Multiple Errands Test (BMET), a site specific version of the MET (Dawson et al., 2009). This would allow for a better discrimination between people with stroke and controls and further expand our understanding of the impact of ED in everyday life. An event recorder was selected as the method that would be best suited to identify a wide range of behaviours as they were occurring in a naturalistic setting. To the best of my knowledge, this methodology has not been previously used in relation to naturalistic assessment of ED. I have developed a classification system to analyze the behaviours performed and categorized them as either task specific or non-task specific and into the following three categories: relevant efficient, relevant inefficient or irrelevant behaviours. I found that participants with stroke performed significantly more task specific relevant inefficient behaviours and non-task specific irrelevant behaviours than controls. I also found significant discrimination between participants with stroke and controls on a number of specific behaviours such as 'going to the 0.99 card rack first', 'asking staff for directions to a location' and 'walking and looking at/marking map'. These results highlighted the importance of performing a detailed analysis of behaviours, which would serve as a valuable measure of ED in everyday life.

70 63 This chapter brings together the findings from the study I undertook with the literature review provided in Chapter 2. First, it describes the theories of executive functions, followed by the importance of real world assessments and behaviour analysis, and suggestion to improve the BMET for future clinical and research use. Also, limitations of my study and suggestions on how to improve future research on the impact of ED on everyday performance is discussed. In the end, summary and conclusions to the entire thesis is presented. Theories of executive functions Six theories and models of executive functions were reviewed in Chapter 2: Duncan's theory of goal neglect, the adaptive coding model, Mesulam's default mode, Grafman's structured event complex framework, Norman and Shallice's supervisory attentional system and Stuss and colleagues' fractionation of the supervisory system. Duncan's theory of goal neglect (Duncan, 1986; Duncan, Emslie, Williams, Johnson, & Freer, 1996) emphasizes the importance of goals in human behaviour and suggests that damage to the prefrontal lobes can have an impact on goal formulation, goal selection and goal monitoring (Turner & Levine, 2004). This is because the main function of the prefrontal lobes is to organize and govern actions in accordance with desired goals. In the adaptive coding model, the prefrontal lobes are viewed as a global, adaptive unit (Duncan & Miller, 2002). Duncan and Miller (2002) suggested that the prefrontal lobes may not have defined regions that mediate specific functions; instead the prefrontal lobes function more generally and adapt to solve various task demands and cognitive problems. Mesulam (2002) proposed that the main role of the prefrontal lobes is to overcome the default mode in which actions are driven by automatic reactions and immediate need to achieve satisfaction without consideration of contextual feedback and experience. This is achieved with the help of executive processes that allow the individual to consider alternatives and act in more flexible ways (Turner & Levine, 2004). In contrast, Grafman (1995) explained that different regions of the prefrontal lobes store different features about a set of events or actions and damage to these regions would lead to impairments in everyday life (Grafman et al., 1996). Norman and Shallice (1986) suggested that the prefrontal lobes have a supervisory role that is involved when attention is required for planning, decision making, multitasking, and in situations that are novel and require inhibition of habitual responses. These abovementioned theories are valuable; however this study was primarily influenced by Stuss and colleagues' fractionation of the supervisory

71 64 system (Stuss, Shallice, Alexander, & Picton (1995); Suss, 2006; Stuss & Alexander, 2007) (which stems from Norman and Shallice's supervisory attentional system). Stuss and colleagues conducted a series of lesion and neuroimaging studies and demonstrated evidence of fractionation within the supervisory system (Stuss & Alexander, 2007; Stuss et al., 1995). Two of the frontal processes identified by Stuss and colleagues were most relevant in this study: task setting and monitoring. Task setting, which is related to the left lateral prefrontal lobe, refers to the ability to set a stimulus-response relationship requiring formation of a criterion, and continuous adjustment and organization of schemata necessary to complete tasks (Stuss & Alexander, 2007). Monitoring, which is related to the right lateral frontal lobe, refers to the ability to check the task over time to ensure quality control by keeping track of timing of the activity, detecting occurrences of errors, and modulating actions to overcome discrepancies (Stuss & Alexander, 2007). Having the knowledge of these two processes helped identify and note specific behaviours. For example, behaviours related to task setting which have not been noted in previous studies using the BMET included 'going to the 0.99 card rack first' as it highlighted participants' ability to set a stimulus-response relationship in which they recognized the need to buy a card (i.e. formation of a criterion) and spend as little as possible (i.e. organization of schema) to complete the birthday card task. In contrast, problems in task setting abilities were demonstrated when participants 'asked for less than four stamps' to purchase as it demonstrated that they were unable to form a criterion correctly since they failed to understand that the task required participants to purchase four stamps. Behaviours that reflected monitoring included 'checking watch' as it highlighted the participants' ability to understand the timing aspect of monitoring, while problems in monitoring were seen in behaviours such as 'arriving too early or too late at the parrot cage' because these participants were unable to recognize the importance of keeping track of time to ensure that they are able to accurately complete the task of "arriving at the parrot cage in 10 minutes". It is important to note that some behaviours that were observed were a combination of task setting and monitoring abilities, for example, when participant 'told the examiner that s/he finished the test but continued to work on it' or when participant 'called Katherine and waited for her to call back at payphone'. These behaviours demonstrated problems in task setting as well as monitoring since the participants who performed these behaviours were unable to set an appropriate stimulus-response relationship at the beginning of the task and adjust their behaviours accordingly while they carried out the task.

72 65 It is also unclear as to how some of the irrelevant behaviours that were observed such as 'licking self-adhesive stamp' or 'taking batteries from the information desk' would fit with Stuss and colleagues' division of task setting and monitoring abilities. It was nonetheless necessary to understand these theories and models of executive function since a significant part of my study was devoted to the development of a classification system to better understand the behaviours performed during the BMET. These theories, in particular Stuss and colleagues' division of task setting and monitoring abilities, provided a basis as I was unable to find relevant classification in the literature. However, as mentioned above, there was a lack of agreement with Stuss and colleagues' model since I observed behaviours that were likely a combination of difficulties in task setting and monitoring. The importance of real world assessments and behaviour analysis In Chapter 2, three traditional measures were presented and critiqued on the basis of their ecological validity (the degree to which findings in a test is related to those observed in a realworld setting (Chaytor and Schmitter-Edgecombe, 2003)). Also, nine different real world, performance-based assessments and questionnaires, in addition to the MET were described and the need for them in both research and clinical setting was discussed. The findings in this study support the need for real world, performance-based assessments as significant differences in the types and frequencies of behaviours performed between participants with stroke and those committed by controls were observed. The situations encountered in the BMET do relate to a significant extent with those found in the real world (see results on ecological validity in Dawson et al., 2009) as they include tasks such as shopping and collecting information in a naturalistic setting. The authors of this test argue that in order to examine the impact of ED, it is important to have participants work on multiple tasks over extended periods of time, without continuous feedback from the examiner (Shallice & Burgess, 1991). Although the MET is based on the principle that individuals with damage to the prefrontal lobes may be specifically impaired in everyday situations that require executive abilities such as planning and multitasking (Bottari & Dawson, 2011; Chan, Shum, Toulopoulou, & Chen, 2008), this study also highlights the value of the MET over and above the assessment of ED. The

73 66 detailed analysis of behaviours allowed the examination of apparent habitual behaviours that were performed by participants with stroke and controls such as 'licking the self-adhesive stamps', 'casual self-talk' and 'casual talk with staff/non-staff/parrot/others'. It can be argued that these behaviours may have occurred because the BMET encourages participants to carry out the tasks in any order and the instructions are intentionally designed to be undefined and illstructured to observe how participants would resolve these situations using their own judgment and experiences. These behaviours were also likely to occur as the situations encountered in the BMET were like those that occur in everyday life. In addition, the authors of another real world performance-based assessment namely, the Cooking Task (Chevignard et al., 2000) suggested that real world assessments should include distracting materials to ensure an ill-structured and a more lifelike environment. Since the MET occurs in a naturalistic, real world setting like a hospital or a shopping mall, distracters are already present in the environment. In this instance, behaviours that were observed related to distracters were 'looking at paintings', 'looking at volunteer display rack', 'looking at patient area', 'looking at television in patient area' and 'drinking water from water fountain near information desk'. Examining these behaviours may help understand the problems in distractibility, inhibition and the inability to self-monitor one's actions. This study also highlights the importance of undertaking a more detailed analysis of behaviours performed, which has not been done previously. This can be emphasized by discussing the birthday card task of the BMET as an example. All participants (controls and those with stroke) bought a birthday card successfully. While this would typically be scored on the BMET as completing the task, this more in-depth analysis showed that participants with stroke did not complete the task as efficiently and smoothly as control participants. Using an event recorder assisted in taking into account the behaviours performed in relation to this task and my classification further helped understand them in the context of BMET and real world. It was found that compared to controls, participants with stroke were more likely to purchase the birthday card from the rack that contained regular priced cards as opposed to the one at the 0.99 card rack. In addition, some participants looked at several cards and read messages inside cards before choosing one to purchase, while others asked the staff for a cheaper card. Therefore, if behaviours are examined in a more detailed manner, it may highlight problems in various aspects of executive functioning such as lack of planning, decision making or monitoring abilities.

74 67 By performing an in-depth analysis of behaviours, we can understand which characteristics of EF need to be targeted during rehabilitation. For example, participants that demonstrated more inefficient behaviours related to the lack of monitoring abilities can be trained to pay close attention to checking their behaviours for accuracy and adjusting them to better fit the demands of an ongoing task. Moreover, external aids such as the use of a watch can be stressed to ensure quality control of the timing of activities. Similar behaviour analyses could also be performed for other real-world assessments such as the Cooking Task (Chevignard et al., 2000) or the Instrumental Activities of Daily Living Profile (Bottari, Dassa, Rainville, & Dutil, 2009a; 2009b), which would further inform rehabilitation techniques. Improving the BMET Although the purpose of the study was not to identify possible refinements to the BMET, the results of our work have implications for both the clinical and research utilization of the MET and BMET. During the course of this study, it became apparent that many participants (stroke and control) misunderstood two rules and had trouble with three of the tasks. The three tasks however, did not discriminate between participants with stroke and controls in this study, as well as in previous study (Dawson et al., 2009) that used the same sample of participants. This section discusses clarification of these tasks and rules to further enhance the ecological validity of the MET and to improve discrimination between healthy controls and people with ED to highlight the impact of ED on everyday tasks. One rule that was misunderstood was "You should not go back into an area you have already been in". This rule was included in the MET to encourage participants to plan their routes so that they could not simply do the test task by task as they were not allowed to use areas such as the gift shop or the information desk more than once. However, some participants understood this rule to mean they were not to use the same hallway more than once. Some tried to navigate to different locations on the main floor using different hallways and walked into prohibited areas. If this rule can be reworded to specify what it is intending, it would ensure that all participants are completing the test without being misunderstood. It can be reworded as "You may not enter the same location you have already been in (e.g. Resident's library)" as it may have been that the word 'area' that had caused the misunderstanding. Also, providing an example may further help clarify the rule for the participants.

75 68 The task of 'collecting something from the examiner from the information desk and doing what is necessary' also caused confusion in some participants. A few thought that they could collect anything from the information desk and one participant with stroke took batteries. Others had a hard time 'doing what is necessary with the letter collected' and kept the letter with them. Still, others gave the letter to the examiner at the end of the test even though it said 'urgent' on the envelope, however these behaviours did not discriminate between participants with stroke and controls. This task seems more artificial considering the other tasks on the BMET because in the real world if the envelope was to be delivered to the examiner urgently, it would be highly unlikely to have been sitting at the information desk. The intent of the task, at least in part, seems to be to provide an interruption to the planning participants might have done up to that point in the test. Thus, the task may be modified to look more like an interruption that would occur in a real world setting, which in turn may improve the ecological validity of the test. For example, the examiner could hand the participant a note stating, "We forgot to tell you this earlier, but you also need to buy a bag of chips." These types of interruptions occur in everyday life when we receive a call or a text message from our spouses or parents telling us to also purchase a specific item which they had forgotten to include on the grocery list. This would allow us to analyze the participants' ability to reorganize their plans in the face of the interruption, and their task setting, decision making, and monitoring abilities, all of which are important characteristics of ED. I would hypothesize that control participants would have less difficulty adjusting to the interruption relative to people with ED. Another task that caused problems for the participants working on the BMET was 'meeting the examiner at the parrot cage 10 minutes after they had started the test and telling the examiner the time'. The instructions on this task were deliberately set to be a bit confusing to see how participants would complete this task because if the instructions are too clear and concrete, the test may not be sensitive to ED. Some participants rushed through the test thinking they had to complete the entire exercise in 10 minutes to meet the examiner at the parrot cage. Also, the rule of 'taking as little time as possible to complete the exercise' may have further strengthened this misconception. Others took their time working on the test and met the examiner at the end of the exercise while paying no particular attention to the time that had elapsed. Some met the examiner at the parrot cage but did not state the time. It was surprising to see this confusion in both participants with stroke and controls especially when they linked the task with the rule during

76 69 briefing. Some rewording of the task and rule might prevent misunderstandings related to this task and better discriminate between control participants and people with ED in future. Also, this task can perhaps be modified to be more representative of everyday situations by providing the participants with an envelope of some sort and asking them that they would have to meet someone at the parrot cage at a specific time to deliver it, as this may enhance the ecological validity of the test. The task requiring participants to report the number of entrances/exits on the main floor of Baycrest caused problems during coding. Some participants asked a staff member to tell them the number of entrances/exits and this was easy to code, however it was difficult to code whether others used the map to count them. The examiner may have also missed this behaviour if the participant had done this while walking and were away from the examiner. Perhaps modifying the task to report which exit is closest to a particular location may eliminate this inconsistency. Further, this would be more representative of a real-world situation than reporting the number of entrances/exits on the main floor. Making these suggested changes to the BMET would further make the assessment more representative of everyday life and allow for better discrimination between controls and people with ED. This is because these tasks caused misunderstanding in both participants with stroke and controls, which may have impacted their performance. This may also explain why Bottari and Dawson (2011) found that only 55.6% of errors in the standard scoring of the MET were attributed to the correct population by clinicians who watched short video clips of participants with acquired brain injury and controls demonstrating single performance errors. Bottari and Dawson (2011) emphasized that clinicians need to be more cautious and aware of the risk of misinterpreting single behaviours demonstrating errors in the real world. This study took into account a wide range of behaviours and highlighted the importance of performing an in-depth analysis of behaviours in order to better understand the impact of ED. In addition, Rand, Basha- Abu Rukran, Weiss, and Katz (2008) suggested that BMET may also be used to assess ED following an intervention or a rehabilitative training. Although Rand et al. (2008) were deliberating on the virtual version of the MET, improvement in EF may also be examined using the BMET. This is especially the case since Rand et al. (2008) found that real-world version of the MET was better than the virtual one because it allowed observing social mistakes such as ignoring that there was a line up ahead when paying for items purchased.

77 70 Limitations One of the main limitations of this study was that it was a secondary analysis that used participant videos not originally meant for the purpose of this study. This resulted in difficulty characterizing some behaviours due to limitations of the audio and video quality. In several instances, it was difficult to understand what the participants were saying when they were speaking to the staff and non-staff members. This was important because I wanted to discriminate between casual and task related conversations as it would highlight planning, task setting, multitasking and monitoring abilities. Similarly, it was especially difficult to comprehend when participants engaged in self-talk as the built-in microphone in the camera was not always close to the participant to capture what was said. There was also a lot of background noise which was captured in the videos (for example, piano being played for the patients near the gift shop, people talking amongst themselves in the gift shop or the cafeteria, parrot squawking) and some times, the participants spoke too softly, which further limited our ability to examine their talking behaviours. Moreover, the videographer would sometimes film from behind the participants because s/he would not be able to keep up with them or someone passing by would come in front of the camera. This would restrict our ability to code various behaviours. However, it was understood that some of these difficulties would be expected while filming a video in a naturalistic, uncontrolled setting like the main floor of a hospital. The study used a relatively small sample size, particularly the stroke-ed group. A small sample size has a greater probability that the results obtained may be due to chance (i.e. the magnitude of the results may be overestimated) or that a type II error was made (i.e. failing to reject the null hypothesis when the null hypothesis is false) (Hackshaw, 2008). There was also lack of lesion data and medical histories available. This meant that we could not examine the impact of ED in relation to lesion location, which would have helped strengthen the association between the categorization of certain behaviours and theories of executive functions, for example, the relationship between damage to the left lateral frontal lobe and poor task setting behaviours. In addition, as mentioned in chapter 4, it may be that some participants with stroke had conditions too mild to demonstrate ED. The participants with stroke were many years post-stroke and may have developed compensatory strategies that helped them in their daily life. This, in turn may have impacted their performance on the BMET.

78 71 This study was unique to use an event recorder to code the behaviours captured in the videos. However, there were several limitations of the Behaviour Tracker. The Behaviour Tracker permitted one configuration file to include a maximum of 37 events. As a result, we had to create more configuration files and run multiple instances of Behaviour Tracker using separate configuration files. Due to this limitation, once the coding was completed the spreadsheets from each instance had to be merged manually into a single spreadsheet. In addition, although Behaviour Tracker allowed coding of duration of specific behaviours, considerable variability in coding duration between computers and platforms was found (Windows or MAC). Thus, the reliability of the duration coding was very low. As a result, the duration of behaviours was not included in the study. If it had been possible to code the duration of behaviours accurately, perhaps we would have been able to see more discrimination among the participants with stroke and controls. Another limitation of the Behaviour Tracker was that if a behaviour was missed, during the original coding of behaviours, we had to manually enter this behaviour once the data was exported to the spreadsheet. The software did not allow entering the missed behaviour in the editor mode. Since this was a secondary analysis, some of the abovementioned limitations could not be avoided. Ideally, participants with stroke that would be recruited would have documented ED which would be separated on the basis of the number of years post-stroke as participants with stroke that had documented ED in this study had their stroke more recently than the participants in the stroke group without neuropsychologically defined ED. These participants would be matched with their controls on age, gender and education like this study. It would be feasible to include more participants and controls (i.e. a higher 'n') to support the study. The idea of filming participants as they complete the BMET would still be important as it would permit further analyses to be performed. However, the use of more sophisticated equipment would be warranted to enhance accuracy of behaviour coding. For example, head cameras and eye tracking technology, which detects a person's gaze would be used to enhance audio and video quality, and eliminate background noise. This would also eliminate the need of a videographer and examiner to walk closely behind the participant and permit the examiner to take notes from a distance and only come close to the participant when it is part of the test (for example, when he/she needs to give an interrupting message).

79 72 Future directions Future research should address the limitations described earlier in this chapter to further our understanding of the impact of ED on everyday activities in participants with stroke, especially if the BMET or some variation of the MET is used as the measure of ED. Modification to the rules and tasks of the BMET would allow the test to be more representative of everyday life. In relation to this study, future research should consider refining the behaviour classification we proposed and as mentioned earlier, this type of detailed behaviour analysis can also be incorporated in other naturalistic assessments of ED such as the Cooking Task (Chevignard et al., 2000) or the Instrumental Activities of Daily Living Profile (Bottari et al., 2009a; 2009b). This type of behaviour analysis also has important clinical implications as information obtained from this study can pave way for the development of more targeted rehabilitation programs. Summary & conclusions This research study explored a wide range of behaviours performed by participants with stroke and their matched controls as they completed the BMET to further understand the impact of ED on everyday activities. An event recorder was used to code the occurrences and frequencies of behaviours. To the best of my knowledge, this methodology has not been previously used in relation to naturalistic assessment of ED following stroke. A classification system was developed to characterize these behaviours. It was found that participants with stroke performed significantly more task specific relevant inefficient behaviours and non-task specific irrelevant behaviours compared to controls. Specific behaviours were also analyzed and it was found that participants with stroke were significantly more likely to ask staff for directions to a location, and significantly less likely to go to the 0.99 card rack first and use the map while walking in comparison to controls. In summary, this study served as an example to highlight the importance of carrying out an in-depth analysis of behaviours performed. These results are promising and support the need for future investigation of a wide range of behaviours, in addition to assessing performance errors, inefficiencies, interpretation failures and rule breaks on the BMET to better understand the impact of ED on everyday life.

80 73 References Alderman, N., Burgess, P. W., Knight, C., & Henman, C. (2003). Ecological validity of a simplified version of the multiple errands shopping test. Journal of the International Neuropsychological Society, 9, Alexander, M. D., & Stuss, D. T. (2003). Executive disorders. In J. Noseworthy (Ed.), Neurological Therapeutics, Principles and Practice, Vol. 2 (pp ). London: Martin Dunitz. Allain, P., Le Gall, D., Etcharry-Brouyx, F., Aubin, G., & Emile, J. (1999). Mental representation of knowledge following frontal-lobe lesion: dissociations on tasks using scripts. Journal of Clinical and Experimental Neuropsychology, 21, Anderson, C. V., Bigler, E. D., & Blatter, D. D. (1995). Frontal lobe lesions, diffuse damage, and neuropsychological functioning of traumatic brain-injured patients. Journal of Clinical and Experimental Neuropsychology, 17, Andre, M., Anderson, N. D., Stuss, D. T., & Dawson, D. R. (2009). Executive dysfunction and strategy use during completion of the Multiple Errands Test. Neuroscience Meeting Planner Program No Chicago, IL: Society for Neuroscience, Online. Barceló, F., Escera, C., Corral, M. J., & Periañez, J. A. (2006). Task switching and novelty processing activate a common neural network for cognitive control. Journal of Cognitive Neuroscience, 18, Barkley, R. (1997). Behavioural inhibition, sustained attention, and executive functions: Constructing a unifying theory. Psychological Bulletin, 121, Baum, C. M., Connor, L. T., Morrison, T., Hahn, M., Dromerick, A. W., & Edwards, D. F. (2008). Reliability, validity, and clinical utility of the Executive Function Performance Test: a measure of executive function in a sample of people with stroke. The American Journal of Occupational Therapy, 62(4),

81 74 Baum, C. M., & Edwards, D. (1993). Cognitive performance in senile dementia of the Alzheimer s type: The kitchen task assessment. American Journal of Occupational Therapy, 47, Bechara, A., Damásio, A. R., Damásio, H., Anderson, S. W. (1994). Insensitivity to future consequences following damage to human prefrontal cortex.cognition 50 (1-3), Behaviour Tracker. (2003). Behaviour Tracker (Version 1.5). Available from Bennett, P. C., Ong, B., & Ponsford, J. (2005). Measuring executive dysfunction in an acute rehabilitation setting: Using the dysexecutive questionnaire (DEX). Journal of the International Neuropsychological Society, 11, Benton, A., Hamsher, K, & Sivan, A. (1994). Multilingual Aphasia Examination (3rd ed.). San Antonio, TX: Psychological Corporation. Bogod, N. M., Mateer, C. A., & MacDonald, S. W. S. (2003). Self-awareness after traumatic brain injury: A comparison of measures and their relationship to executive functions. Journal of the International Neuropsychological Society, 9, Bottari, C., Dassa, C., Rainville, C., & Dutil, E. (2009a). The criterion- related validity of the IADL Profile with measures of executive functions, indices of trauma severity and sociodemographic characteristics. Brain Injury, 23, Bottari, C., Dassa, C., Rainville, C., & Dutil, E. (2009b). The factorial validity and internal consistency of the Instrumental Activities of Daily Living Profile in individuals with a traumatic brain injury. Neuropsychological Rehabilitation, 19, Bottari, C., Dassa, C., Rainville, C., & Dutil, E. (2010). The IADL Profile: Development, content validity, intra- and interrater agreement. Canadian Journal of Occupational Therapy, 77(2), Bottari, C., Swaine, B., &Dutil, E. (2007). Interpreting activity of daily living errors for treatment and discharge planning: The perception of occupational therapists. Journal of Head Trauma Rehabilitation, 22(1),

82 75 Bottari, C., & Dawson, D. R. (2011). Executive functions and real-world performance: How good are we at distinguishing people with acquired brain injury from healthy controls? OTJR: Occupation, Participation and Health, 31(1), S61-S68. Bowie, C. R., & Harvey, P. D. (2006). Administration and interpretation of the Trail Making Test. Nature Protocols, 1(5), Braswell, D., Hartry, A., Hoornbeek, S., Johansen, A., Johnson, L., Schultz, J., Sohlberg, M. M. (1992). Profile of executive control system. Puyallup, WA: Association for Neuropsychological Research and Development. Bryan, J., & Luszcz, M. A. (2000). Measurement of executive function: Considerations for detecting adult age differences*. Journal of Clinical and Experimental Neuropsychology, 22(1), Burgess, P. W. (2000). Strategy application disorder: the role of the frontal lobes in human multitasking. Psychological Research, 63, Burgess, P. W., Alderman, N., Evans, J., Emslie, H., & Wilson, B. A. (1998). The ecological validity of tests of executive function. Journal of the International Neuropsychological Society, 4, Burgess, P. W., Alderman, N., Forbes, C., Costello, A., Coates, L. M., Dawson, D. R., et al. (2006). The case for the development and use of "ecologically valid" measures of executive function in experimental and clinical neuropsychology. Journal of the International Neuropsychological Society, 12, Burgess, P. W., & Robertson, I. H. (2002). Principles of the rehabilitation of frontal lobe function. In D. T. Stuss & R. T. Knight (Eds.), Principles of frontal lobe function (pp ). New York, NY: Oxford University Press. Burgess, P. W., Veitch, E. J., Costello, A. de L., & Shallice, T. (2000). The cognitive and neuroanatomical correlates of multitasking. Neuropsychologia, 38, Boyd, T. M. & Sautter, S. W. (1993). Route-finding: A Measure of Everyday Executive Functioning in the Head-injured Adult. Applied Cognitive Psychology, 7,

83 76 Cabeza, R., & Nyberg, L. (2000). Imaging cognition II: an empirical review of 275 PET and fmri studies. Journal of Cognitive Neuroscience, 12, Chan, R. C. K., Shum, D., Toulopoulou, T., & Chen, E. Y. H. (2008). Assessment of executive functions: Review of instruments and identification of critical issues. Archives of Clinical Neuropsychology, 23, Chaytor, N., & Schmitter-Edgecombe, M. (2003). The ecological validity of neuropsychological tests: A review of the literature on everyday cognitive skills. Neuropsychology Review, 13(4), Chaytor, N., Schmitter-Edgecombe, M., & Burr, R. (2006). Improving the ecological validity of executive functioning assessment. Archives of Clinical Neuropsychology, 21, Chevignard, M., Pillon, B., Pradat-Diehl, P., Taillefer, C., Rousseau, S., Le Bras, C., & Dubois, B. (2000). An ecological approach to planning dysfunction: Script execution. Cortex, 36, Chevignard, M. P., Taillefer, C., Picq, C., Poncet, F., Noulhiane, M., & Pradat-Diehl, P. (2008). Ecological assessment of the dysexecutive syndrome using execution of a cooking task. Neuropsychological Rehabilitation, 18(4), Cicerone, K., Levin, H., Malec, J., Stuss, D., & Whyte, J. (2006). Cognitive rehabilitation interventions for executive function: Moving from bench to bedside in patients with traumatic brain injury. Journal of Cognitive Neuroscience, 18, Dawson, D. R., Anderson, N., Burgess, P., Cooper, E., Krpan, K. M., & Stuss, D. T. (2009). Further development of the Multiple Errands Test: Standardized scoring, reliability and ecological validity for the Baycrest Version. Archives of Physical Medicine & Rehabilitation, 90(11), S41-S51. Dawson, D. R., Anderson, N., Burgess, P. W., Levine, B., Rewilak, D., Cooper, E. K., Farrow, S., Koscik, K. M., Krpan, K. M., Lo, A., Peer, M., & Stuss, D. T. (2005a). The ecological validity of the Multiple Errands Test Hospital Version: Preliminary findings. Poster

84 77 presented at Meeting of the International Neuropsychological Society, St. Louis, USA, February. Demakis, G. J. (2003). A meta-analytic review of the sensitivity of the Wisconsin Card Sorting Test to frontal and lateralized frontal brain damage. Neuropsychology, 17, Duncan, J. (1986). Disorganization of behaviour after frontal lobe damage. Cognitive Neuropsychology, 2, Duncan, J. (1995). Attention, intelligence and the frontal lobes. In M. S. Gazzaniga (Ed.), The cognitive neurosciences (pp ). Cambridge, MA: MIT Press. Duncan, J. (2001). An adaptive coding model of neural function in prefrontal cortex. Nature Reviews Neuroscience, 2, Duncan, J., Emslie, H., Williams, P., Johnson, R., & Freer, C. (1996). Intelligence and the frontal lobe: The organization of goal-directed behaviour. Cognitive Psychology, 30, Duncan, J., & Miller, E.K. (2002). Cognitive focus through adaptive neural coding in the primate prefrontal cortex. In D.T. Stuss & R.T. Knight (Eds.), Principles of frontal lobe function (pp ). New York, NY: Oxford University Press. Duncan, J., & Owen, A. M. (2000). Common regions of the human frontal lobe recruited by diverse cognitive demands. Trends in Neuroscience, 23, Duncan, J., Seitz, R. J., Kolodny, J., Bor, D., Herzog, H., Ahmed, A., Emslie, H. (2000). A neural basis for general intelligence. Science, 289, Dutil, E., Bottari, C., Vanier, M., & Gaudreault, C. (2005). ADL Profile: Description of the instrument (H. Scott & C. Bottari, Trans. 4th ed., Vol. 1). Montreal: Les Éditions Émersion. Dutil, E., Forget, A., Vanier, M., & Gaudreault, C. (1990). Development of the ADL Profile: An evaluation for adults with severe head injury. Occupational Therapy in Health Care, 7, 7-22.

85 78 Elliot, R. (2003). Executive functions and their disorders. British Medical Bulletin, 65, Franzen, M. D., and Wilhelm, K. L. (1996). Conceptual foundations of ecological validity in neuropsychology. In Sbordone, R. J. & Long, C. J. (Eds.), Ecological Validity of Neuropsychological Testing (pp ). Delray Beach, FL: GR Press/St. Lucie Press. Garlinghouse, M. A., Roth, R. M., Isquith, P. K., Flashman, L. A., & Saykin, A. J. (2010). Subjective rating of working memory is associated with frontal lobe volume in schizophrenia. Schizophrenia Research, 120, Gilbert S. J.,& Burgess, P. W. (2008). Executive function. Current Biology, 18(3), R110-R114. Gillen, G. (2008). Managing executive function impairments to optimize function. Cognitive and perceptual rehabilitation optimizing function. Cognitive and Perceptual Rehabilitation: Optimizing Function (pp ). St. Louis: Mosby Elsevier. Giovagnoli, A. R. (2001). Relation of sorting impairment to hippocampal damage in temporal lobe epilepsy. Neuropsychologia, 39, Gladsjo, J., Schuman, C., Miller, S., & Heaton, R. (1999). Norms for letter and category fluency: Demographic corrections for age, education, and ethnicity. Odessa, FL: PAR. Godbout, L., Grenier, M. C., Braun, C. M., & Gagnon, S. (2005). Cognitive structure of executive deficits in patients with frontal lesions performing activities of daily living.brain Injury, 19, Gonzalez-Hernandez, J. A., Cedeno, I., Pita-Alcorta, C., Galan, L., Aubert, E., & Figueredo- Rodriguez, P. (2003). Induced oscillations and the distributed cortical sources during the Wisconsin card sorting test performance in schizophrenic patients: New clues to neural connectivity. International Journal of Psychophysiology, 48, Gonzalez-Hernandez, J. A., Pita-Alcorta, C., Cedeno, I., Bosch-Bayard, J., Galan-Garcia, L., Scherbaum, W. A., Figueredo-Rodriguez, P. (2002). Wisconsin Card Sorting Test synchronizes the prefrontal, temporal and posterior association cortex in different frequency ranges and extensions. Human Brain Mapping, 17,

86 79 Goverover, Y., Kalmar, J., Gaudino-Goering, E., Shawaryn, M., Moore, N. B., Halper, J.,...DeLuca, J. (2005). The relation between subjective and objective measures of everyday life activities in persons with multiple sclerosis. Archives of Physical Medicine and Rehabilitation, 86, Grafman, J. (1995). Similarities and distinctions among current models of prefrontal cortical functions. Annals of the New York Academy of Sciences, 769, Grafman, J., Schwab, K., Warden, D., Pridgen, A., Brown, H. R.,& Salazar, A. M. (1996). Frontal lobe injuries, violence, and aggression: A report of the Vietnam Head Injury Study. Neurology, 46, Green, M. F., Kern, R. S., Braff, D. L., & Mintz, J. (2000). Neurocognitive deficits and functional outcome in schizophrenia: Are we measuring the right stuff? Schizophrenia Bulletin, 26(1), Heaton, R., Chelune, G., Talley, J., Kay, G., & Curtiss, G. (1993). Wisconsin Card Sorting Test manual: revised and expanded. Odessa, FL: Psychological Assessment Resources. Henry, J. D. & Crawford, J. R. (2004a). A meta-analytic review of verbal fluency performance following focal cortical lesions. Neuropsychology, 18, Hughes, C., & Graham, A. (2002). Measuring executive functions in childhood: Problems and solutions. Child and Adolescent Mental Health, 7, Igarashi, K., Oguni, H., Osawa, M., Awaya, Y., Kato, M., Mimura, M., Kashima, H. (2002). Wisconsin card sorting test in children with temporal lobe epilepsy. Brain & Development, 24, Josman, N., & Birnboim, S. (2001). Measuring kitchen performance: What assessment should we choose? Scandinavian Journal of Occupational Therapy, 8, Jurado, M. B., & Rosselli, M. (2007). The Elusive nature of executive functions: A review of our current understanding. Neuropsychological Review, 17, Kadafar, K. (2003). John Tukey and Robustness. Statistical Science, 18(3),

87 80 Katz, N., Tadmor, I., Felzen, B., & Hartman-Maeir, A. (2007). Validity of the Executive Function Performance Test in Individuals with Schizophrenia. Occupational Therapy Journal of Research, 27, Keil, K., & Kaszniak, A. W. (2002). Examining executive function in individuals with brain injury: A review. Aphasiology, 16(3), Kingstone, A., Smilek, D., Birmingham, E., Cameron, D., & Bischof, W. F. (2005). Cognitive ethology: Giving real life to attention research. In J. Duncan, L. Phillips, & P. McLeod (Eds.), Measuring the mind: Speed, control and age (pp ). Oxford, UK: Oxford University Press. Kingstone, A., Smilek, D., & Eastwood, J. D. (2008). Cognitive ethology: A new approach for studying human cognition. British Journal Psychology, 99(Pt. 3), Knight, C., Alderman, N., & Burgess, P. W. (2002). Development of a simplified version of the multiple errands test for use in hospital settings. Neuropsychological Rehabilitation, 12(3), Koechlin, E., Corrado, G., Pietrini, P., & Grafman, J. (2000). Dissociating the role of the medial and lateral anterior prefrontal cortex in human planning. Proceedings of the National Academy of Sciences, 97, Kvavilashvili, L. & Ellis, J. (2004). Ecological validity and the real-life/laboratory controversy in memory research: A critical and historical review. History & Philosophy of Psychology, 6, Lamberts, K. F., Evans, J. J., & Spikman, J. M. (2010). A real-life, ecologically valid test of executive functioning: The executive secretarial task. Journal of Clinical and Experimental Neuropsychology, 32(1), Levine, T. R., & Hullett, C. R. (2002). Eta squared, partial eta squared, and misreporting of effect size in communication research. Human Communication Research, 28(4), Levine, B., Turner, G. R., & Stuss, D. T. (2008). Rehabilitation of frontal lobe functions. In D. T. Stuss, G. Winocur & I. H. Robertson (Eds.), Cognitive Neurorehabilitation, 2nd

88 81 Edition: Evidence and Application (pp ). Cambridge: Cambridge University Press. Lewis, M. W., Babbage, D. R., & Leathem, J. M. (2011). Assessing executive performance during cognitive rehabilitation. Neuropsychological Rehabilitation, 21(2), Lezak, M. D. (2004). Executive function and motor performance.in M. D. Lezak, D. B. Howieson & D. W. Loring (Eds.), Neurological Assessment. New York: Oxford University Press. Lezak, M. D., Howieson, D. B., & Loring, D.W. (2004). Neuropsychological assessment.4th Edition. New York: Oxford University Press. Lie, C. H., Specht, K., Marshall, J. C., & Fink, G. R. (2006). Using fmri to decompose the neural processes underlying the Wisconsin Card Sorting Test. Neuroimage, 30, Malloy, P., & Grace, J. (2005). A review of rating scales for measuring behaviour change due to frontal systems damage. Cognitive and Behavioural Neurology, 18, Manchester, D., Priestley, N., & Howard, J. (2004). The assessment of executive functions: Coming out of the office. Brain Injury, 18(11), Manly, T., Hawkins, K., Evans, J. J., Woldt, K., & Robertson, I. H. (2002). Rehabilitation of executive function: Facilitation of effective goal management on complex tasks using periodic auditory alerts. Neuropsychologia, 40(3), Marcotte, T. D., & Grant, I. (2010). Future directions in the assessment of everyday functioning. In T. D. Marcotte & I. Grant (Eds.), Neuropsychology of everyday functioning (pp ). New York, NY: Guilford Press. Marcotte, T. D., Scott, J. C., Kamat, R., & Heaton, R. K. (2010). Neuropsychology and the prediction of everyday functioning. In T. D. Marcotte & I. Grant (Eds.), Neuropsychology of everyday functioning (pp. 5-38). New York, NY: Guilford Press.

89 82 Mesulam, M. (2002).The human frontal lobes: Transcending the default mode through contingent encoding. In D. T. Stuss & R. T. Knight (Eds.), Principles of frontal lobe function (pp. 8-30). New York, NY: Oxford University Press. Minassian, A., Perry, W., Carlson, M., Pelham, M., & DeFilippis, N. (2003). The Category Test perseveration, loss of set, and memory scales. Three New Scales and Their Relationship to Executive Functioning Measures. Assessment, 10(3), Miyake, A., Emerson, M. J., & Friedman, N. P. (2000). Assessment of executive functions in clinical settings: Problems and recommendation. Seminars in Speech and Language, 21, Miyake, A., Friedman, N., Emerson, M., Witzki, A., & Howerter, A. (2000). The unity and diversity of executive functions and their contributions to complex frontal lobe tasks: A latent variable analysis. Cognitive Psychology, 41, Mohlman, J. (2005). Does executive dysfunction affect treatment outcome in late-life mood and anxiety disorders. Journal of Geriatric Psychiatry and Neurology, 18(2), Monchi, O., Petrides, M., Petre, V., Worsley, K., & Dagher, A. (2001). Wisconsin Card Sorting revisited: Distinct neural circuits participating in different stages of the task identified by event-related functional magnetic resonance imaging. The Journal of Neuroscience, 21, Mukhopadhyay, P., Dutt, A., Kumar Das, S., Basu, A., Hazra, A., Dhibar, T., Roy, T. (2008). Identification of neuroanatomical substrates of set-shifting ability: Evidence from patients with focal brain lesions. Progress in Brain Research, 168, Nagahama, Y., Fukuyama, H., Yamauchi, H., Katsumi, Y., Magata, Y., Shibasaki, H., Kimura, J. (1997). Age-related changes in cerebral blood flow activation during a card sorting test. Experimental Brain Research, 114, Neistadt, M. E. (1992). The Rabideau Kitchen Evaluation Revised: An assessment of meal preparation skill. Occupational Therapy Journal of Research, 12,

90 83 Norman, D. A., & Shallice, T. (1986). Attention to action: Willed and automatic control of behaviour. In R.J. Davidson, G.E. Shwartz & D. Shapiro (Eds.), Consciousness and self regulation: Advances in research and theory (pp. 1-18). New York, NY: Plenum. Nyhus, E. & Barceló, F. (2009). The Wisconsin Card Sorting Test and the cognitive assessment of prefrontal executive functions: A critical update. Brain and Cognition, 71, Pineda, D. A., & Merchan, V. (2003). Executive function in young Colombian adults. International Journal of Neuroscience, 113, Proctor, A., Wilson, B., Sanchez, C., & Wesley, E. (2000). Executive function and verbal working memory in adolescents with closed head injury (CHI). Brain Injury, 14(7), Rabin, L. A., Barr, W. B., & Burton, L. A. (2005). Assessment practices of clinical neuropsychologists in the United States and Canada: A survey of INS, NAN, and APA division 40 members. Archives of Clinical Neuropsychology, 20, Rabin, L. A., Roth, R. M., Isquith, P. K., Wishart, H. A., Nutter-Upham, K. E., Pare, N., Saykin, A. J. (2006). Self- and informant reports of executive function on the BRIEF-A in MCI and older adults with cognitive complaints. Archives of Clinical Neuropsychology, 21, Radloff, L. (1977). The CES-D scale: A self-report depression scale for research in the general population. Applied Psychological Measurement, 1, Rand, D., Rukan, S. B., Weiss, P. L. T., & Katz, N. (2008). Validation of the Virtual MET as an assessment tool for executive functions. Neuropsychological Rehabilitation, ifirst, Reitan, R., & Wolfson, D. (1985). The Halstead-Reitan Neuropsychological Test Battery. Tucson, AZ: Neuropsychology Press. Robertson, I. H., Ward, T., Ridgeway, V., & Nimmo-Smith, I. (1996). The structure of normal human attention: The Test of Everyday Attention. Journal of the International Neuropsychological Society, 2(6),

91 84 Rogers, R. D., Andrews, T. C., Grasby, P. M., Brooks, D. J., & Robbins, T. W. (2000). Contrasting cortical and subcortical activations produced by attentional-set shifting and reversal learning in humans. Journal of Cognitive Neuroscience, 12, Roth, R.M., Isquith, P.K., & Gioia, G.A. (2005).BRIEF-A: Behaviour Rating Inventory of Executive Function - Adult Version. Lutz, FL: Psychological Assessment Resources. Royall, D. R., Lauterbach, E. C., Kaufer, D., Malloy, P., Coburn, K. L., & Black, K. J. (2007). The cognitive correlates of functional status: A review from the Committee on Research of the American Neuropsychiatric Association. Journal of Neuropsychiatry and Clinical Neurosciences, 19, Salthouse, T., Atkinson, T., & Berish, D. (2003). Executive functioning as a potential mediator of age-related cognitive decline in normal adults. Journal of Experimental Psychology: General, 132, Shallice, T. (2002).Fractionation of the supervisory system. In D. T. Stuss & R. T. Knight (Eds.), Principles of frontal lobe function (pp ). New York, NY: Oxford University Press. Shallice, T., & Burgess, P. (1991). Deficits in strategy application following frontal lobe damage in man. Brain, 114, Shallice, T. & Evans, M.E. (1978). The involvement of the frontal lobes in cognitive estimation. Cortex, 14, Shilling, V.M, Chetwynd, A., & Rabbitt, P.M.A. (2002). Individual inconsistency across measures of inhibition: An investigation of the construct validity of inhibition in older adults. Neuropsychologia, 40, Sirigu, A., Zalla, T., Pillon, B., Grafman, J., Agid, Y., & Dubois, B. (1996). Encoding of sequence and boundaries of scripts following prefrontal lesions. Cortex 32, Spielberger, C. D., Gorsuch, R.L., and Lushene.R.E. (1970). Manual for the State-Trait Anxiety Inventory. Palo Alto, CA: Consulting Psychologists Press.

92 85 Spooner, D. M., & Pachana, N. A. (2006). Ecological validity in neuropsychological assessment: A case for greater consideration in research with neurologically intact populations. Archives of Clinical Neuropsychology, 21, Strauss, E., Sherman, E.M.S., & Spreen, O. (2006). A compendium of Neuropsychological Tests: Administration, Norms and Commentary. Third Edition. New York: Oxford University Press. Stuss, D. T. (2006). Frontal lobes and attention: Processes and networks, fractionation and integration. Journal of the International Neuropsychological Society, 12, Stuss, D. T. (2007). New approaches to prefrontal lobe testing. In B Miller & J Cummings (Eds.), The Human Frontal Lobes: Functions and Disorders, 2nd Edition (pp ). New York: Guilford Press. Stuss, D. T., & Alexander, M. P. (2000). Executive functions and the frontal lobes: a conceptual view. Psychological Research, 63, Stuss, D. T., & Alexander, M. P. (2007). Is there a dysexecutive syndrome? Philosophical Transactions of the Royal Society B, 362, Stuss, D. T., Alexander, M. P., Floden, D., Binns, M. A., Levine, B., McIntosh, A. R., Hevenor, S. J. (2002). Fractionation and localization of distinct frontal lobe processes: Evidence from focal lesions in humans. In D. T. Stuss & R. T. Knight (Eds.), Principles of frontal lobe function (pp ). New York, NY: Oxford University Press. Stuss, D. T., Levine, B., Alexander, M. P., Hong, J., Palumbo, C., Hamer, L., Izukawa, D. (2000). Wisconsin Card Sorting Test performance in patients with focal frontal and posterior brain damage: Effects of lesion location and test structure on separable cognitive processes. Neuropsychologia, 38, Stuss, DT, Floden, D, Alexander, MP, Levine, B, & Katz, D (2001). Stroop performance in focal lesion patients: Dissociation of processes and frontal lobe lesion location. Neuropsychologia, 39,

93 86 Stuss, D. T., Shallice, T., Alexander, M. P., & Picton, T. W. (1995). A multidisciplinary approach to anterior attentional functions. Annals of the New York Academy of Sciences, 769, Turner, G. R. & Levine, B. (2004). Disorders of Executive Functioning and Self Awareness. In J. Ponsford (Ed.), Cognitive and Behavioural Rehabilitation: From Neurobiology to Clinical Practice ( ). New York: Guilford Press. Vrankrijker, K. De. M. W. De. (2003). The long way from the International Classification of Impairements, Disabilities and Handicaps (ICIDH) to the International Classification of Functioning, Disability and Health (ICF). Disability and Rehabilitation, 25 (11/12), Vuilleumier, P., & Driver, J. (2007). Modulation of visual processing by attention and emotion: windows on causal interactions between human brain regions. Philosophical Transactions of the Royal Society B: Biological Sciences, 362, Wechsler, D. (1985). The Wechsler adult intelligence scale revised. New York, NY: Psychological Corporation. Wilson, B. A., Alderman, N., Burgess, P. W., Emslie, H., & Evans, J. J. (1996). Behavioural Assessment of the Dysexecutive Syndrome. Bury St. Edmunds, UK: Thames Valley Test Company. Wilson, B. A., Cockburn, J., & Baddeley, A. D. (1985). The Rivermead Behavioural Memory Test. Reading, UK: Thames Valley Test. Wilson, B. A., Evans, J. J., Emslie, H., Alderman, N., & Burgess, P. W. (1998). The development of an ecologically valid test for assessing patients with dysexecutive syndrome. Neuropsychological Rehabilitation, 8(3), Wolf, T. J., Stift, S., Connor, L. T., Baum, C., and The Cognitive Rehabilitation Research Group. (2010). Feasibility of using the EFPT to detect executive function deficits at the acute stage of stroke. Work, 36,

94 87 Wood, J.N., & Grafman, J. (2003). Human prefrontal cortex: Processing and representational perspectives. Nature Reviews Neuroscience, 4(2), World Health Organization. (2001). International Classification of Functioning, Disability and Health. ICF, Geneva. Yantz, C. L., Johnson-Greene, D., Higginson, C., & Emmerson, L. (2010). Functional cooking skills and neuropsychological functioning in patients with stroke: An ecological validity study. Neuropsychological Rehabilitation, 20(5),

95 88 Appendix A: Behaviour Tracker Modes (a) Configuration Mode (b) Record Mode

96 89 (c) Editor Mode (d) Viewer Mode

97 90 Appendix B: BMET Participant Package Instructions In this exercise you should complete the following three tasks: 1. You should do the following 6 things: Collect something for the examiner* from the Main Information Desk (at the Khedive Entrance) and do what is necessary Buy 4 local stamps (considered 1 item) Buy a birthday card Buy a can of Coca-Cola Telephone (Name) at (Phone Number) and say where you are, who you are, and what time it is Mail something to Dr. Dawson** at the University of Toronto. 2. You must meet me at the parrot cage 10 minutes after you have started the exercise and tell me the time 3. You should obtain the following information and write it down in the spaces below: What is the closing time of the resident s library on a Thursday? What is the opening time of the gift shop on a Friday? What is the price of a Mars Bar? How many entrances/exits are there on the main floor of Baycrest? Tell me when you have completed the exercise. While carrying out this exercise you must obey the following rules: Rules You should carry out all these tasks but may do so in any order You should spend no more than $7.50 You should stay within the limits of the main floor of the hospital You should not enter any of the hospital treatment areas or staff only areas You should not go back into an area you have already been in You should buy no more than 2 items in the gift shop Take as little time to complete this exercise without rushing excessively Do not speak to us unless this is part of the exercise *Your examiner is: **Dr. Dawson University of Toronto University Ave., Suite 160 Toronto, Ont., M5G 1V7

98 Baycrest Centre for Geriatric Care Toronto, Ontario 91

99 92 Appendix C: Meta Appendix D: Metb

100 93 Appendix E: Metc Appendix F: Metd