Designing a Game-Based Cognitive Assessment for a Tablet. Tiffany Tong

Transcription

1 Designing a Game-Based Cognitive Assessment for a Tablet by Tiffany Tong A thesis submitted in conformity with the requirements for the degree of Master of Applied Science Graduate Department of Mechanical and Industrial Engineering University of Toronto c Copyright 2014 by Tiffany Tong

2 Abstract Designing a Game-Based Cognitive Assessment for a Tablet Tiffany Tong Master of Applied Science Graduate Department of Mechanical and Industrial Engineering University of Toronto 2014 The growth of ageing populations in many countries has prompted interest in the assessment of cognitive ability in elderly individuals. The early detection of cognitive impairment, and decline of cognitive abilities, is relevant to the diagnosis of mental health conditions such as dementia, mild cognitive impairment, and delirium. The detection and monitoring of cognitive status is contingent on the tools available and current methods are non-comprehensive and difficult to use. This thesis presents the design of a prototype to address the lack of readily available cognitive assessments for the elderly. The developed prototype is a tablet-based game designed for use in a clinical setting. A usability study was conducted to assess the prototype and based on these findings; a set of recommendations were developed. The limitations and implications for future work on this topic are outlined. ii

3 Acknowledgements I would like to acknowledge and express great gratitude to Dr. Mark Chignell, for which this thesis would not have been possible. The Sunnybrook Health Sciences Centre team including Dr. Jacques Lee and Dr. Mary Tierney for their continued patience, expertise and guidance in the project. The Bridgepoint Active Healthcare team including Dr. Tammy Sieminowski for her support, enthusiasm, and guidance. Last but not least, I would like to thank my family, friends, and members of the Interactive Media Lab for their support, and encouragement. iii

4 Contents 1 Introduction Context of the Research Introduction Scope Thesis Outline Literature Review Introduction to Cognitive Assessments and Central Executive Functions Standards For Assessing Cognitive Status Cognitive Impairment Conditions Standard Cognitive Assessments Methods Assessing Cognitive Status Through Central Executive Functions Computerization of Cognitive Assessments Computerizing Cognitive Assessments Use of Serious Games and Gamification in Cognitive Assessment Existing Computerized Assessments Justification For Novel Software Feasibility in a Healthcare Setting Methodology for Assessing Game Performance Speed-Accuracy Trade-off Fitts Law Elderly Technology Use Usability Engineering for Clinical Technologies User Research Methodologies Evaluation Methodologies Summary Requirements Analysis Meetings Meetings at Sunnybrook Health Sciences Centre Meetings at Bridgepoint Active Healthcare Requirements Analysis Managing Patients Recording Test Results iv

5 3.2.3 Summary of Requirements Analysis Use Cases Scope of the Design Solution Design and Prototyping of a Game-Based Cognitive Assessment Game Concept Design Game Play Features Hardware Considerations Software Considerations Prototype Development Low Fidelity Paper-and-Pencil Prototype Medium Fidelity Prototype High Fidelity Prototype Summary Collaboration with King Mongkut University of Technology Thonburi Design Elements Button Size Typography Character Design and Selection Compatibility Summary Methodology Goals Hypotheses Participants Experimental Design Background Information Questionnaires Cognitive Capacity Study Whack-A-Mole Study Post-Whack-A-Mole Questionnaires Data Analysis Results Background Information Questionnaires Cognitive Capacity Study Cognitive Capacity Test Results Observations Whack-A-Mole Game Performance Data Analysis Standardized Performance Metric Understanding Game Performance Data Relationship Between Game Performance and Cognitive Ability Observations Exit Questionnaire v

6 6.5 Revisiting Hypotheses Updated Game-Based Cognitive Assessment Prototype 57 8 Discussion Cognitive Capacity Study Whack-A-Mole Game Performance Study Standardized Performance Metric Speed-Accuracy Trade-Off Fitts Law Relationship Between Game Performance and Cognitive Ability Game Performance and Delirium Using Other Games and Serious Games as Cognitive Assessments Updated Prototype Exit Questionnaire Usability Issues Tablet-Sensitivity Limitations Sample Size Evaluating Whack-A-Mole Enjoyability Evaluating Audio Feedback Usability Study Setting Non-Validated Cognitive Ability Tests Recommendations Haptic Feedback Designing Game-Based Cognitive Assessments Conclusions and Future Directions Contributions Future Work Hospital-based study Control Population Study With Non-Experts Modelling Fitts Law Investigating The Stroop Task Introducing A Non-EF Test Final Words Bibliography 71 Appendices 81 A Demographic and Computer Skill Questionnaire 81 B Whack-A-Mole Questionnaire 84 C Relationship Between Game Performance and Cognitive Abilities 86 vi

7 List of Tables 2.1 Summary comparing the gold-standard assessments to screen for cognitive impairments Summary of usability tests in the design cycle Requirements analysis summary Experimental design of the tablet-based study Cognitive abilities correlations Standardized performance metric correlated with cognitive ability scores Summary of exploratory factor analysis results for the exit questionnaire using rotated factor loadings (N = 24) C.1 Correlations between median RT of cognitive abilities and -Z(accuracy)-Z(time) value of each game parameter with a target size of 150px C.2 Correlations between median RT of cognitive abilities and -Z(accuracy)-Z(time) value of each game parameter with a target size of 175px C.3 Correlations between median RT of cognitive abilities and -Z(accuracy)-Z(time) value of each game parameter with a target size of 200px vii

8 List of Figures 2.1 The design cycle process Affinity diagram Diagram visualizing the use cases for a patient Diagram visualizing the use cases for a researcher Google Nexus 7 tablet [1] Paper-and-pencil prototype sketch of the over-arching screening tool software. The wireframe begins on the home/welcome screen (top left screen) Overview of the whack-a-mole game based on the low-fidelity, paper-and-pencil prototype Overview of the medium-fidelity prototype. New wireframes are highlighted with a blue border Overview of the whack-a-mole game in the working prototype. New wireframes are highlighted with a green border, and wireframes added in the medium-fidelity prototype are highlighted with a blue border Wireframes depicting the sign on and registration screens Wireframe depicting the sign on and registration screens Wireframes illustrating the game menu with different setting, whack-a-mole game board, and end screen Comparison of the game settings screen Comparison of the whack-a-mole game board The design cycle process followed for the collaboration with King Mongkut University, Thonburi Visual depiction of 48 dp compared to a button [2] Example of characters used in the game Screen capture of the Stroop Test software developed by PEBL Screen capture of the Stroop Test software developed for Vocalage/Toyota [3] Screen capture of Wisconsin Card Sort Test Frequency of device usage Most frequently used touch-based devices viii

9 6.3 Venn diagram illustrating the relationship between the three EFs. PEBL version of inhibition is represented in the diagram. Squared partial correlations were used to determine the percent overlaps between the EFs. Squared partial correlations were used to determine the percent overlaps between the EFs Graphical representation between a tradeoff and correlation relationship Game performance data analyzed using standardized scores of accuracy and time Scatterplot and regression lines of inhibition against participants z-score for each of the game conditions Scatterplot and regression lines of shifting against participants z-score for each of the game conditions Correlation between game performance and inhibition ability in games with different combinations of distractor style X feedback style. The error bars represent one standard error Venn diagram illustrating the overlap between EFs and overall performance scores. Note that, since the overlap between overall performance scores and shifting ability accounted for less than one percent of the variance in overall performance scores, that overlap is not shown in the diagram. Squared partial correlations were used to determine the percent overlaps between game performance and EFs Scatterplot of questions 1/2/6 against question 3. Questions 1/2/6 are related to the tablet setup and ease-of-use. Question 3 is related to the feeling of some touches not being registered by the tablet. Cluster 1 represents users that are comfortable with the tablet setup and felt that their touches were being registered by the tablet. Cluster 2 represents users uncomfortable with the tablet setup and felt that some of their touches were not being registered Wireframes depicting the sign on and registration screens Wireframes of the game menu and settings screens Wireframes showing the instructions modal screen and whack-a-mole game board Wireframes of the tutorial screens on how to play the whack-a-mole game Comparison of the game settings screen Example of game board on two different devices ix

10 List of Abbreviations AD ADL ANOVA CAM CANTAB CogDR CRT DSM DSST ED EF FIM IADL ICD ICU ID IML KMO MCI MMSE MoCA MT OT PCA PD POCD RM RT TMT WCST Alzheimer s Disease Activities of Daily Living Analysis of Variance Cognitive Assessment Method Cambridge Neuropsychological Test Automated Battery Cognitive Drug Research Correct Reaction Time Diagnostic and Statistical Manual of the American Psychiatric Association Digit Symbol Substitution Test Emergency Department Executive Function Functional Independence Measure Instrumental Activities of Daily Living International Classification of Diseases Intensive Care Unit Index of Difficulty Interactive Media Lab KaiserMeyerOlkin Mild Cognitive Impairment Mini-Mental State Examination Montreal Cognitive Assessment Movement Time Occupational Therapist Principal Component Analysis Postoperative delirium Postoperative Cognitive Dysfunction Repeated Measures Response Time Trail-Making Test Wisconsin Card Sorting Task x

11 List of Units dp mm px s sp density-independent pixels millimetre pixels seconds scale-independent pixels xi

12 Chapter 1 Introduction 1.1 Context of the Research In Canada the fastest growing population group consists of individuals over the age of 65 years [4]. This rapid growth places more people at higher risk of developing age-related cognitive impairments such as dementia, mild cognitive impairment and delirium.these ageing-related changes often make it difficult for elderly persons to participate in their usual daily tasks, and leisure activities. Cognitive decline in the short-term (delirium), and longer-term (dementia), is a key symptom of diseases that may benefit from early detection. Methods are urgently needed for detecting cognitive decline in the ageing population, since early detection of cognitive disorders is key to preventing serious outcomes such as lengthy hospitalization or death [5]. Available screening tools for cognitive disorders are often time consuming, costly to use, and they lack sensitivity to specific symptoms, which can lead to misdiagnosis or under-diagnosis. This may be due to inexperience with recognizing signs of cognitive decline, lack of screening, and the failure to use standard screening methods [6]. In Canada, approximately two-thirds of delirium cases are under-diagnosed [7]. The foregoing discussion highlights the need for a robust screening technique to identify increased risk of cognitive decline so that appropriate prevention methods can be used. Ideally, cognitive assessment should be enjoyable and self-administered so that large scale cognitive assessment does not place an undue burden on healthcare professionals. Computerization and gamification are promising strategies for making cognitive assessment easy to self-administer and enjoyable to use. The research reported in this thesis aimed to develop and assess a cognitive screening tool that addresses some key usability and human factors related issues present in existing cognitive assessments. 1.2 Introduction Cognitive screening through specially designed games has the potential to modernize measurement of cognitive status. Assessing patients through digital means can measure cognitive status efficiently and remotely, and enable serial testing over time. An additional benefit is that a software-based solution logic can adapt game/assessment properties to better match the characteristics of the user (e.g. making targets larger for people with low manual dexterity). 1

13 Chapter 1. Introduction 2 Cognitive impairments can result in the interruption of daily activities, increase the length of hospital stays, and can lead to irreversible damage [8]. Ideally, cognitive disorders should be detected as early as possible, so that appropriate treatments can be given, and assistance provided. These considerations motivated this research into the utility of cognitive assessment tools for the elderly that are cognitively stimulating, and user-friendly. The results of this research are intended to assist healthcare professionals in developing appropriate interventions and therapies to maintain and improve a patient s quality of life. The research conducted for this thesis explores the usability of a game-based cognitive assessment tool. Through a usability study conducted at the University of Toronto, key recommendations and preliminary findings were obtained to inform future research in this area. Cognitive assessments are used to identify cognitive impairments and to assess a patient s health status and improvement through the course of therapy. Determining patients cognitive strengths and weaknesses makes it possible to assess their ability to make medical decisions, live independently, and manage their finances [9]. Traditional assessments that are administered orally or using pencil-andpaper are often expensive and time consuming to carry out multiple times during a patient s course of therapy. Other limitations include available resources, language barriers between the patient and test administrator, and sensory impairments that decrease test performance (e.g. visual impairments or immobility). An alternative to conventional testing methods is serial testing with computerized game-based cognitive assessments that can be used to assess trends in cognitive status over the course of therapy. An automated tool that is able to detect predictors of cognitive deficits would have useful clinical applications. The tool could alert clinical staff to modify their therapy strategies when patients show deterioration in their test performance. The goal of this research is to design a novel screening tool that assesses a patient s cognitive status. In the particular application that motivates this research, results from game-based cognitive assessment will enable researchers to estimate a patient s risk for cognitive decline. This thesis will aim to address the following research questions: 1. Is it feasible to integrate technology into a healthcare setting such as hospital emergency department to allow healthcare workers to screen patients cognitive status? 2. If feasible, how can cognitive assessment prototypes be designed for use in a clinical setting? This thesis details the usability study conducted, and the results found. A discussion of the results presented, and suggestions for how the findings can inform future research and design of game-based cognitive assessments are provided. 1.3 Scope This thesis will focus on the design of a game-based cognitive assessment that is evaluated on a healthy, non-elderly sample. The motivation for this work and overarching goal is to develop a game-based method to assess delirium in elderly patients in a hospital emergency department. The scope of this research involves initial steps toward this goal, using central executive functioning as a criterion of relevant cognitive status.

14 Chapter 1. Introduction Thesis Outline In Chapter 2, I will introduce the subject of cognitive assessments, and current methods to assess cognitive status. Then, the topic of computerization of conventional cognitive assessments and other technological methods to assess cognition will be presented through a review of literature. Next, in Chapter 3, I will introduce the user-centred design methodologies used in this research to identify the design problem, and to gather user requirements for the design of a game-based cognitive assessment tool for use in a healthcare environment. Chapter 4 describes the design process including sketching, wireframing, and prototyping of the screening tool based on the requirements discussed in Chapter 3. Chapter 5 details the methodology used in an experiment conducted to evaluate the design solution proposed in Chapter 3. The participants in the study were recruited from the University of Toronto. In Chapter 6, the results of the experimental study are presented. Chapter 7 presents the design of a revised iteration of the screening tool based on the results of the usability study. Chapter 8 discuss the results and implications of the experimental results obtained are discussed, along with the limitations of the study. Some proposed recommendations based on the experimental results will also be presented. Finally, Chapter 9 summarizes the conclusions, and contributions of this research, and recommendations are made for future work.

15 Chapter 2 Literature Review This chapter will begin with an overview of cognitive assessments and how they be used to assess central executive functions and cognitive impairments. The importance of cognitive assessments in the diagnosis of age-related cognitive disorders such as delirium, mild cognitive impairment and dementia will be introduced. This will be followed by an overview of the standard screening methods used to assess these cognitive disorders. Next, more general neuropsychological assessments tools that evaluate cognitive decline will be discussed. A discussion of the computerization of the cognitive assessments will be presented. The use of serious games and gamification as methods for cognitive assessments will be discussed. Following this, the use of methodologies to assess game performance will be presented. The chapter will end with a discussion on usability engineering in healthcare technology design. 2.1 Introduction to Cognitive Assessments and Central Executive Functions Changes in cognitive function are part of the normal ageing process [10]. The assessment of cognitive function is particularly important in elderly adults, as there is a higher prevalence of age-related cognitive impairments such as dementia, mild cognitive impairment, and delirium[11]. For example, the prevalence of dementia in adults over 65 years of age is 3 11 % compared to less than 1 % in adults under 65 years of age [12]. Accurate screening of these cognitive impairments can assist in differentiating age-related versus abnormal cognitive decline [12]. Cognitive assessments are essential tools used by clinicians and researchers to: assess the cognitive status of a patient, make differential diagnosis, and monitor the course of diseases [12]. They can be used to test a variety of cognitive skills including attention, memory, perception, and central executive functions. Determining a patient s cognitive strengths and weaknesses allows clinicians to assess and plan ahead for future support of patients and to estimate their ability to make medical decisions, and live independently [13]. Conventional neuropsychological tests such as the Mini-Mental State Examination [14] are administered verbally by trained staff or completed by a patient using paper-and-pencil such as the Montreal Cognitive Assessment [15]. These methods of test administration require face-to-face interaction between a trained test administrator and patient, and are time-consuming (e.g. can range from 5 to 15 minutes) and costly. The frequency of cognitive assessment differs based on the patient population and 4

16 Chapter 2. Literature Review 5 healthcare setting. For instance, in one hospital emergency department, patients are typically assessed once at admission [Dr. Jacques Lee, Personal Communication, February 2013]. A major concern with such testing method is under-diagnosis resulting from under-trained staff and inaccessibility of testing centres for patients. Additional challenges associated with conducting cognitive assessments include handling people who have sensory impairments such as visual and auditory deficits. Patients may perform poorly on tests that have visual and/or auditory stimuli, thus making the interpretation of results difficult. Also, patients with mobility limitations may demonstrate reduced speed and accuracy in timed tests. Language barriers can be remedied by using a translated version of a test, however, many translated tests do not have adequate normative data [13]. One method to assess cognitive ability through screening tools is by evaluating central executive functions (EFs). Central EFs are regulated within the pre-frontal cortex of the human brain, and they incorporate complex cognitive processes such as working memory, problem solving, and reasoning [16], [17]. One model of central EFs proposed by [18], suggests that the three EFs of inhibition (ability to prevent an action or behaviour), shifting (ability to switch between tasks), and updating (ability to update one s working memory) can be used to understand complex cognitive tasks and task performance. There are many tasks that have been designed to specifically assess each of these three functions, and some of these will be described in Subsection In [19], EFs are associated with system 2 the effortful processing associated with complex tasks and with controlling attention. In [20], it is stated that executive functions make possible mentally playing with ideas; taking the time to think before acting; meeting novel, unanticipated challenges; resisting temptations; and staying focused. The core EFs that Diamond [20] listed were inhibition, interference control, working memory, which likely overlaps with [18] s conception of updating and [21] s notion of monitoring and cognitive flexibility. There is general agreement that EFs are important indicators of cognitive status, and declines in these abilities have been used to detect adverse changes in cognitive status such as post-operative delirium [18], [22]. Mechanisms of inhibition and selective attention are key constructs of executive functioning which are included in almost all discussions of EFs. Impaired executive functioning is associated with ageing adults and functional decline, thereby impacting quality of life and ability to complete activities of daily living such as maintaining mobility, and bathing [23]. 2.2 Standards For Assessing Cognitive Status In a healthcare setting such as a hospital emergency department (ED) and neuro-rehabilitation centre, there are widely used cognitive assessments for detecting cognitive decline. These assessment tools are validated methods and they focus on the assessment of cognitive ability in the elderly. They include the Mini-Mental State Examination, Montreal Cognitive Assessment, and the Confusion Assessment Method. These assessments are condition-specific, meaning that they were designed to assess specific cognitive impairments. Thus, before reviewing these assessments, a brief overview of the cognitive impairments they were designed to assess (mild cognitive impairment, dementia or delirium) will be presented.

17 Chapter 2. Literature Review Cognitive Impairment Conditions Mild Cognitive Impairment Mild cognitive impairment (MCI) is a phase that occurs between normal ageing and dementia [15]. The prevalence of this condition in adults 65 years and older ranges from 3% to 19% [24]. Patients with MCI often progress to Alzhiemer s Disease with a progression rate varying between 1-25% per year [9], [25], [26]. This estimated range is based on multiple studies, and potentially varies due to the diagnostic criteria of MCI, screening tools, and sample populations [25], [26]. Symptoms of MCI include forgetfulness, depression, and aggression. Other signs of MCI include impaired decision making ability, lack of medication adherence, and minor disruptions in daily activities (e.g. eating, walking, personal hygiene) [27]. Less commons signs of MCI include language difficulties and inattention. Dementia Dementia is a group of diseases characterized by degeneration in mental ability. It is defined as a loss in memory as well as decline in another cognitive domain such as EF that results in a disruption of daily activities and cannot be explained by any other condition [28]. Symptoms of dementia include difficulties in memory, language, perception, and cognition. It is also associated with disruptions of daily activities [29]. The early diagnosis of dementia through cognitive assessments is highly beneficial for elderly adults and their family. Screening for signs of cognitive decline and assessing functional ability can assist in preventing further and irreversible impairments such as memory loss [30]. Delirium Delirium is a neuropsychiatric syndrome that sometimes occurs in the elderly and can develop during hospitalization. It is characterized by acute confusion, varying levels of consciousness, and decreased attention. Other features include psychomotor agitation, sleep-wake cycle alterations, and perceptual disturbance [13]. Delirium has occurrence rates of 11-42% per admission, and in Canada, approximately two-thirds of delirium cases are under-diagnosed [5], [31]. This under-diagnosis rate may be due to inexperience with recognizing delirium, lack of screening, and the failure to use standard screening methods [6]. In addition, in hospital intensive care units (ICU), incidence rates have been reported of up to 80% for admitted patients [32]. Delirium is also associated with increased mortality and morbidity [31]. In a hospital setting, it is associated with a mortality rate of up to 25-33% [33]. Delirium can result in the interruption of daily activities, and increase the length of hospital stays [8]. In the United States, it has been estimated that delirium costs the healthcare system between $ billion [34]. This estimate was based on the number of patients that developed delirium during hospitalization between in a single medical centre with the results being extrapolated to estimate the annual costs nationwide. The expenses due to delirium also extend beyond a hospital as additional costs are also incurred for patients needing a nursing home, rehabilitation, medical services, and re-hospitalization [33] Standard Cognitive Assessments Methods The following section will now focus on describing standard cognitive assessment methods.

18 Chapter 2. Literature Review 7 Montreal Cognitive Assessment (MoCA) The Montreal Cognitive Assessment(MoCA) is a cognitive test used to screen for mild cognitive impairment (MCI) based on eight cognitive items (orientation, attention and concentration, executive function, memory, visual-spatial skills, language, conceptual ability, and calculation)[28]. MCI is a phase that occurs between normal ageing and dementia[15]. Although the MoCA was designed to screen for MCI, it is commonly used to screen for delirium as well. The MoCA has a maximum score of 30 points, and a score of 26 or above indicates that a patient is not cognitively impaired. It consists of a one-page test available in multiple languages that takes ten minutes to administer [15]. Mini-Mental State Examination (MMSE) The Mini-Mental State Examination (MMSE), also known as the Folstein test, is a questionnaire that examines cognitive functions based on 30-point score [35]. A score of 24 or below on the MMSE, indicates cognitive impairments [14]. It evaluates a patient s memory, orientation to time and location, ability to name objects and follow commands, and motor skills in writing and drawing [14]. The MMSE is not as effective when examining patients with a low education level and or poor English literacy [28]. In addition, the MMSE is copyright protected and a license must be purchased to administer this assessment. Compared to the MoCA, the MMSE is more sensitive to patients with more severe cognitive impairment such as Alzheimers Disease (AD) [15]. Since it has low specificity, and ceiling effects, when used with non-demented patients, it is generally not suitable for assessing mild cognitive impairment (MCI) [27], [36]. A study by [14], found that parts of the MMSE were difficult to complete in an elderly population (70 90 years of age) with delirium [14]. The writing, and drawing tasks that require the use of a pencil from the exam were difficult to complete for reasons such as lack of strength, attached to an intravenous line, or being in the supine position (lying down on the back) [14]. Confusion Assessment Method (CAM) The Confusion Assessment Method (CAM) [36] is perhaps the most popular method of delirium diagnosis at present. This test was developed in to help non-psychiatrically-trained clinicians identify, and recognize delirium in hospital, and research settings [5]. The CAM takes approximately five to ten minutes to complete, and has high sensitivity and specificity. It consists of nine questions that evaluate the patient based on the following criteria: (1) acute onset and fluctuating course, (2) inattention, (3) disorganized thinking, and (4) altered level of consciousness [37]. The CAM requires that criteria 1 and 2, and either feature 3 or 4 are present in a patient, in order to diagnose them as delirious. The CAM does not provide information on the severity of delirium; instead it reports a binary score indicating the presence or absence of delirium [36]. Despite its widespread use, the CAM has low sensitivity when administered by under-trained staff, primarily due to under-recognition of symptoms such as inattention [38]. There exist many challenges in diagnosing a patient with delirium as it is an under-recognized condition and often misdiagnosed as dementia or depression [39]. Possible explanations for its underrecognition include lack of awareness, and a disregard for delirium as an important clinical syndrome. Healthcare professionals such as physicians and nurses may expect patients to exhibit visible signs such as hallucinations, agitation, and inappropriate behaviour. However, in elderly patients, delirium of-

19 Chapter 2. Literature Review 8 Table 2.1: Summary comparing the gold-standard assessments to screen for cognitive impairments. Assessment Administration Time (minutes) Rating Scale Primary Condition MoCA Ten [15] 30 points [15] MCI CAM Five to ten [5] Nine questions [5] Delirium MMSE Five to ten [35] 30 points [35] Dementia ten presents as lethargy and decreased activity [39]. Delirium can also present in a fluctuating course, clinicians may be diagnosing a patient during a non-delirious state. Other challenges associated with the clinical assessment of delirium include the changing nature of its definition and associated diagnostic criteria. Currently, the gold standard definitions and diagnostic criteria are based on the Diagnostic and Statistical Manual, fifth edition (DSM-5) and International Classification of Diseases, tenth edition (ICD-10) [39]. These two diagnostic standards are difficult for clinicians to apply on a daily basis with each patient, and thus, screening tools such as the CAM have been developed [40]. Summary The most commonly used standards to assess cognitive function in elderly adults are the MoCA, MMSE, and CAM (Table 2.1). Each of these methods are best-suited to screening for one condition, and their results vary in assessing individuals based on the severity and progression of their cognitive impairment(s). These standard screening tools all require trained staff to administered them in a face-to-face interaction. 2.3 Assessing Cognitive Status Through Central Executive Functions The previous section focused on standard methods for the diagnosis of dementia, MCI, and delirium in elderly adults. These screening tools (MoCA, MMSE, and CAM) are condition-specific methods that do not adequately assess a healthy adult. Thus, other methods that are sensitive to relatively small changes in cognition are needed. In this section, some tasks that require central EFs (and which are therefore sensitive to changes in cognitive status), will be discussed. Inhibition Ability and Go, No-Go Discrimination Task The go, no-go discrimination task requires a participant to respond to certain stimuli (go target) but not others (no-go target). Stimuli is often in the form of a flashing light or image. This task can be structured so that participants must learn which cues result in a positive or negative outcome. It looks at a participant s omission errors (i.e. failure to respond to a go stimulus) and commission errors (i.e. failure to inhibit a response to a no-go stimulus) [41]. This discrimination test examines inhibition and impulse, and is commonly used to assess inhibition ability. The EF of inhibition is a particularly important for cognitive assessment because it is sensitive to age, and cognitive status [42].

20 Chapter 2. Literature Review 9 Shifting Ability and Trailing Making Test (TMT) The Trail Making Test (TMT) is a commonly used neuropsychological test that is sensitive to brain damage, and examines working memory. This test presents a participant with 25 circles, depending on which variant of the test they are performing: Part A (numbers only) or B (numbers and characters). In TMT Part A, the participant is required to connect the dots in a sequential, increasing numerical order. In TMT Part B, the participant is to connect the dots in an alternating numerical-alphabetical order (e.g. 1-A-2-B). The participant is given a time limit of five minutes to complete the test. A test administrator typically corrects the patient if any errors are made before allowing the patient to continue with the test. Metrics collected from the TMT include the time to completion, and the number and type of errors. Both variants of the test look at visual search and scanning abilities, psychomotor speed of processing, and mental flexibility. Only TMT Part B specifically looks at executive functioning [29], [43]. Updating Ability and Digit Symbol Substitution Test (DSST) The Digit Symbol Substitution Test (DSST) is used to assess associative ability, psychomotor speed, and visual short-term memory, often after a patient is given a pharmaceutical. It also examines learning, translation, and intellectual abilities [44]. The test presents a patient with a legend containing a digit (numbers one through nine) paired with a symbol on the top of the page. The bottom portion of the page contains a set of digits in random order, and the patient s objective is to draw the corresponding symbol based on the digit-symbol legend. A patient is given 90-seconds to draw as many correct symbols matching the digits as possible [45]. In total there are 100 items to be matched. Performance on the DSST is correlated to ageing, such that lower scores are associated with cognitive decline [44]. There are three hypotheses that explain why this occurs. First, a peripheral motor speed hypothesis suggests that lower scores are due to reduced speed in drawing symbols. Another hypothesis proposes that memory is a limiting factor such that older adults have more difficulty in recalling the digit-symbol code compared to younger adults. Finally, the third hypothesis suggests that reduced scores are due to overall decline in global features such as perception, motor skills, and cognitive processes [46]. 2.4 Computerization of Cognitive Assessments The previous section focused on standard (MoCA, MMSE, CAM) and general (go/no-go discrimination task, TMT, DSST) cognitive assessments used to assess cognitive ability. The following section will discuss the computerization of cognitive assessments for assessing cognitive status. Existing computerized assessments will then be discussed, followed by a discussion of how this thesis will address the challenge of developing suitable computerized assessments Computerizing Cognitive Assessments Traditional cognitive assessments discussed above are typically administered verbally (e.g. MMSE) or using paper-and-pencil (e.g. TMT) methods. These screening tools can benefit from being converted into an electronic form on a desktop computer or other technological device, such as a handheld tablet. The benefits associated with computerizing cognitive screening tools will be discussed below.

21 Chapter 2. Literature Review 10 Computerized cognitive assessments enable serial testing to gather baseline data, and compare it to cognitive states following treatments and at different time intervals. In contrast, the costs and resource requirements of conventional tests make it prohibitive to administer them repeatedly to large numbers of people. Conventional neuropsychological tests are typically not sensitive enough to detect relatively small changes in cognitive status over time that may be diagnostically relevant. As a consequence they are neither suited for testing serially, nor for detecting the unpredictable changes in cognitive status that are associated with cognitive impairment such as delirium. One of the many advantages that computer-based tests have over conventional tests is their ability to accurately record response time (RT) in small units such as milliseconds. This high resolution in RT data provides the ability to measure variability within a participant more accurately compared to traditional tests (which often record a patient s overall time to complete the test rather than separate times for each question [47]). Computerized assessments allow for easy randomization in tests with configurable parameters such as the TMT and DSST. In the conventional paper-and-pencil versions of these tests, it is often not feasible for test administrators to randomize them for every patient. This randomization feature is important for reducing practice effects, and increases test-retest reliability. In the case when multiple versions of a conventional test are available, it is not always clear that the different versions are equivalent, which then introduces another systematic source of error [47]. For an appropriately constructed game-based assessment it may be possible to have almost limitless types of playing conditions so that people do not become habituated to the test, and there is no possibility of learning the answers. Another attractive feature of computerized assessments is the ability to store and share data as soon as it is recorded. This allows specialists to interpret data immediately and share it with other specialists for feedback [47]. This can also facilitate the remote data sharing between clinicians and researchers. Administration of conventional assessments can result in variability due to differences in administration protocols between patients. In comparison, computerized tests allow for standardization that reduces measurement error from administrators [47]. Although there are many benefits in using computerized tests, there also disadvantages to be taken into consideration. Criticisms associated with the use of computerized assessments include poorly designed interfaces, usability concerns, and lack of normative data and psychometric standards [9]. One of the major limitations of using technology to deliver computerized assessments is that hardware changes rapidly, which requires frequent software updates. Consequently, the associated costs may be high, especially in the initial adoption of these tests. There may be additional costs in training administrators on how to use the new technology. Another major limitation associated with computerized assessments is the lack of normative and test-retest data. However, this limitation can be overcome by additional testing and validation studies with further adoption of this technology [47] Use of Serious Games and Gamification in Cognitive Assessment As discussed above, there are many benefits of computerizing cognitive assessments, which include the unique ability to design an interactive and entertaining screening tool. Two potential methods to do so are serious games and gamification.

22 Chapter 2. Literature Review 11 Serious Games Serious games are games that are have been designed with a primary focus other than entertainment [48]. The leisure aspect of games can be leveraged to engage with participants while facilitating skill development or learning. Many serious games have been designed for use in health care, such as managing juvenile diabetes[49], reducing obesity[50], and managing asthma[51]. These serious games vary in their implementation from a role-playing game [49] to a puzzle game [52]. Benefits of using serious games in healthcare include the ability to design personalized games, promote health-related behavioural change, and educate participants. The concept of personalization can be applied to serious games to create adaptive gameplay for participants. Personalization enables a game to adapt to a participant s unique needs, and abilities to promote engagement. Through this method, a participant s in-game performance has a direct influence on game elements such as the story, and level-of-difficulty. In addition, the use of an adaptive framework minimizes learning curves associated with playing a new game, thus increasing participant engagement [53]. For instance, RehaCom is a cognitive rehabilitation game that adapts to a participant s performance, and adjusts the difficulty level accordingly[54]. This ensures that a participant maintains interest in this game without developing frustration thereby encouraging the participant to continue playing the serious game, and perhaps improve rehabilitation as a consequence. A more advanced method of personalization in serious games is to use a participant s current health status. For example, [55] examined the efficacy of tailoring and translating physical activity information to older adults. The program suggested exercises such as walking, and cycling that best matched the participant s location (e.g. home, local community). Results from the study indicated that participants who received the computer-based tailoring exhibited long term behavioural change. increasing physical activity levels [55]. In developing serious games, there a number of game components that can enhance and influence user experience and performance. Examples include audio and visual effects, which can be used to represent game feedback. For instance, different auditory sounds and visual cues can be used to indicate successes (e.g. cheerful tones, visual check-marks) and errors (e.g. loud beeps, visual x marks) during gameplay [56]. The use of audio in a serious game can also assist participants who have visual impairments as an alternative way to understand and play the game [57]. As the number of cases of auditory and visual impairments increases with age, including both audio and visual cues in a serious game can enhance an elderly adult s experience and interaction [58]. Although there are many potential benefits to using serious games for cognitive assessment, there also exists several drawbacks. These limitations include development costs, lack of theoretical frameworks and design guidelines, and restricted access due to cost and availability. The cost to develop a serious game with a specific focus can be expensive, and requires many resources including software programmers, and medical experts. There is also a lack of theoretical frameworks and guidelines in the design of serious games for health care, which makes it difficult for game designers and clinicians to devise a game that delivers the appropriate health intervention whether it be knowledge transfer or behavioural change [50]. In addition, the availability of serious games may be hindered by commercial costs, which can dissuade and pose as a barrier for some patients. These are major challenges that hinder the inclusion of serious games in patient-centred care.

23 Chapter 2. Literature Review 12 Gamification The second method to create an enjoyable and entertaining assessment tool is by using gamification, which is the use of game-like elements in non-game contexts to improve user experience and engagement [59]. Some of the goals of gamification in this context include enhancing user participation, and shaping behaviour while increasing knowledge. Introduction of game concepts, such as points (scoring), badges, and leader boards, can motivate users when performing mundane tasks [59]. The goal of combining gamification with a computerized cognitive assessment may encourage patients to be cognitively active. In this way, patients can play a game that is entertaining, while being mentally active to prevent cognitive decline. This patient-centred care approach also promotes patients to take an active role in their cognitive health. In addition to being a useful therapeutic procedure, a game may allow clinicians to assess the patient s cognitive status in a non-traditional form of testing. Examples of gamification in healthcare include managing Type I diabetes [60], and chronicling cancer pain[61]. In managing Type 1 diabetes, a mobile application was developed by[60] and targets adolescent users. The purpose of this application is to encourage young adults to frequently measure and record their blood glucose levels in the application by providing them with points. These points can then be used to purchase digital goods from Apple itunes and Apple App Store. This application is an example of a gamified tool that relies on rewards as a mechanism to promote behavioural change. Another application called Pain Squad developed by [61], is aimed at young cancer patients. The purpose of the application is for patients to journal their cancer-related pain. In this application, patients are given challenges that encourage them to record their pain logs twice-a-day. They are also given rewards and can earn badges by completing tasks such as successfully journalling a specific number of entries [61]. Furthermore, these two gamified applications demonstrate the use of reward systems and points to encourage participants to use their application for healthcare-related conditions. Serious Games versus Gamification Using a serious game as a method of cognitive assessment has a number of advantages over gamification, including the ability to design a custom game with a specific purpose as opposed to gamifying an existing assessment. The ability to design and develop a serious game enables researchers to consider the requirements and needs of the users and design challenge using a user-centred design methodology. For example, in designing a serious game for assessing delirium, the needs of both the participants playing the serious game, and clinicians who are caring for the participant can be considered. As mentioned above, there are existing assessments of central EF such as the Go/No-Go Discrimination Task, TMT, and DSST. These tests could be gamified and translated into a computerized format. However, these tests may not be entertaining and enjoyable to all participants. In comparing one of these computerized assessments to a serious game, a user may find the latter more enjoyable to play repeatedly. In addition, the gamification of an existing neuropsychological assessment might alter the validity of the screening tool. Additional research on the influence of gamification on existing screening tools would need to be carried out to determine their effectiveness and impact. The research reported in this thesis will focus on the design of a serious game that assess cognitive status.

24 Chapter 2. Literature Review Existing Computerized Assessments ElderGames The ElderGames is a European project focused on maintaining and improving cognitive ability and social engagement in older citizens to prevent cognitive decline due to ageing [62]. Its current prototype consists of a computer, four cameras, one television (either plasma or LCD), four pointers (to be used as game pieces), and a table (to support the television and pointers). This apparatus is meant to mimic tabletop games. This computerized cognitive tool has been evaluated in healthy elderly samples across Europe to assess its acceptance and usability. Results have shown that it is well-accepted and usable by this age group [63]. This tool is not directly aimed at, nor has it been evaluated as a diagnostic tool to screen for certain conditions such as delirium. At present, there are no studies demonstrating the diagnostic value of ElderGames. Cambridge Neuropsychological Test Automated Battery The Cambridge Neuropsychological Test Automated Battery (CANTAB) is one of the most widely used computerized cognitive assessment tool that examines EF, working memory, and RT. The CANTAB has been validated in several studies, and results have shown that a general decline in CANTAB performance is associated with ageing [64], [65]. Many of the studies involving CANTAB have been performed on healthy samples, and individuals with AD or MCI [66]. No studies to date have reported using CANTAB as a method of delirium assessment. CogTest CogTest is an example of a software suite that offers translated versions of traditional cognitive assessments such as the go/no-go discrimination task, and DSST into a computerized form [67]. This software can be administered on a touch-based device such as a tablet or using a computer with mouse and keyboard input [68]. Numerous studies have been conducted using CogTest in evaluating patients with schizophrenia, but none to date using this software to assess patients for delirium Justification For Novel Software As briefly discussed in the previous subsection, there already exist computerized cognitive assessment tools. Why design a novel tool to assess changes in cognition? Current systems suffer from the disadvantages listed below: Cost: Proprietary software can be costly to purchase and administer, and timely to learn how to use. Game-like features: Computerized assessments such as CANTAB and CogTest are translated versions of traditional assessments, and lack game-like features integrated into them. Thus, there is no intrinsic motivation for the patient to repeatedly conduct the assessment in the absence of coercion, or encouragement, from others. Clinician-centered use: Software like CogTest requires a trained administrator to set-up the software that the patient is evaluated on.

25 Chapter 2. Literature Review 14 Form factor: Many computerized assessments require a computer, keyboard, and mouse input. In environments where mobile work stations are unavailable or infeasible such as a hospital ED, this method of cognitive screening is ineffective. Patient familiarity and knowledge of technology use: Patients require a basic understanding and familiarity with a computer in order to complete a computerized assessment administered via this method. This may be a barrier for some patients. Given that using an existing software suite is not an ideal option for the purpose of screening for delirium, it would also be wise to review other options such as serious games that assess cognitive functions. As of October 2012, both the Google Play Store and Apple App Store, offered 700,000 applications available to the public [69]. This raises the question of why not use an existing application as a screening tool for delirium? Existing applications are not used as they have many disadvantages such as: Customizability: Games developed by third-party entities may not allow for full customization of game settings.. Privacy and security concerns: Personal information is collected thus maintaining privacy and security is of high importance. Some applications collect data that is stored on a private server that may not be secure. This data may also be sold and can possibly contain personal identifiers, which poses a security risk. Software upgrades and failures: With the use of any technology, there runs the risk of software failures occurring due to programming issues and bugs. Monetization features and advertisements: Many mobile applications feature advertisements to help generate revenue. These are undesirable and distracting features that are inappropriate in a healthcare setting Feasibility in a Healthcare Setting Given the limitations of the current standard cognitive assessments (e.g. MoCA, MMSE, CAM) in diagnosing cognitive impairments(e.g. MCI, dementia, delirium), and in order to prevent under-diagnosis, an alternate assessment method is warranted. Another concern is that cognitive disorders such as delirium, is associated with fluctuations in cognitive status. Thus a patient may test well at one point, and yet have a relatively quick decline in status a few hours later. Tests for detecting risks of cognitive decline need to be capable of repeated administration in situations where the prevalence of delirium tends to be high, such as, in a hospital ED. One solution is to administer a computerized assessment on a portable device such as a handheld tablet. In a clinical setting, this tool should work well as it can be easily sanitized allowing for use by multiple patient. Font and image sizes can be adjusted to accommodate people with visual impairments, and a wide range of interface designs can be used depending on the requirements of users. One strategy for computerized cognitive assessment would be to use existing neuropsychological tests that assess central EFs, some of which are already computerized, such as the go, no-go discrimination task. Given a computerized test (like go, no-go discrimination) of a particular EF like inhibition, different randomized versions could be generated, and RT data could be collected.

26 Chapter 2. Literature Review Methodology for Assessing Game Performance This section will discuss methods to assess performance on a computerized cognitive assessment. These methods include the speed-accuracy trade-off and Fitts Law Speed-Accuracy Trade-off In assessing game performance when provided with response time and accuracy measures, data can be modelled with a speed-accuracy trade-off. In a speed-accuracy trade-off, a user s movement time is linearly related to their accuracy on a task [70]. For instance, lower response times generally result in lower accuracy, and longer response times result in higher accuracy. In [71], a linear relationship (trade-off) between speed and log odds in favour of a correct response was noted. Reference [72] argued that Obtaining an entire speed-accuracy trade-off function provides much greater knowledge concerning information processing dynamics than is obtained by a reaction-time experiment. He noted the variety of methods which could be used to obtain speed-accuracy trade-off functions including instructions, payoffs, deadlines, and partitioning of response time. He also cited research that had used each of these methods. In [73], examples were cited of research where reaction-time results may in fact have been artefacts of underlying speed-accuracy trade-offs. To avoid these problems some researchers analyzed only correct reaction times. For instance, [74] used response times as the dependent measure after exclusion of high-error participants. While some researchers were concerned about the implications of the speed-accuracy trade-off for interpretations of response time results obtained in studies, others examined the speed-accuracy trade-off as an indicator of neural processes (e.g. [70], [75], [76]), and changes due to ageing (e.g. [77], [78]). In spite of the relatively large amount of research on speed-accuracy trade-offs in recent decades, there seems to have been relatively little progress towards[71] s original goal of developing a measure (or set of measures) that could characterize both speed AND accuracy in terms of an overall performance measure. However, there has been related work on how to combine mental effort and performance measures (e.g. [79]). In that work, distance from standardized z-scores to a line representing an efficiency of zero was used to derive an overall measure. The speed-accuracy trade-off can also be used to model game performance and provide insights into changes in cognitive function. For instance, elderly adults compared to young adults demonstrated a decreased speed-accuracy trade-off performance on the Stroop Task [80]. In [81], it was demonstrated that MCI patients exhibit normal accuracy performance compared to normal adults, but significantly longer response times in performing instrumental activities of daily living (IADLs) such as grocery shopping and using a telephone [81]. IADLs are used to assess a patient s functional abilities in completing tasks of daily life. This research demonstrates the significance of differences exhibited in speed and accuracy as potential signs of cognitive decline. This highlights the significance of evaluating both speed and accuracy in task performance, and not just speed or accuracy alone [81], [82]. The speed-accuracy trade-off is also an important measure of game performance and can vary based on different game types. For instance, [83] compared the speed-accuracy trade-off of game performance in action-based versus puzzle-based games. Their findings demonstrated that players in action-based games were faster but less accurate. This study demonstrates that the speed-accuracy trade-off can

27 Chapter 2. Literature Review 16 vary based on the game type and objectives of the game. In an action-based game, faster responses are more desirable, whereas in a puzzle game such as a figure-matching task, an accurate response is favourable. Moreover, using a speed-accuracy trade-off to examine game performance is a useful tool that can provide insights for game design Fitts Law Performance on a game-based cognitive assessment can also be assessed using Fitts Law, which may lend insight into a user s game performance. Fitts Law examines a human s movement time (MT) to move to a target area based on the distance to the target and its size. Fitts developed Equation 2.1, where A describes the amplitude or movement distance (pixels), which is the distance from the user s last hit to the centre of the target. The W represents the width of the target (mole/butterfly character width in pixels). The constants a and b are empirically determined by graphing the relationship between MT and the index of difficulty (ID) of the task (Equation 2.2). ( ) 2A Movement Time = a+blog 2 (2.1) W Index of Difficulty = log 2 ( 2A W ) (2.2) Fitts Law can also be described in terms of a speed-accuracy trade-off such that if a movement of a given amplitude (A) is required to be made within a constrained time (MT) then the precision of the movement (W) will decrease. In a Fitts Law task, a participant uses a pointing device such as a pen to move between two targets drawn on a surface such as a piece of paper [84]. This task can be adapted to current technology using an input device such as a mouse or stylus, and the surface can be a computer user interface or touchscreen tablet [85], [86]. Studies have demonstrated that Fitts Law is maintained on touchscreen interfaces with different input devices such as a bare finger [87], and stylus [88]. Additional research has also demonstrated that ageing has an effect on a Fitts Law task, such that the MT of elderly adults is greater in comparison to younger adults [89], [90], [91], [92], [93]. This research on Fitts Law and elderly participants provides insight into how user interfaces such as those designed for serious games can be designed to consider the abilities and performance of ageing adults. Furthermore, the analysis of human performance on a computerized cognitive assessment using Fitts Law can help inform the design of assessment features such as text and button spacing, and expected performance based on these design considerations. 2.6 Elderly Technology Use The use of touch-based technologies, compared to other computerized options(e.g. computer and mouse) may be easier for novice and elderly users as there is less hand-eye coordination [48], [94]. The use of a handheld tablet may also be easier to learn than a computer. Previous studies comparing young versus elderly users (with a computer and mouse), have demonstrated that elderly users require longer MT [89], [85], [95], [96]. Similar studies have also been performed looking at age-related MT using a stylus as a pointing device on a tablet [97], [98], [99]. The effects of age with a computer mouse and stylus,

28 Chapter 2. Literature Review 17 were observed to be similar. Moreover, age-related effects of MT have also been studied on tablets (e.g. [100], [101]). In [101], it was demonstrated that MT decreased for elderly adults using a tablet compared to a computer mouse. It is hypothesized that this effect was due to the decreased attentional resources required to operate a touchscreen compared to a computer mouse. Research has also demonstrated the successful use of touchscreen technologies with both healthy (e.g. [102], [103], [104]) and cognitively impaired elderly adults (e.g. [105]). Research by [104] has demonstrated that healthy elderly adults were more comfortable, and less anxious, in learning and using a touchscreen compared to a computer [104]. Other research has examined the usability of touch-based technologies with cognitively impaired elderly adults. For instance, the CIRCA project demonstrated the successful use of a touchscreen between elderly dementia patients and their caregivers as a medium for meaningful social interactions [105]. 2.7 Usability Engineering for Clinical Technologies The previous sub-sections focused on cognitive assessments and their use to assess cognitive status. While these are essential components of any healthcare tool design, it is also important to consider the users and environment User Research Methodologies Understanding the needs and requirements of users is important in the design of a healthcare product. This research can help determine the limitations of users, their cognitive capabilities, as well as understand the context and setting for the technology use, and implementation [106]. Common methods include conducting focus groups to help brainstorm features, administering questionnaires that are open or focused, conducting naturalistic studies to observe target user interaction with similar technology, and developing personas and use-cases Evaluation Methodologies The assessment of novel technologies for use in a clinical setting is a key process to ensure that those technologies meet the needs of their users (e.g. physicians, occupational therapists, research assistants, and patients) and their context of use (e.g. hospital ED, neurorehabilition hospital). In human factors engineering, an iterative design cycle is followed consisting of three primary stages: planning, designing, and evaluating [107, p. 28]. Figure 2.1 illustrates the design process. The evaluation phase is often carried out using representative users [107, p. 21]. A method of evaluation includes a usability test, which is a technique used to assess a system [107, p. 21]. Findings from this stage of the cycle can be used to inform future iterations of the design. As mentioned, usability testing can be conducted at any stage during the engineering design life-cycle. There are three primary types of usability tests based on the design cycle: exploratory, assessment, and validation [107, p. 27]. Table 2.2 summarizes the key features of the different types of usability tests. An exploratory approach is usually performed early in the design cycle to evaluate the effectiveness of early design prototypes [108, p. 268]. Next, an assessment usability study is commonly used to evaluate the basic functionality of a product, and to extend the findings from the exploratory tests [107, p. 34]. This stage often involves wireframing and developing prototypes [109, p. 260]. A validation

29 Chapter 2. Literature Review 18 START PLAN - Requirements analysis - Use cases - Define problem and goals DESIGN - Wireframes - Prototypes - Development Iterate EVALUATE - Usability testing - Feedback and findings - Recommendations Figure 2.1: The design cycle process. Table 2.2: Summary of usability tests in the design cycle. Type Stage Description Exploratory Early [108, p. 268] Evaluate prototypes and initial designs [108, p. 268] Assessment Mid-cycle [107, p. 34] Examine effectiveness of prototypes [107, p. 34] Validation Late [109, p. 260] Assessment prototype against a standard [109, p. 260] test is carried out to assess how the product compares to a pre-established standard and to evaluate the product s utility [108, p. 269]. 2.8 Summary There is rising concern about age-related diseases such as delirium in the elderly population. Although there has been progress in research concerns for interventions for cognitive disorders, there is also a need for advances in screening tools. One solution to address current issues with conventional neuropsychological tests is to use computerization and gamification. The computerization of traditional cognitive assessments offers many benefits such as potentially increasing diagnosis rates, and reducing associated costs. It is also important to reduce risk factors leading to cognitive decline by encouraging cognitive activity through the development of stimulating games. The development of serious games as screening tools based on psychometric features of conventional standards can help detect cognitive decline. Through the application of human factors and usability engineering, a game-based cognitive assessment was designed, and will be described in later chapters, and results obtained in usability testing the game in a university environment will also be presented.

30 Chapter 3 Requirements Analysis This chapter will describe the user-centred design methodology followed for this research project, after first describing the requirements analysis conducted to gather information on the type of users and usecases. The subsequent designs developed with the user-centred design methodology were based on the functional needs discovered in the requirements analysis. 3.1 Meetings This section will describe a series of meetings held at two different healthcare sites: Sunnybrook Health Sciences Centre, and Bridgepoint Active Healthcare. Sunnybrook Health Sciences Centre is a multidisciplinary research centre with a hospital. It is affiliated with the University of Toronto and has four primary focuses: cardiac health, cancer, women s health, and emergency medicine [110]. Bridgepoint Active Healthcare is a neurorehabilitation hospital part of the Bridgepoint Active Healthcare organization, and is affiliated with the University of Toronto. This hospital provides short-term care for patients with complex conditions including chronic diseases and disabilities [111] Meetings at Sunnybrook Health Sciences Centre A series of unstructured meetings were conducted at Sunnybrook Health Sciences Centre with Dr. Jacques Lee and Dr. Mary Tierney, to learn about delirium in a hospital ED, and how it is assessed. These meetings helped develop an understanding of their requirements for a novel cognitive assessment application. Each session took place in a meeting room at Sunnybrook Health Sciences Centre, and lasted approximately 60 to 90 minutes. The interview notes were typed up on a laptop. Analysis of the meeting data informed the types of cognitive assessments currently used to screen for delirium, as well as challenges with these methods. In addition, the meeting participants discussed the type of technology they would like to use as well as specific software features. Key points learned from these meetings included the suggested goals: A digital screening tool administered using a portable, lightweight, and hygienic device. A device that can securely store data and meets hospital standards on technology security and privacy requirements. 19

31 Chapter 3. Requirements Analysis 20 Use of an entertaining, game-based screening tool to assess changes in cognitive status. A customizable screening tool that can be used by both researchers and patients. A game-based tool that can be played serially to observe potential health trends. An easy game to understand and play due to the potentially complex health condition of patients. Meeting Summary The insight and information captured from these informal meetings were categorized into key areas of focus. The data was used to identify users needs, and establish requirements for solution development. An affinity diagramming technique was used to organize ideas from the meetings at Sunnybrook Health Sciences Centre into an easy-to-understand format [107, p. 248]. This method was used to develop some of the use cases and features, and written down on sticky notes. Each sticky note (i.e. use case or feature) was grouped with similar sticky notes. Through this process, the following categories were identified as common patterns across the data: Patient Safety and Considerations, Technology Requirements, and Software Features. Yellow sticky notes represent the themes, pink sticky notes denote features and use cases, and blue sticky notes represent more specific functions (Figure 3.1) Meetings at Bridgepoint Active Healthcare Another set of meetings were held at Bridgepoint Active Healthcare with Dr. Tammy Sieminowski, and a team of occupational therapists (OTs). These meetings were focused on demonstrating working prototypes, and initial design concepts, to this team. OTs were included in the discussion as they administer cognitive assessments to evaluate the ability of patients to complete tasks such as Activities of Daily Living (ADL) (e.g. mobility, showering, eating, toileting) and Instrumental Activities of Daily Living (IADL) (e.g. paying bills, grocery shopping, managing medication). These meetings focused on usability whereby the OTs played with the tablet-based game and provided their feedback on the ease of use, graphics, and provided additional insight for future iterations. Standard tests used in their therapy sessions to assess their patients, were also discussed. Key points learned from these meetings included the following: The type of cognitive assessments administered in neurorehabilition (e.g. MMSE, MoCA) and the ED (e.g. CAM, MMSE) vary. OTs administer cognitive assessments focused on functional ability such as the Functional Independence Measure (FIM). 3.2 Requirements Analysis The following sections discuss the current state of, and barriers preventing administration of cognitive assessments. Based on the analysis of the current state, requirements for the proposed technology are listed.

32 Admin features Admin can sign in to check results Can be used by both admin and end-users (patients) User features Log on and off Portable Customizable features Lightweight Technology Requirements Various levels of difficulty Files must be secure and password-protected Software Features Low-cost solution Figure 3.1: Affinity diagram. Easy to use Fun to play Record data Minimal text Instructions Easy to sanitize for between patient-use Patient Safety and Considerations Lightweight Chapter 3. Requirements Analysis 21

33 Chapter 3. Requirements Analysis Managing Patients In the context of an ED, patients at Sunnybrook Health Sciences Centre are assessed for delirium using the CAM at the time of admission [Dr. Jacques Lee, Personal Communication, February 2013]. In contrast, patients receiving neurorehabilitation at Bridgepoint Active Healthcare receive the MMSE, and MoCA within a week of the patient s admission. Other tests such as the FIM are administered within 72 hours of admission and discharge [Dr. Tammy Sieminowski, Personal Communication, June 2013]. These two clinical examples demonstrate the variability of both the type of assessment administered, and the frequency of administration. Challenges Since delirium varies over time, its presence may be missed if assessments of cognitive status are made infrequently. In addition, the method of administration requires a trained healthcare professional to administer the test. Thus, adequate testing may or may not occur, depending on the worker s time, workload, and expertise. Requirements The following requirements were derived from these observations: Provide the ability for serial testing of cognitive status. Record information such as date, time, and score automatically. Provide self-administered tests of cognitive status, whose scores/results can be reviewed by a healthcare professional at a later date and time Recording Test Results The results from a patient s assessment are currently documented in paper-and-pencil format. This information can be collected by a number of people on the patient s healthcare team, such as the physician, research assistant, or OT. There are many advantages in documenting patient information in paper format, as opposed to electronically: Inexpensive and readily available: The cost of acquiring, and using paper-and-pencil to record data is comparatively lower compared to technology. Little to no training: Healthcare professionals require little to no training on how to record the data results on paper. Portable: Information kept on paper is easy to store and transport. Secure: Paper charts can be locked in cabinets or kept in an office. Reliable: Paper can be more reliable than technology in the context of not requiring power, software updates, and potential hardware failures.

34 Chapter 3. Requirements Analysis 23 Hygiene: Healthcare workers follow safety regulations such as proper hand sanitization to minimize disease transmission. However, using paper documentation also has the following shortcomings: Illegibility: Healthcare settings such as hospitals are fast-paced, which can lead to shorthand documentation and illegible writing. Damaged or misplaced: Paper can easily go missing or be damaged by water and other physical stressors such as tearing. Non-searchable: Data documented on paper cannot be easily searched through and compared. Challenges Cognitive assessments regarding a patient are most often recorded on paper by various individuals, and rarely computerized. As outlined, there are many issues with using paper to record sensitive patient data. From the meetings held at Sunnybrook Health Sciences Centre and Bridgepoint Active Healthcare, meetingparticipantswereopentotheideaofusingatouch-baseddevicesuchasatablet torecordpatient data. This technology medium would ideally encompass similar advantages as paper while minimizing and potentially ]eliminating its challenges. Requirements The following requirements were derived from these observations: Records data easily. Use a portable, and lightweight device that requires minimal training. Use secure data storage Summary of Requirements Analysis Based on the literature review, and information acquired in meetings, requirements concerning features and functionality for the cognitive assessment tool were developed. The requirements are guidelines for how the cognitive assessment should function, and the features it should have. The requirements have been categorized into three groups: hardware requirements, user requirements for researchers, and user requirements for patients (Table 3.1). From the requirements analysis, two user groups were identified: researchers and patients. The research user group refers to those administering the cognitive assessment: physicians; clinicians; OTs; researchers. The patient user group consists of end-users, individuals completing the cognitive assessment. These requirements will be used to assess the screening tool in Chapter Use Cases Based on the meetings, a set of use-cases were developed to describe the types of users, and their interaction with the screening tool [107, p. 118]. As mentioned, two primary user groups have been

35 Chapter 3. Requirements Analysis 24 Table 3.1: Requirements analysis summary. Hardware Researchers Patients Inexpensive. Portable and lightweight. Easy to sanitize. Long battery life and reliable. Easy to replace if lost. Require little to no training to use. Ability to register patients, administer a game, and review patient results. Easy to transfer data. Easy to sanitize. Easy to learn, play, and understand. Be able to play the game independently. Easy to sanitize. identified: researchers and patients (Figures 3.2, 3.3). The primary goal for patients with this tool is to play through the game-based assessment, and review their score with the clinician. The goal is to motivate patients to continuously play the game to keep cognitively active while trying to improve their score. The information used to develop these use-cases were informed by the meetings with physicians and OTs who interact with elderly adults with cognitive impairments. No meetings were held with cognitively impaired patients. Moreover, for researchers, the primary goals of the system are to register a patient, set up the game to be played by the patient, and extract patient data. Below are the use cases from the perspective of a patient: Register as a new patient. Sign in as an existing patient. Select custom game settings and play the corresponding game. Repeat the game they just played. View results. Below are the use cases from the perspective of a researcher: Register a new patient. Set up a game for a patient to play. See a list of registered patients. Review a patient s results. 3.4 Scope of the Design Solution The goal of this project is to propose a design and working prototype for a novel game-based cognitive assessment to screen for changes in cognitive status, through user-centred design. The design solution will be evaluated through a usability study conducted in a controlled environment.

36 Chapter 3. Requirements Analysis 25 Sign On Patient Patient Database Table Start Game Select Game Level Game Results Game Game Database Table Sign Off Figure 3.2: Diagram visualizing the use cases for a patient. Researcher Sign On Patient Database Table View Patient List View/Extract Game Results Game Database Table Set Up Game For Patient Sign Off Figure 3.3: Diagram visualizing the use cases for a researcher.

37 Chapter 4 Design and Prototyping of a Game-Based Cognitive Assessment The requirements established in the previous chapter were used to inform the design of the user interface, and interaction design process. Through iterations of sketching and wireframing, the requirements were translated into visual formats and programmed into a working prototype. The design exercise primarily focused on proposing a design solution with the following properties: Game-based screening tool: The software should be a game-based tool that measures cognitive function that is also easy and fun to play. Allows easy test administration: In a hospital setting, healthcare professionals such as physicians, clinicians, and researchers have limited time to spend with each patient. Minimizing the amount of time needed to administer a test can help minimize their workload and increase the number of patients that receive the assessment. Allows patient-initiated use: The software should be easy to use to help motivate patients to play the game at their leisure. It should allow patients to independently set up the game and play multiple times over the course of their therapy or hospital stay. Allows serial testing over a continuous time period: The goal of collecting more data from the patient is to observe a trend in their game performance results. As cognitive disorders such as delirium is a fluctuating condition, a patient s cognitive status can change at random. Designing software that can be easily repeated, enables a patient to easily play the game. This chapter discusses the details of the interface, along with the interaction design decisions made, and the rationale behind them. 4.1 Game Concept Design Based on the literature review of how to assess cognitive function through central EFs and the discussions held at Sunnybrook Health Sciences Centre and Active Healthcare, a serious game based on the carnival 26

38 Chapter 4. Design and Prototyping of a Game-Based Cognitive Assessment 27 game called whack-a-mole was created. In the classic rendition of the game, a mole character appears and the objective is for the player to whack the mole using a mallet. The objective of the game is to hit as many moles as possible within a given time frame, and to avoid hitting a distractor character (e.g. butterflies). The whack-a-mole game was adapted to emphasize the central executive component of working memory [112], [113] and the inhibitory function in particular. This game was selected as it mimics properties of a go, no-go discrimination task, wherein users respond only to particular targets (i.e. moles). The goal is to assess a user s ability to inhibit the whacking response on distractor trials (trials where a butterfly, no-go target, is present), since inhibition ability has been shown to decline with age, independent of processing speed [113]. While the basic whack-a-mole game seems most closely related to inhibition, it can be modified in various ways to assess different central EFs. For instance, this game concept can be easily changed to implement features of other cognitive assessments such as the TMT Part B and DSST. In these versions, the game could be designed to have moles appear in a particular order with the requirement that users respond by hitting the numbered moles in sequence (adding a requirement to use the updating EF). A further modification would have different rules (e.g. hitting the moles in increasing versus decreasing order) allowing the game to test shifting abilities (in a similar fashion to a Wisconsin Card Sorting Task [22]). Alternatively, shifting ability could be tested using a TMT Part B, approach where moles need to be hit in an alternating number-letter sequence. In the version of the game developed for this research, a finger is used to tap a mole on a touchbased device. A matrix represents the number of holes that a character can appear from. The game has different levels of difficulty, depending on the grid size, the target size, and the number of different character types, as explained below. When a distractor character is added, a butterfly appears (a different distractor character could be chosen, but the butterfly is the default). This distractor character is not meant to be whacked, so the user must suppress or inhibit the response of hitting a character when it appears (pops-up out of one of the holes in the grid). Note that in principle any image could be used to define the go (hit), and the no-go (do not hit), characters. The mole was chosen because of its association with the whack-a-mole game, while the butterfly was chosen because it is a delicate and attractive insect that people would not normally want to hit Game Play Features This subsection describes the dynamics of the game with special consideration given to addressing the features identified from the requirements analysis discussed in the previous chapter. To address the requirement of configurable game settings, a set of adjustable game parameters were designed, which include target size, grid size, distractor style, feedback style, and game duration. The features the following game features are customizable settings: Character size: there are three target sizes: 150 pixels, 175 pixels, and 200 pixels. This feature of the design was selected to assess which target size is most usable for users. Grid size: this option describes how many rows and columns the game board will have (2 x 2, and 3 x 3). This feature modifies the number of potential holes that a character can appear in. Distractor style: this option allows users to select whether they want to play a game with only moles or a game with both moles and butterflies (the distractor character).

39 Chapter 4. Design and Prototyping of a Game-Based Cognitive Assessment 28 Figure 4.1: Google Nexus 7 tablet [1]. Feedback style: this parameter allows users to select whether they want feedback when they hit a character. In the feedback mode, a check-mark appears over the mole if it is hit, and a X appears over the butterfly if it is hit. Game duration: this option allows the user to specify the duration of the game. Media controls: a set of pause, start, and stop buttons were included in the design to allow the user to start/pause/stop the game at their convenience. These controls are useful in a busy hospital environment where there are many interruptions Hardware Considerations Based on the findings of the requirements analysis, a touch-based tablet was selected for use. This medium was chosen since tablets are wireless, portable, and many charge via USB. They can also be easily sanitized with rubbing alcohol to minimize disease transmission. For this research project, the tablet selected for use is the Google Nexus 7 tablet, which measures x 120 x mm, and weighs 340 g (Figure 4.1) [1]. This device can be easily held in one hand due to its small size, and weight. It has a high-resolution capacitive touch screen and rubberized back cover, which helps with grip. It is also one of the more affordable tablets available on the market. The tablet battery charges via USB, which also allows data to be easily transferred via this connection Software Considerations The software was developed on the Android platform, as it is an open-source operating system, and free to use. The game is designed to collect data by logging each touch event and storing this information in the device s internal storage, which can be subsequently transferred to a computer via USB cable.

40 Chapter 4. Design and Prototyping of a Game-Based Cognitive Assessment Prototype Development The design was focused on developing a game that can be used by both researchers and patients, which were the two user groups identified from the requirements analysis discussed in the previous chapter. In particular, design solutions for the following tasks were explored: Register new patients to play the game. See a list of registered patient and their results. Select custom settings for a game. Record data automatically and store it securely Low Fidelity Paper-and-Pencil Prototype The preliminary concepts were sketched on paper (Figure 4.2). For the first concept, the design was focused on creating the least number of wireframes possible for a minimum viable product. This strategy was adopted to generate a first prototype faster in an agile process. An overview diagram depicting these screens can be seen in Figure 4.3. In this diagram, each wireframe in the paper-and-pencil prototype has been represented by the name or purpose of screen. Figure 4.2: Paper-and-pencil prototype sketch of the over-arching screening tool software. The wireframe begins on the home/welcome screen (top left screen).

41 Chapter 4. Design and Prototyping of a Game-Based Cognitive Assessment 30 (if patient) Sign On Settings Game Score Screen (if researcher) Patient List Results Figure 4.3: Overview of the whack-a-mole game based on the low-fidelity, paper-and-pencil prototype Medium Fidelity Prototype The low-fidelity wireframes were then translated into a digital, interactive prototype using software called Axure [114]. This prototyping tool was used as it is provides the ability to rapidly create user interfaces with dynamic content and conditional logic. The medium-fidelity prototypes developed in this stage included two new wireframes: an instructions screen and a sign off screen. An updated overview of the current whack-a-game application can be seen in Figure 4.4, where the two new screens are highlighted in blue. An instructions screen was designed to enable a patient to play the game independently, without the assistance of a researcher. In this wireframe, instead of dedicating an entirely new screen to display the instructions to a user, this information has been designed to appear in a modal window that hovers above the current screen. A sign off feature was included to enable a user to sign off from the application to protect their data, and privacy. In a healthcare environment such as a hospital ED where there are constant interruptions, this feature would assist in preventing a different user from using the application. Export Data Sign! Menu Settings Instructions Game Score Screen Results Sign Off Tutorial Figure 4.4: Overview of the medium-fidelity prototype. New wireframes are highlighted with a blue border.

42 Chapter 4. Design and Prototyping of a Game-Based Cognitive Assessment High Fidelity Prototype The first working prototype was developed in Java using the integrated development environment Eclipse [115]. In this prototype, new wireframes were added to the flow of the whack-a-mole game (Figure 4.5). Each screen will be described in further detail below. Export Data Sign Menu Settings Instructions Game Score Screen Results Sign Off Tutorial Figure 4.5: Overview of the whack-a-mole game in the working prototype. New wireframes are highlighted with a green border, and wireframes added in the medium-fidelity prototype are highlighted with a blue border. In this working, high-fidelity prototype, the first screen that a user is presented with is the sign on screen. There is a separate sign on for an administrator (e.g. test administrator, healthcare professional, researcher) and patient (Figure 4.6). If the user is an administrator, they can sign in through the administrator sign on screen. After signing in, the user has the option to view the patient list or export the results from all patients (Figure 4.7). The patient list displays all registered patients in a vertical list that can be scrolled through. There is also an option to register a new patient (this is the same screen used to sign on as a patient in Figure 4.6. These screens are only available to an administrator to prevent other users (e.g. patients, participants) from viewing the confidential patient list. Moreover, administrators have the ability to export data from the application. Data is exported in the form of a SQL file. If the user is a patient, they will be asked to sign on with their patient identification (ID) name. After signing in as a patient, the user is displayed a game settings screen. On this screen, the user can select a variety of game settings (Figure 4.8). Configurable game parameters are made available to the users to allow them to modify the game based on their capabilities. This feature also satisfies a researcher s requirement for configurable software. After setting the game parameters, the user is displayed the whack-a-mole game. On the whack-a-mole game board wireframe, the game board is displayed in a large matrix, as this is the focal point of this screen (Figure 4.8). Additional information such as the user s score and remaining time left in the game are shown on the top of the screen. The location of this secondary information was selected as this is generally where essential but non-primary information is displayed. This location was also selected to minimize information access cost, thereby allowing the user to quickly glance at

43 Chapter 4. Design and Prototyping of a Game-Based Cognitive Assessment 32 (a) First screen displayed to all users. (b) Sign on option as an administrator. (c) Sign on form as a patient. Figure 4.6: Wireframes depicting the sign on and registration screens. this information without losing selective attention on the whack-a-mole game [116]. In addition, a set of simple of instructions are displayed on the bottom of the screen to help remind the user of the task. The whack-a-mole game has been programmed to collect ambient data (e.g. time, date, touch position) of the user s interaction with the tablet while playing the game. This data is automatically recorded in a database to satisfy the requirement for secure data storage. Furthermore, after the user completes the whack-a-mole game, an end screen is shown. On this wireframe, the user can either go back to the game menu or the administrator can select a different patient to administer the game with Summary Through this prototyping exercise, different prototypes of varying fidelity-levels were designed. This process demonstrates how each wireframe is developed using the information gathered from the requirements analysis. Figures 4.9, and 4.10 illustrate the same wireframe at different stages.

44 Chapter 4. Design and Prototyping of a Game-Based Cognitive Assessment 33 (a) Sign on form as a patient. (b) List view of all the patients registered in the game. Figure 4.7: Wireframe depicting the sign on and registration screens. (a) Configurable game menu wireframe. (b) Whack-a-mole game board. (c) End screen after the whack-amole game is complete. Figure 4.8: Wireframes illustrating the game menu with different setting, whack-a-mole game board, and end screen.

45 Chapter 4. Design and Prototyping of a Game-Based Cognitive Assessment 34 Select Level: Level 1 Select Go and No, Go Duration: 5 seconds 60 seconds Select Game Duration: Select Go Character: Select No-Go Character: START GAME (a) Low-fidelity wireframe. (b) Medium-fidelity wireframe. (c) High-fidelity wireframe. Figure 4.9: Comparison of the game settings screen. (a) Low-fidelity wireframe. (b) Medium-fidelity wireframe. (c) High-fidelity wireframe. Figure 4.10: Comparison of the whack-a-mole game board.

46 Chapter 4. Design and Prototyping of a Game-Based Cognitive Assessment 35 START DESIGN Design specifications including detailed wireframes, and usecases created in Canada. EVALUATE Usability testing of latest software from Thailand conducted in Canada. DISCUSS Discussion of design specifications conducted via and Skype with both teams from Canada and Thailand. DEVELOP Software developed based on design specifications in Thailand. Figure 4.11: The design cycle process followed for the collaboration with King Mongkut University, Thonburi Collaboration with King Mongkut University of Technology Thonburi This sub-section will describe the international collaboration with researchers from King Mongkut University of Technology Thonburi in Thailand. A research team led by Dean Nipon Charoenkitkarn assisted in the translation of the prototypes designed for this project into a working prototype. The developed prototype was programmed in Java using an open-source version of whack-a-mole. Through this process, design specifications were created for discussion with the team in Thailand. Following this stage, a version of the software was developed by the Thailand team and pilot tested in the Interactive Media Lab. A new set of design specifications detailing usability issues were documented. This iterative process was followed throughout a series of prototype iterations (Figure 4.11).

47 Chapter 4. Design and Prototyping of a Game-Based Cognitive Assessment 36 Figure 4.12: Visual depiction of 48 dp compared to a button [2]. This was a unique learning experience as it involved international collaboration whereby the design and usability testing were conducted in Canada, while development efforts were from Thailand. I eventually developed the working prototype evaluated in this thesis by myself. Reasons for this change in development strategy included having direct access to the software code and the ability to modify the software in real-time without having to create a set of design specifications for small changes. Moreover, lessons learned from this international collaboration included the importance of effective communication, especially in the creation of design specifications. Other lessons include how to incorporate and utilize different skill sets into a project. 4.3 Design Elements This section discusses the design elements produced in the first working prototype in detail, with particular emphasis on features such as buttons, and typography Button Size By default, Android graphic elements such as buttons and text fields are 48 density-independent pixels (dp) high, which translates to a physical size of 7 mm (Figure 4.12). This is their recommended minimum element size for both smartphones and tablets [2]. This design guideline was followed and all elements used in the working prototype are at minimum 48 dp in height and width. This guideline was used only as a lower limit as elderly adults might have vision problems, and larger text and images would be favourable Typography The minimum font size used across the entire application is 18 scale-independent pixels (sp). By Android Developers standards, this is considered to be a medium text size [117]. This minimum font size was selected to minimize accessibility issues with reading text on a small screen. The primary font colour selected is a dark grey displayed against a light coloured background. This contrast was selected to help minimize eye strain when reading text, increase readability, and facilitate faster searching [118]. The contrast between text and background colours is also regarded as more favourable [119].

48 Chapter 4. Design and Prototyping of a Game-Based Cognitive Assessment 37 (a) Mole character (b) Mole character with feedback (c) Butterfly character (d) Butterfly character with feedback Figure 4.13: Example of characters used in the game Character Design and Selection The mole and butterfly characters were designed using contrasting colours and shapes to help users distinguish between the two designs (Figure 4.13). These two characters were selected as they can be easily discriminated between by the user, as similarity can cause confusion [116] Compatibility At present, the latestversionofandroid is 4.4, and the softwarewasdeveloped foruse ondevicesrunning on Android 2.2, which was released May 20, This feature allows for backwards compatibility, which also serves to expand the number of devices this software can be used on (Figure 7.6). 4.4 Summary The user requirements gathered in the previous chapter were used to inform the design process. The application was first conceptualized using pencil-and-paper prototypes. These low-fidelity wireframes were used as guidelines for the medium-fidelity prototype, which were then translated into a working, high-fidelity prototype.

49 Chapter 5 Methodology This chapter will detail the methodology of an empirical study conducted to investigate human performance on a game-based cognitive assessment. This study explores the usability of a touch-based device to record data from users playing a game. 5.1 Goals The study was designed to address the following questions: 1. What usability issues are associated with using a tablet version of the whack-a-mole game? 2. How can the whack-a-mole game be calibrated? What are the rules for determining what a hit/point is? 3. Is there a Fitts Law component to the whack-a-mole game? 4. How does the whack-a-mole game relate to central EFs? 5.2 Hypotheses 1. Cognitive abilities can be reflected in whack-a-mole game performance. 2. Game performance touch-data can be modelled using Fitts Law Participants For the usability study described below, all participants were recruited from the University of Toronto. The following were the inclusion criteria: Age 18 years or older; Can read texts on a computer monitor display and hand-held tablet; Have no self-reported difficulty in hearing or understanding English instructions; 38

50 Chapter 5. Methodology 39 Have no self-reported problem distinguishing between red, blue, purple, green, yellow, orange, brown and gray, and Have no self-reported history or current cognitive impairments (e.g. MCI, dementia, delirium). These criteria were listed to ensure that all participants were capable to consent and participate in the study. The usability study was conducted with 24 healthy participants, consisting of 7 females and 17 males, aged 21 to 51. There were 21 right, 3 left-handed participants, and no participants reported being ambidextrous. 5.3 Experimental Design The experiment consisted of three parts: (i) a demographic and technology use survey, (ii) cognitive capacity tests, and (iii) whack-a-mole game, and a tablet use questionnaire. The study began by asking participants to complete a questionnaire on their demographic information and use of touch-based technologies. Next, participants were asked to play three games on a computer that assessed their cognitive abilities. Lastly, participants were asked to play the whack-a-mole game on the tablet. This was followed by an exit questionnaire asking users to evaluate their experience with the tablet-based game consisting of 9 items to be answered on a 5-point Likert scale Background Information Questionnaires Participants were asked to complete a survey on their demographic information and technology use (Appendix A). No personally identifiable information was collected. These two questionnaires were used to gain background information on the participants and to understand their familiarity with touchbased technologies Cognitive Capacity Study In this part of the study, participants were asked to complete a series of four different cognitive capacity tasks that evaluate central EFs. The four different cognitive capacity tasks were from two different software packages. The first package was developed by PEBL [120]. The second was developed for Vocalage Inc. by Pierre Duez and Dr. Sachi Mizobuchi, under a contract with the Toyota IT Centre in Tokyo [3]. This part of the experiment was performed using a computer, monitor, standard US keyboard, and mouse. The four tests completed by the participants are described below. Stroop Test (Inhibition) The Stroop Test is a classic psychology test that displays a colour word on the screen that can appear in a different colour than the meaning of the word. Participants were asked to play two different versions of the Stroop Test. In the first version of the task by PEBL [120], words (e.g. red, green, blue, yellow) appear on the screen one at a time in random order. On incompatible trials the meaning (colour name) of the word is different from the ink colour that it appears in. The objective is to respond to the colour of the ink and

51 Chapter 5. Methodology 40 Figure 5.1: Screen capture of the Stroop Test software developed by PEBL. Figure 5.2: Screen capture of the Stroop Test software developed for Vocalage/Toyota [3]. to ignore the name of the word. During each trial, there are four options that the user can select. The users indicate their responses using a keyboard, pressing the number 1, 2, 3 or 4 (Figure 5.1). In the Stroop Test developed for Vocalage/Toyota [3], the word can appear in six different colours: black, white, yellow, orange, purple and green. Only one word is displayed at a time and the sequence is random. During each trial, there are three response options that the user can select. To respond the user must use a keyboard and press either the left arrow, down arrow or right arrow key that corresponds with the answer (Figure 5.2). Wisconsin Card Sort Test (Shifting) In the Wisconsin Card Sort Test (WCST), a set of four cards are displayed to the user that differ based on the following properties: colour, number of objects on the card, and shape. There is a fifth additional card that the user must sort. The user can sort this card based on any of the properties mentioned. After each sort, the user will receive feedback if they sorted the card correctly. The sorting rule is random and changes throughout the game (Figure 5.3). The WCST was used to assess a user s shifting ability. Colour Monitoring Test (Updating) In the Colour Monitoring Test, the user is shown one coloured circle at a time that can either be blue, red or yellow. The goal is to count the number of appearances of each coloured circle. The goal is

52 Chapter 5. Methodology 41 Figure 5.3: Screen capture of Wisconsin Card Sort Test. to indicate when the third appearance of a circle of a particular colour appears. Users indicate their responses by pressing the down arrow key on a keyboard. If the user responds too early, the count for that specific colour sequence is reset to zero. If the user fails to indicate that the third appearance of a colour appears, then a X appears on the screen. The Colour Monitor Test is designed to assess a user s updating ability Whack-A-Mole Study The design of this experiment is shown in Table 5.1. The tablet-based portion of the study was designed using a within-subjects repeated-measures design. The independent variables were the four adjustable game parameters: target size, grid size, distractor style, and feedback style. Each trial lasted 20 seconds, and there were 8 blocks of six trials each in total. The four distinct combinations of grid size and distractor style (i.e. 2x2 grid with moles only, 2x2 grid with moles and butterflies, 3x3 grid with moles, and 3x3 grid with moles and butterflies) were presented in the first four blocks in an order intended to facilitate learning (smaller to larger grid size, and no-distractor to with-distractor). The second four blocks were then presented in the reverse order to provide a measure of counterbalancing. In the first half of the experiment (blocks 1 4), there was no feedback provided to users, and feedback was then provided for the second half of the experiment. For the six trials within each block, each target size (150, 175, 200 pixels) was repeated twice and the order of target sizes within the block was randomized (subject to the constraint that all three target sizes were shown twice). Participants were encouraged to take breaks after a trial whenever they felt like it. The entire experiment took approximately 60 minutes to complete. The study was conducted over a period of one week. In summary, the design was: 24 participants x 3 target widths (150, 175, 200 pixels) x 2 grid sizes (2x2, 3x3) x 2 distractor styles (distractor absent, distractor present) x

53 Chapter 5. Methodology 42 Table 5.1: Experimental design of the tablet-based study. BlockGrid Size Distractor 1 2x2 M 2 2x2 MB 3 3x3 M 4 3x3 MB 5 3x3 MB 6 3x3 M 7 2x2 MB 8 2x2 M An example of the experimental design for a participant. Blocks 1 through 4 are games with feedback absent, and blocks 5 through 8 are games with feedback present. M stands for mole only games, and MB stands for games with both moles and butterflies. 2 feedback styles (feedback absent, feedback present) x 2 repetitions = 1152 trials in total For this portion of the experimental study, a Google Nexus 7 tablet (Figure 4.1) was used to play the whack-a-mole game. After the data collection on the tablet, a USB cable was used to transfer the data from the tablet onto a computer. An exit questionnaire was used to gain insight into the usability of the tablet (Appendix B) Post-Whack-A-Mole Questionnaires Participants were asked to complete a survey (Appendix B), on their experience with the whack-a-mole game after playing it on the tablet Data Analysis Data was processed using MathWorks MatLab [121] and Microsoft Excel [122], and analyzed using IBM SPSS [123].

54 Chapter 6 Results This chapter will present the results from the usability study discussed in Chapter Background Information Questionnaires The following data reported was gathered from the demographic and technology-use questionnaires given at the start of the experiment (Appendix A). All participants reported using a computer on a daily basis in comparison to only 20 participants that reported using a touch-based device daily. All of the participants reported some use of both a touch-based device and a computer (Figure 6.1). The most frequently used touch-based devices were smartphones (N = 13), laptops (N = 7), and tablets (N = 6) (Figure 6.2). 6.2 Cognitive Capacity Study The following sub-section will report the data gathered from the four cognitive ability tests administered on the computer. Observations from this portion of the experiment will also be described Cognitive Capacity Test Results A correlation analysis was performed using the median correct reaction times (CRT) from the cognitive capacity tests performed on the computer. This analysis was performed to gain an understanding of the relationships between the different EFs. The inhibition scores from both versions of the Stroop Test were included in this analysis. The inhibition ability score calculated from the Vocalage/Toyota version of the Stroop test is denoted as inhibition Vocalage/Toyota. There was a significant relationship between the three different EF scores (Table 6.1). This could suggest that inhibition ability may not be independent of shifting and updating ability. A Venn diagram has been created to illustrate the relationships between the three EFs (Figure 6.3). Squared partial correlations were used to determine the percent overlaps between the EFs (and corresponding circles that depicted those EFs in Figure

55 Chapter 6. Results 44 Figure 6.1: Frequency of device usage. Figure 6.2: Most frequently used touch-based devices.

56 Chapter 6. Results 45 Updating 14% 3% Inhibition 36% Shifting Figure 6.3: Venn diagram illustrating the relationship between the three EFs. PEBL version of inhibition is represented in the diagram. Squared partial correlations were used to determine the percent overlaps between the EFs. Squared partial correlations were used to determine the percent overlaps between the EFs. Table 6.1: Cognitive abilities correlations. Inhibition Inhibition Vocalage/Toyota Shifting Updating Inhibition 1.561**.610**.373* Inhibition Vocalage/Toyota.561** 1.363*.293 Shifting.610**.363** Updating.373* Absence of an asterisk next to a value in the table indicates that the value was not statistically significant (p >.05), **p <.001, *p <.05. Inhibition Vocalage/Toyota are scores from the Stroop Test developed by the IML [3].

57 Chapter 6. Results Observations This following subsection will describe observations noted while participants were completing the cognitive capacity tasks. These observations focus on the use of strategies, which may have influenced a participant s performance on the cognitive capacity tests. Stroop Test While completing the Stroop Tasks, some participants were observed using strategies such as reading either the meaning of the word or the colour of the word aloud. WCST Test There were no observable strategies used by participants during this task. Colour Monitoring Test In the Colour Monitoring Test, there were two strategies that were observed. The first strategy involved participants using their non-dominant hand to count the circles. In the second strategy, participants recited the count aloud. 6.3 Whack-A-Mole Game Performance Data Analysis This section will present the results from the whack-a-game played on the tablet. The game performance data will be analysed using a proposed standardized performance metric, which focuses on the interpretation of reaction time and accuracy data. Next, the data will be analyzed in terms of a speed-accuracy trade-off, and the applicability of Fitts Law to the data. To conclude, the relationship between the game performance and cognitive abilities data will be examined Standardized Performance Metric The two primary game metrics gathered from the usability study are RT and accuracy, which are measured in two different units: seconds and pixels, respectively. To normalize these metrics, the z-score of RT and accuracy were used to standardize the values, and address the issue of units. The resulting trade-off relationship between standardized speed and accuracy is described by a line with slope -1 if there is a perfect trade-off between speed and accuracy. Alternatively, a line with slope of +1 describes a situation where people get faster as they get more accurate (i.e. instead of a tradeoff there is a perfect correlation). As shown in Figure 6.4, these two idealized lines intersect at the point (0,0), representing average performance on both speed and accuracy. The difference between an actual pair of standardized speed and accuracy scores and this zero or average point is an indicator of performance relative to the average, with the exact expression depending on the metric used. For instance, in a city-block metric, game performance is proportional to the sum of standardized speed and standardized accuracy. However, if error and RT measures are used then higher performance is indicated by lower values and thus the standardized scores must be negated, i.e. -Z(RT) - Z(error) as is the case in the results reported here (Equation 6.1).

58 Chapter 6. Results Group Tradeoff Correlation Tradeoff: R 2 Linear = 1 Correlation: R 2 Linear = 1 Zscore: Delta Distance From Hit to Target Center Zscore: Delta Time Between Hit and Target Appearance Figure 6.4: Graphical representation between a tradeoff and correlation relationship.

59 48 Chapter 6. Results Figure 6.5: Game performance data analyzed using standardized scores of accuracy and time. Speed-Accuracy Trade-Off The standardized values of RT, -Z(time), and accuracy, -Z(accuracy), can then be modelled using a speed-accuracy trade-off, which describes the relationship between RT and accuracy [70]. The Z(time) was plotted on the x-axis, and -Z(accuracy), the distance from a user s hit to the centre of the target, on the y-axis (Figure 6.5). For this plot, the data pooled across participants and conditions. In this analysis, only the RT and accuracy of hitting a mole are taken into account. A linear fit with a R2 = was observed, which suggests that participants are more accurate (less pixel distance) when they take longer to respond (hit the target). Overall Performance Score = Z(accuracy) Z(time) (6.1) Relationship Between Standardized Performance Metric and Cognitive Abilities To explore which standardized metric best correlates with cognitive abilities, the metrics -Z(accuracy), Z(time), and -Z(accuracy)-Z(time) were calculated for hitting a mole, and correlated with the scores from the cognitive ability tests (Table 6.2). The RT and accuracy standardized scores were also included separately in this analysis for comparison. There was a significant relationship between the following EF scores and the standardized performance metric (Table 6.2). The correlation analysis results suggest that -Z(accuracy)-Z(Time) is a more sensitive measure of EF, compared to -Z(accuracy), and Z(time) alone. This supports the use of this metric when looking at RT and accuracy data. Thus, the remaining data analyses will be focused on the overall performance score defined above.

60 Chapter 6. Results 49 Table 6.2: Standardized performance metric correlated with cognitive ability scores. Inhibition Inhibition Vocalage/Toyota Shifting Updating Z(time) Z(accuracy) Z(accuracy)-Z(time) -.600** * -.354* Absence of an asterisk next to a value in the table indicates that the value was not statistically significant (p >.05), **p <.001, *p <.05. Inhibition Vocalage/Toyota are scores from the Stroop Test developed by the IML [3]. From this analysis, it is also evident that the inhibition scores from the PEBL version of the Stroop Task are more strongly correlated with the standardized performance metric. Thus, for the remaining analyses, the inhibition ability scores from the PEBL software will be used Understanding Game Performance Data Fitts Law The whack-a-mole data was also analyzed using Fitts Law to understand the participants game performance as is related to the difficulty of using different game settings. The participants MT were plotted against the ID of the task. The lack of fit (R 2 < 0.001) indicates that the Fitts Law model did not apply in this context Relationship Between Game Performance and Cognitive Ability Accounting for Cognitive Ability in Game Performance To gain an understanding of how the game performance and cognitive abilities results are related, a repeated measures analysis of variance (RM ANOVA) was carried out. The four game parameters constituted the four factors in three analyses, each with a different one of the three EFs as the single covariate. The median -Z(accuracy)-Z(time) transformation for the RT of hitting a mole was used for this analysis as the dependent variable. The results were analyzed for statistically significant interaction terms (p <.05). The primary interest was in significant interaction effects involving two or more game parameter and an EF (covariate). These interactions are detailed below: Grid size * Feedback * Inhibition, F(1, 20) = 9.68, r = 0.57, and Grid size * Feedback * Shifting, F(1, 20) = 6.29, r = Interaction Between Grid Size, and Feedback Style - Covaried with Inhibition Ability In the interaction between grid size and feedback, the assumption of sphericity is not violated, as the grid size variable only has two levels (2 x 2, 3 x 3) [124]. (The assumption of sphericity is considered in a RM ANOVA when a variable has three or more levels.)

61 Chapter 6. Results 50 There was a significant main effect of the type of grid size on game performance, F(1, 20) = 9.229, r = Contrasts revealed that -Z(accuracy)-Z(time) values with a smaller grid size of 2 x 2 were significantly higher than a larger grid size of 3 x 3. There was no significant main effect of feedback style on game performance, F(1, 20) < 1. There was a significant interaction between grid size and feedback style, F(1, 20) = 9.681, r = This indicates that feedback had different effects on -Z(accuracy)-Z(time) values depending on the game s grid size. Figure 6.6: Scatterplot and regression lines of inhibition against participants z-score for each of the game conditions. Interaction Between Grid Size, and Feedback Style Covaried with Shifting Ability In the interaction between grid size and feedback style with shifting as a covariate, the assumption of sphericity was not violated as assessed by Mauchly s test. There was a significant interaction between grid size and feedback style when covaried with shifting ability, F(1, 20) = This indicates that grid size had different effects on -Z(accuracy)-Z(time) values depending on the game s grid size, after taking into account the effect of shifting ability. In Figures 6.6 and 6.7, the scatterplots shows a relationship between the covariate (inhibition and shifting) and the z-score (participants game performance) for each of the four experimental conditions. These plots all have a negative relationship (the regression line slopes downwards from left to right) between participants z-scores, and their inhibition and shifting ability in all game conditions. This means that users with greater inhibition and shifting abilities (smaller RT) had better performance than those with less of those abilities. These results also reflect findings as predicted by Fitts Law, whereby smaller target areas are more difficult to respond to compared to larger target areas.

62 Chapter 6. Results 51 Figure 6.7: Scatterplot and regression lines of shifting against participants z-score for each of the game conditions. Assessing Relationship Strength Between EF and Game Performance Next, a set of bivariate correlations were conducted between each EF and different combinations of game parameters to explore which settings are most predictive of specific EFs. Following this, partial correlation analyses were performed to determine how each EF and game performance were related. Inhibition Inhibition had a statistically significant correlation with overall game performance and 22 out of the 24 individual game parameters (Appendix C). The correlations between inhibition and the performance metric for the four combinations of distractor style X feedback style have been depicted in Figure 6.8. When both distractors and feedback are present, it is apparent that correlation increased with target size in the smaller grid size (2 x 2) condition. In contrast, when the distractor was present but feedback was absent, the correlation decreased with target size in games with a 2 x 2 grid. In gameswith distractorspresent and the 2 x 2 grid, the increasing size of feedback characters as the target size increased may have been more distracting for participants with lower inhibition ability. Overall, the largest correlation was when distractors were present, and feedback was absent, in the 2 x 2 grid with 150 px targets (r = -0.69). The relatively strong correlations between inhibition ability and the performance metric indicate that the whack-a-mole game is requiring inhibition ability. The relationship occurs broadly across all the game parameters, and when the analysis is restricted to particular game parameters some significant correlations still occurred. While the strongest correlation occurred with the 2 x 2 grid, the small target size (150 px) and in the presence of distractors, significant correlations were also observed for the 3 x 3 grid (medium and large targets) when there were no distractors. Thus it seems that inhibitory ability

63 Chapter 6. Results px 175 px 200 px px 175 px 200 px Grid Size 2x2 Grid Size 3x3 Grid Size 2x2 Grid Size 3x3 (a) Correlation between game performance and inhibition ability in games when both distractors, and feedback are absent. (b) Correlation between game performance and inhibition ability in games with no distractors, and feedback is present px 175 px 200 px px 175 px 200 px Grid Size 2x2 Grid Size 3x3 Grid Size 2x2 Grid Size 3x3 (c) Correlation between game performance and inhibition ability in games with distractors present, and no feedback. (d) Correlation between game performance and inhibition ability in games when both distractors, and feedback are present. Figure 6.8: Correlation between game performance and inhibition ability in games with different combinations of distractor style X feedback style. The error bars represent one standard error. may have been required not only to avoid hitting butterflies but also to maintain general task focus (consistent with [18] s view of inhibition as being part of a common or shared EF). Shifting There were also some(8 out of 24) significant relationships between shifting ability and overall and parameter-specific performance (Appendix C). However, the correlations were not as strong and may at least partially be due to the correlation that exists between the inhibition EF and the shifting EF. Updating There were few (6 out of 24) significant relationships between updating, and overall and parameter-specific performance. This suggests that updating ability is required for some game settings but not others (Appendix C). An alternative explanation is that these correlations are actually due to the shared variance between the updating and inhibition ability measures and are not intrinsically due to updating ability. Next, a partial correlation analysis was performed to assess the relationship between each EF and game performance while controlling for the effect of the other two EFs. For instance, inhibition was partially correlated with game performance while controlling for the effect of shifting and updating ability. This analysis showed that inhibition had the strongest independent relationship with game performance, while the independent relationship with game performance was much smaller for updating and almost non-existent for shifting. To illustrate this relationship, a Venn diagram has been designed to depict

64 Chapter 6. Results 53 Updating 3.2% 14.0% Game Performance 18.5% Inhibition Figure 6.9: Venn diagram illustrating the overlap between EFs and overall performance scores. Note that, since the overlap between overall performance scores and shifting ability accounted for less than one percent of the variance in overall performance scores, that overlap is not shown in the diagram. Squared partial correlations were used to determine the percent overlaps between game performance and EFs. the approximate variance shared between each participant s game performance and their corresponding inhibition and updating scores (Figure 6.9). Squared partial correlations were used to determine the percent overlaps between game performance and EFs Observations While playing the whack-a-mole game, many participants asked if they were able to play the game using two hands/multiple fingers/digits. As noted above, variability in the hand position and in the fingers used probably accounts for why Fitts Law did not apply at the individual trial level, although the overall effect of target size was consistent with index of difficulty (and movement) time being greater for a smaller target. Overall, the participants found the game easy to understand and play. 6.4 Exit Questionnaire At the end of the experiment, participants were asked to complete an exit questionnaire focused on the usability of the game and tablet. A principal component analysis (PCA) was conducted on the questionnaire data to reduce the questions into related usability themes.

65 Chapter 6. Results 54 A PCA was conducted on the nine items with orthogonal rotation (varimax). The KaiserMeyerOlkin (KMO) measure verified the sampling adequacy for the analysis, KMO = 0.598, and only two KMO values for individual items were not above the acceptable limit of 0.5 [124]. Bartlett s test of sphericity χ 2 (36) = , p <.05, indicated that correlations between items were sufficiently large for PCA. An initial analysis was run to obtain eigenvalues for each component in the data. Three components had eigenvalues over Kaiser s criterion of 1 and in combination explained 50.32% of the variance. The scree plot showed an inflexion that would justify retaining 3 components. Given the small sample size (N = 24), and the convergence of the scree plot and Kaiser s criterion on three components, this is the number of components that were retained in the final analysis. Table 6.3 shows the factor loadings after rotation. The items that cluster on the same components suggest that component 1 represents tablet setup, component 2 represents text and image readability, and component 3 represents game speed and discriminability. A reliability analysis was also performed to measure the consistency of the exit questionnaire. Component 1 had a high reliability, Cronbach s α > 0.7, which is the only reliable factor [124]. Components 2 and 3 had relatively low reliabilities, Cronbach s α < 0.5 (Table 6.3). It is interesting to point out that 16 out of 24 participants either agreed or strongly agreed that their touches were not being registered with the tablet. A k-means cluster analysis was performed on the exit questionnaire data to examine which questions are related to each other. Questions 1, 2 and 6 were combined by averaging them into one factor as they are related to the tablet setup. Next, questions 4 and 5 were combined into one variable regarding text and image readability. Finally, questions 8 and 9 were combined into one variable as those questions are related to game speed. Overall, there were four variables in the cluster analysis, averaged questions 1/2/6, averaged questions 8/9, question 3, and question 7. Based on an initial cluster analysis, there were significant differences (p <.05) for the averaged questions 1/2/6 variable and question 3 only. Another cluster analysis was conducted using only these two variables. Figure 6.10 demonstrates that users were split into two predominant groups: (1) users comfortable with the tablet setup and felt that their touches were being registered by the tablet, and (2) users uncomfortable with the tablet setup and felt that some of their touches were not being registered by the tablet. 6.5 Revisiting Hypotheses In summary, the above sections presented the quantitative results of the whack-a-mole game performance data. Each hypothesis will be revisited to report if they have been supported or rejected. Cognitive abilities can be reflected in game performance. By using the standardized performance metric, it was shown that the median RT from the game performance data is strongly related to overall EF ability. Thus, the null hypothesis that game performance is related to cognitive ability is supported. Game performance touch-data is related to Fitts Law at a general level. Response time was found to significantly increase with smaller targets and the larger grid size, as would be predicted by Fitts Law. Thus, the null hypothesis was supported. However, Fitts Law was not predictive for individual trials. This was likely because each trial was composed of a target

66 Chapter 6. Results 55 Table 6.3: Summary of exploratory factor analysis results for the exit questionnaire using rotated factor loadings (N = 24) Item Component 1 (tablet setup) Component 2 (tablet sensitivity and game readability) Component 3 (game speed and discriminability) Q2 - The tablet was easy to use in the portrait orientation. Q6 -The instructions were easy to understand Q1-The tabletwaseasytouseonatable Q5 - The images and text on the tablet were too large. Q4 - The images and text on the tablet were too small. Q3 - The tablet did not register some of my touches. Q7 - The go, and no-go targets were easy to distinguish between. Q8 - The speed at which the targets appeared was too slow. Q9 - The speed at which the targets appeared was too fast Eigenvalues % of variance Cronbach s α Note: Factor loadings over.40 appear in bold.

67 Chapter 6. Results 56 Figure 6.10: Scatterplot of questions 1/2/6 against question 3. Questions 1/2/6 are related to the tablet setup and ease-of-use. Question 3 is related to the feeling of some touches not being registered by the tablet. Cluster 1 represents users that are comfortable with the tablet setup and felt that their touches were being registered by the tablet. Cluster 2 represents users uncomfortable with the tablet setup and felt that some of their touches were not being registered. detection phase followed by target acquisition and Fitts Law only applied to the target acquisition phase of the trial. Since neither the starting time of the target acquisition, nor the starting point of the required motion were controlled, this was not a Fitts Law task at the individual trial level and thus it is not surprising that Fitts Law did not have predictive value for individual trial RTs.

68 Chapter 7 Updated Game-Based Cognitive Assessment Prototype Based on the results and feedback from the usability study, additional features were added and existing functions were refined. In the latest prototype, the first screen that a user is presented with is the sign on screen. From here, the user can register as a new user or sign in using their username. The user is only required to provide a username to register. Compared to the prototype used in the usability study, there is no longer a separate sign on for administrators. (a) Wireframe of the sign on process. This is the first screen that users encounter when first opening the application. (b) Registration screen for new users to sign up. Figure 7.1: Wireframes depicting the sign on and registration screens. 57

69 Chapter 7. Updated Game-Based Cognitive Assessment Prototype 58 A new games menu wireframe has been added to the latest prototype (Figure 7.2). On this screen, the user can select from games with preset settings, or select a custom game. Preset game settings were added to enable a user to quickly begin a game without having to set up a custom game. Also, a user can read an non-interactive tutorial on how the game is played. This tutorial feature was not present in the usability study. It was added in the latest prototype to provide participants with more detailed instructions on how to play the game. The settings wireframe displays the five adjustable settings on one-screen in a vertical list (Figure 7.2). The option for audio was added to the latest prototype and not used in the usability prototype, which had no audio feedback option, only visual feedback. Moreover, a set of basic instructions on how to play the whack-a-mole game have also been added to the top of this screen. (a) Menu screen with different game options. (b) Game settings screen with configurable paramters Figure 7.2: Wireframes of the game menu and settings screens. A set of five tutorial screenswere designed to instruct the user on how to play the whack-a-molegame (Figure 7.4). These screens educate a user on what to do when a mole or butterfly character appear from a hole on the game board. The instructions modal screen is displayed over the menu screen (Figure 7.3). This screen displays a short description of how to play the game and displays an image of the target (mole) and distractor (butterfly) character. This screen is displayed to the user before each game begins. On the whack-a-mole game board wireframe, the game board has been updated with a set of two media control buttons that are located on the top right (Figure 7.3). These media controls were added to allow the patient or administrator to pause the game due to the interruptions. At the end of each whack-a-mole game, the user s results are clearly displayed. In the previous iteration, a patient was unable to view their results, only the administrator had access to this information.

70 Chapter 7. Updated Game-Based Cognitive Assessment Prototype 59 (a) Instructions modal screen. (b) Whack-a-mole game board. Figure 7.3: Wireframes showing the instructions modal screen and whack-a-mole game board. In addition, a separate user s results wireframe has been created to satisfy the requirement of enabling a researcher or patient to view their game performance and past data (Figure 7.5). The information displayed on this screen can assist the user in understanding their current and past cognitive status. This screen appears as a modal window over the current background screen. A sign off wireframe has been designed to allow users to sign out of their session at any time. This feature can be accessed from the top header bar (Figure 7.5). This function satisfies the requirement to preserve a patient s privacy. The ability to export data from the application has been added to the header bar (Figure 7.5). This export feature can be accessed from all screen where the header bar is displayed (except for the whack-a-mole game). The software was also updated to be scalable and responsive such that it resizes to any device resolution and orientation (i.e. portrait or landscape). This enables text and images displayed on a smartphone versus a larger tablet device to be scaled accordingly. Responsive design offers the benefit of flexibility and enables this software to be run on any device with an Android operating system.

71 Chapter 7. Updated Game-Based Cognitive Assessment Prototype 60 (a) Game trial when a mole appears. (b) A user is instructed to tap the mole. (c) In games with feedback present, a checkmark appearas over the mole when it is succesfully hit. (d) Game trial when a butterfly appears. (e) In games with feedback present, a X appears over the butterfuly if it is accidentally hit. Figure 7.4: Wireframes of the tutorial screens on how to play the whack-a-mole game.

72 Chapter 7. Updated Game-Based Cognitive Assessment Prototype (a) Patient results modal window. (b) Wireframe showing the sign off feature in the header bar. 61 (c) Wireframe depicting the export options from the navigation bar. Figure 7.5: Comparison of the game settings screen. (a) Screen capture taken of the game in portrait mode on a Nexus 7 device. (b) Screen capture taken of the game in landscape mode on a Nexus 4 device. Figure 7.6: Example of game board on two different devices.