CRITICALLY APPRAISED PAPER (CAP) Zickefoose, S., Hux, K., Brown, J., & Wulf, K. (2013). Let the games begin: A preliminary study using Attention Process Training-3 and Lumosity brain games to remediate attention deficits following traumatic brain injury. Brain Injury, 27(6), 707 716. http://dx.doi.org/10.3109/02699052.2013.775484 CLINICAL BOTTOM LINE: Attention retraining is a crucial aspect of recovery from traumatic brain injury (TBI). Occupational therapists work directly with individuals recovering from TBI in areas of cognition and attention. The use of computer-based and technology programs as a treatment intervention provides an alternative approach for remediating cognitive deficits experienced by survivors of TBI. Brain-training computer-based programs allow for individuals to work on problem-solving skills, attention, and memory. These programs incorporate widespread availability, decreased cost, immediate built-in feedback, and entertainment into occupational therapy treatment sessions. This study explored whether attention improvements from computer-based treatment programs generalized to improvements on untrained functional tasks. Participants of this study engaged in treatment phases involving the computer-based programs of Lumosity and Attention Process Training 3 (APT-3). The researchers assessed 4 male participants recovering from severe TBI with standardized and nonstandardized measurements in a clinic setting to evaluate attention, memory, and personal opinion. Generalization of attention improvements on everyday tasks was found to be limited. Participants did report enjoying the Lumosity programs more than APT-3. This study, although not statistically significant, provides occupational therapists with an alternative and enjoyable intervention approach, along with the opportunity to research further evidence. The use of computer-based programs as intervention alternatives is lacking in statistically significant evidence; however, brain-training programs are unlikely to have any negative ramifications and may provide enjoyable treatment interventions. These computer-assisted programs may serve as beneficial treatment alternatives for clients who have interest in technology training or learn better from visual programming. RESEARCH OBJECTIVE(S) List study objectives. 1
To assess whether the improved performance of individuals with TBI on computer-based treatment programs generalized to improvements on untrained tasks DESIGN TYPE AND LEVEL OF EVIDENCE: Level III, single subject SAMPLE SELECTION How were subjects recruited and selected to participate? Please describe. Adult men with severe TBI served as study participants. The study did not state the recruitment and participant selection process. Inclusion Criteria Participants must have sustained severe TBI, as indicated by a period of coma exceeding 1 day or a period of posttraumatic amnesia exceeding 1 week. They must have been a minimum of 3 years postinjury. Participants needed to speak English as a primary language as well as have normal or corrected-to-normal hearing and vision and pass a motor and mathematical screening. The participant with a history of TBI had a specific attention deficit score on standardized tests at baseline. Exclusion Criteria Participants were excluded if they had a history of learning disabilities or neurological damage other than that associated with TBI. SAMPLE CHARACTERISTICS N= (Number of participants taking part in the study) 4 #/ (%) Male 4/(100%) #/ (%) Female 0/(0%) Ethnicity N/A Disease/disability diagnosis Severe TBI INTERVENTION(S) AND CONTROL GROUPS Add groups if necessary Group 1: APT-3 group Brief description of the intervention APT-3 is a computer program that requires individuals to complete tasks based on sustained attention, selective attention, working attention, suppression, and alternating attention. During each treatment session, participants were given the opportunity to work through everyday tasks on the computer, as provided by APT-3. Scores were then calculated by the computer and recorded by the research assistants. 2
How many participants in the group? Where did the intervention take place? Who Delivered? How often? For how long? Group 2: Lumosity group Brief description of the intervention How many participants in the group? Where did the intervention take place? Who Delivered? How often? For how long? 2 at each phase, 4 overall NR Four trained research assistants instructed participants to use the APT-3 program 20 treatment sessions lasting 30 min for 5 days a week 1 month Lumosity is a computer program that aims to improve cognitive processing speed, flexibility, attention, memory, and problemsolving skills through the delivery of various brain games. Participants were given the opportunity to use five different attention-oriented brain games on the Lumosity program that were selected by the researchers. Scores were calculated by the program on the basis of participant performance and recorded by the researchers. 2 at each phase, 4 overall NR Four trained research assistants instructed participants to use the Lumosity computer program. 20 treatment sessions lasting 30 min for 5 days a week 1 month Intervention Biases: Check yes, no, or NR and explain, if needed. Contamination: X Co-intervention: X Timing: Participants were 3 years postinjury, so it was unlikely for 3
X spontaneous recovery to take place. Site: X Use of different therapists to provide intervention: X The researchers trained four research assistants to set up and administer the computer-based treatment programs. The research assistants also conducted the data collection and analysis. MEASURES AND OUTCOMES Complete for each measure relevant to occupational therapy: Measure 1: Name/type of Test of Everyday Attention (TEA) measure used: What outcome was measured? reliable? valid? When is the measure used? The TEA uses situational scenarios to measure visual selective attention and speed, attention switching, sustained attention, and auditory verbal working memory. X X Pretest, Posttest 1, and Posttest 2 Measure 2: Name/type of measure used: What outcome was measured? reliable? valid? When is the measure used? Researcher-generated repeatable probe Attention targets of the APT-3 and Lumosity programs (sustained attention, selective attention, and divided attention). Tasks within the probe measures were adapted from the Neurological Assessment Battery. X X One during pretesting, three during Intervention Phase 1, one during Posttesting 1, three during Intervention Phase 2, and one during Posttesting 2 4
Measure 3: Name/type of measure used: What outcome was measured? Perceptual rating form Participants perception of enjoyment and willingness to continue with the two intervention computer programs, independently. The following questions were rated by participants, who pointed to a number between 1 and 5 on a Likert scale: How much are you enjoying these activities? Would you complete these activities on your own? X reliable? X valid? When is the After each intervention phase measure used? Measurement Biases Were the evaluators blind to treatment status? Check yes, no, or NR, and if no, explain. X The researchers and research assistants were unable to be blinded to treatment status in this circumstance because the scores of each treatment computer program indicated which program had been used. To assess the scores, the evaluators had to be aware of which treatment the individual received. The evaluators also had to receive specific training about set-up and data collection to administer the computer programs; therefore, they were unable to be blinded to treatment status. Recall or memory bias. Check yes, no, or NR, and if yes, explain. X The perceptual rating form outcome measure was based on selfreport, which allowed for possible faulty recall or memory from the participants. RESULTS List key findings based on study objectives Include statistical significance where appropriate (p<0.05) Include effect size if reported Researcher-generated repeatable probe results Participant OE: No significant results, but indicated a trend toward improvement in divided attention Participant NG: No significant results Participant KX: Significant results found for Task 2 (counting the number of Xs appearing in rows; p =.0479) Participant KS: No significant results Generalization of improved attention was evident with one participant (KX) during the 5
probe tasks throughout the intervention programs. TEA Results Participant OE seemed to generalize attending behavior through both intervention programs and multiple TEA subtests. The other three participants displayed mixed results. No statistically significant results were reported. Perceptual Rating Form Scores reflected strong enjoyment and interest in continuing to use APT-3 from 2 of the 4 participants. All 4 participants indicated enjoyment of the Lumosity games, but only 2 participants stated that they would continue to use the games independently. Overall, participants were skeptical about the functional applicability of the two computer program activities. Results indicate limited generalization for the attention improvement on everyday functioning tasks among the participants. Was this study adequately powered (large enough to show a difference)? Check yes, no, or NR, and if no, explain. X The sample size for this study only consisted of 4 participants, which makes it difficult to get an accurate, statistically significant difference that could be generalized to a larger population. Were appropriate analytic methods used? Check yes, no, or NR, and if no, explain. X Were statistics appropriately reported (in written or table format)? Check yes or no, and if no, explain. X Was the percent/number of subjects/participants who dropped out of the study reported? X No dropouts were reported. Limitations: What are the overall study limitations? 6
The wide range of time postinjury across the participants made generalization of results difficult. Because of repetition of probe questions, a practice effect might have affected the participants answers. Day-to-day variability and motivation of TBI participants might have varied and affected performance. Task and task sequence selection from the two programs could be altered to suit different participants. CONCLUSIONS State the authors conclusions related to the research objectives. The use of technology programs such as Lumosity and APT-3 as an intervention to remediate attention deficits among individuals with TBI requires further research for support of efficacy. However, brain-training programs such as these are unlikely to have negative cognitive outcomes and may be an enjoyable treatment alternative. Participants reported enjoyment of the computer programs as a treatment alternative but were not likely to continue using them outside of therapy sessions. Lumosity was found to have greater appeal to participants than the APT-3 computer program. The researchers suggested that the greater appeal of the Lumosity program might have been due to participants believing that APT-3 consisted of mental games and that Lumosity constituted the actual therapeutic intervention. Given the lack of generalization of attention improvement on everyday functional tasks, the study results indicate skepticism toward manufacturers of such computer-based brain programs, who insist that their products are beneficial for all people. This work is based on the evidence-based literature review completed in November 2015 by Haley Bassett, OTS, and Kelly Erickson, PhD, OTR/L, faculty advisor, College of Saint Scholastica. CAP Worksheet adapted from Critical Review Form Quantitative Studies. Copyright 1998 by M. Law, D. Stewart, N. Pollack, L. Letts, J. Bosch, and M. Westmorland, McMaster University. Used with permission. 7