EPC Review of Cognitive Training Findings: Reactions, Current Evidence, Future Directions Sherry L. Willis, PhD University of Washington Preventing Dementia & Cognitive Impairment Workshop National Academies of Science, Engineering, Medicine October 25, 2016 Washington, DC
Outline of Talk To summarize key points from EPC Review on Cognitive Training To review briefly major findings from ACTIVE trial To discuss key concerns in EPC Review on Cognitive Training, with focus on ACTIVE Limitations in publications reviewed, with focus on ACTIVE Concerns related to Transfer ( diffusion, spillover ) Concerns related to Attrition & implications for ratings of Strength of Evidence Concerns on limitations in findings on slowing or reducing dementia incidence Summary & Future Directions
EPC Review: Results, Discussion, Conclusions Of the 13 classes of interventions we examined we found no high-strength evidence for the effectiveness of interventions to delay or prevent age-related cognitive decline Cognitive Training: Moderate-strength evidence indicates that cognitive training produces sustained benefit over 2 year for cognitive performance that was target of training. little evidence of diffusion of benefits in other cognitive areas e.g., transfer Strength of evidence..low in longer follow-up..high attrition
Major Findings from ACTIVE Trial
Does intervening on basic abilities transfer to real-world tasks? Baseline Characteristics: Mean age: 73.6 Range: 65-94 Gender: 75.9% female Race: 26% African American Education: HS diploma 88.6% Marital Status: 35.6% married Cognitive Status: MMSE 27.3 Multi-site Trial: N = 2802 Source: Jobe et al., 2001
Design of ACTIVE Trial pre-specified in RFA-AG-96-001 INTERVETIONS focus on cognitive abilities PRIMARY OUTCOME: Everyday independence; functional competence
% Participants above Baseline Memory Reason Speed Control Memory 35.9% 31% Reason 73%** 61% Speed 70%** 48% 10-yr Post Training Memory Reason Speed Control After initial Training Memory 26%* 15% Reason 74%** 39% Speed 87%** 31%
Effect Sizes at 5 Years (JAMA 2006) 0.6 0.5 Standardized Training Effect Size (Control Group as Reference) 0.4 0.3 0.2 0.1 Memory composite Reasoning composite Speed composite 0 Memory trained Reasoning trained Speed trained -0.1 Training Group
Primary Outcome as Mandated by RFA Everyday Functioning (IADL)
10-Year Trajectory of Self-Reported IADL Difficulty, by Training Group (JAGS 2014)
Secondary Outcomes related to Everyday Functioning: Findings at 5-,10-yr Follow-up Mobility and Driving Reduction in Auto Crashes: Speed & Reason Training Internal Control Increase in Control Beliefs: Speed & Reason Training Depression Less depressive symptoms in Speed training Subjects in Memory & Reasoning Training with depressive symptoms profit from training
Reduction in Auto Crashes: Comparison of Speed & Reasoning Training & Control Group: 5- and 10-yr Findings Reduced Crashes for participants in Speed & Reasoning Training at high risk for crash For training groups, % high risk participants who crashed was reduced to level of low risk participants
Booster Training: Increased Reasoning & Speed Training Effects S Reasoning Training Speed Training
Key Concerns with EPC Review on Cognitive Training, Limitations in publications reviewed, with focus on ACTIVE Concerns related to Transfer ( diffusion, spillover ) Concerns related to Attrition & implications for ratings of Strength of Evidence Concerns on limitations in findings on slowing or reducing dementia incidence
Limitation in Number and Scope of Publications cited in EPC Review (1) Only 38 cognitive training studies met EPC inclusion criteria (e.g., attrition, 6 mo follow up) Only 11 training studies rated as medium or low risk of bias EPC review found NO high-strength evidence/studies 10 of 11 training studies rated with medium or low risk bias had less than 2 yr follow-up, except ACTIVE (2 yr). Strength of evidence is low in longer follow-up high attrition
Limitation in Number and Scope of Publications cited in EPC Review (2) Only 7 publications were included in EPC Review of ACTIVE out of approximately 40 training publications Many key publications reporting Primary and Secondary outcome effects were not included Driving: Ball et al., 2010; Edwards et al., 2009 Depression: Gallo et al., 2003; Wolinsky et al., 2009 Generalizability of ACTIVE findings: Prindle & McArdle, 2013 MCI: Cook et al., 2013; Unverzagt et al., 2009 6 of 7 publications reviewed were rated as high risk of bias due to attrition.
Key Concerns with EPC Review: TRANSFER (1) The ACTIVE trial showed most clearly that cognitive training could improve performance on domains trained.. but there was little spillover to other domains. There may be an IADL effects at 10 yrs, but high attrition. EPC reviewers appear to have taken a traditional interpretation of transfer (spillover) E.g., Does training on memory show spillover (transfer) to reasoning ability. Implicit assumption that transfer (spillover effects) should occur immediately after training at immediate posttest. Current views of transfer are broader and extend beyond this traditional view.
Key Concerns with EPC Review: TRANSFER (2) The ACTIVE RFA mandated a different view of transfer. The PRIMARY outcome was everyday functioning, not performance on abilities trained. Key Question: Does training on cognitive abilities transfer to maintenance or enhancement of everyday functioning? Everyday functioning is a criteria in both MCI and dementia diagnoses and thus is of importance in study of training effects on dementia incidence. Our ACTIVE findings on transfer to everyday functioning suggest that transfer does not occur immediately after training. Analogous to medical treatments: rapid change in risk factors; delay in outcomes such as stroke
Key Concerns with EPC Review: TRANSFER (3) In ACTIVE, everyday functioning was assessed in multiple ways: Self Report of IADL difficulty significant differences in all training groups compared to control at 10-yr follow-up Secondary outcomes related to everyday functioning: State motor vehicle records of auto crash Speed and Reasoning training showed reduced crash at 5- and 10-yr follow-up Locus of Control Beliefs Speed and Reasoning training showed increased control beliefs Depression Speed training showed lowered depressive symptoms compared to control. Reason and Memory training showed those with depressive symptoms profited from training.
Key Concerns with EPC Review: Attrition (1) The 5- and 10 year outcomes had a high risk of bias due to attrition. Key findings regarding PRIMARY OUTCOME (everyday functioning) were found at 5- and 10-yr follow up and thus were given little attention in review. Incorrect statement: much of the sample loss was unexplained A consort table was included in 10-year report (Rebok et al, 2014) providing a detailed accounting of sources of attrition (next slide)
Pattern of attrition was similar for each training group compared to control at each follow-up occasion. No group differences. Total Attrition: 1582/2832 (Approximate 5% attrition/yr) Deaths: 31% Subject withdraw: 35% Lost to follow-up: 4.1% Site decision: 17% Family refusal: 3.3% Much of attrition is involuntary (death, illness/frailty) Minority sample (26%) contribute to higher attrition
Key Concerns with EPC Review: Attrition (3) The rate of attrition in ACTIVE given a communitydwelling sample (baseline mean 73 yrs) is not surprisingly high. Over 10 years, annual attrition of 5% Attrition in longitudinal studies of aging was discussed in Rebok et al, 2014. Attrition in ACTIVE was comparable to other longitudinal studies. ACTIVE dealt with attrition in statistically appropriate ways (e.g. linear mixed methods; multiple imputation, propensity weighting; Hedeker missing data approach; sensitivity analyses). Statistical approaches indicate attrition tends to affect LEVEL, not RATE of CHANGE
Key Concerns with EPC Review: Study of Dementia Incidence The ACTIVE trial was not designed to study the incidence of dementia. The MMSE was the only clinically valid measure of dementia included. At 5- and 10-yr follow, there have been reports on group differences in incidence of dementia based primarily on a psychometric score approach to impairment (see following slides). The 10-year report has not been published and is thus preliminary. We report on some promising findings regarding the potential for cognitively impaired to profit from training.
Memory Impaired at Baseline Profited from Reasoning and Speed Training Memory impaired at baseline did not benefit from memory training. Memory impaired at baseline did benefit, to same degree as cognitively normal, on Reasoning and Speed training.
Dementia Findings: 5-year Follow-up
Definitions of Dementia 1. Memory -1.5 SD below mean; and Reasoning, Speed, or Vocabulary -1.5 SD below mean; and functional impairment (MDS IADL <=10%tile); or, 2. First visit in which MMSE < 22 and all subsequent visits are MMSE < 22 or are missing; or, 3. Self- or proxy-report of diagnosis of dementia or AD during the follow-up; or, 4. Self- or proxy-report of institutionalization during the follow-up; or, 5. Deactivation: family refuses access to subject
Incident dementia by Tx groups Memory Reasoning Speed All Tx grps Control Total N 700 693 700 2093 693 Event 46 49 44 139 50 Event rate 0.066 0.071 0.063 0.066 0.072 Incidence rate/ 1000 person yrs (95% CI) 17.1 (12.2,22.1) 18.9 (13.6,24.2) 16.1 (11.4,20.9) 17.4 (14.5,20.2) 19.2 (13.9,24.6)
Time to dementia by training: 5 yr
Dementia Findings: 10-yr Follow-up N=2785 participants in analyses, n=296 met dementia criteria (10.6%)
Speed Training Delays Dementia Onset 33% risk reduction HR=0.67, 95%CI 0.49-0.91, p=.012
Effects of Speed Training are Dose Dependent 48% risk reduction HR=0.52, 95%CI 0.33-0.82, p=.005
Conclusions on Dementia Findings Dementia incidence and risk factors comparable to large epi studies supports our dementia definition 5- and 10-Yr Findings: 5 Yr: Brief exposure to cognitive training in late life did not reduce the likelihood of developing dementia. Hazard ratio went in the protective direction but was nonsignificant (Unverzagt et al. 2012) 10 Yr: Preliminary, unpublished Findings. Speed training delayed dementia onset; and delay of dementia was dose dependent. (Edwards et al. IAAG presentation, 2016)
Summary (1) Numerous studies indicate immediate benefits of cognitive training Long term follow-up studies are critical to examine whether cognitive training in normal adults can prevent or slow incidence of dementia Attrition is inevitable in long-term studies of cognitive interventions with older adults Rather than dismiss or reduce ratings of long-term studies due to attrition, the more appropriate approach should focus on understanding nature of attrition effects and appropriate strategies for correcting for attrition effects. A broader approach to conceptualization and assessment of transfer effects is needed, including transfer to everyday function, as well as across cognitive abilities, and the possibility of lagged transfer effects.
Summary (2) Cognitive interventions need to be embedded in the everyday lives of older adults, not a separate, short-term activity. Given that the cognitive decline is related to multiple risk factors, multimodal interventions need further study. The recent inclusion of brain scans in assessment of cognitive training effects needs further study. Brain plasticity, a network approach to brain functioning, and brain stimulation needs further exploration
Thank you
Acknowledgements We acknowledge the funding for the ACTIVE trial from the National Institutes of Aging and Nursing Research Comments and contributions of ACTIVE PIs to this presentation are gratefully acknowledged (Rebok, Marsiske, Ball) We thank the 2802 ACTIVE participants and the staff at each of the 6 ACTIVE sites for their contribution.