Application of Item Response Theory to ADAS-cog Scores Modeling in Alzheimer s Disease

Transcription

1 Application of Item Response Theory to ADAS-cog Scores Modeling in Alzheimer s Disease Sebastian Ueckert 1, Elodie L. Plan 2, Kaori Ito 3, Mats O. Karlsson 1, Brian W. Corrigan 3, Andrew C. Hooker 1 1. Dept Pharmaceutical Biosciences, Uppsala University, Uppsala, Sweden 2. Metrum Institute and Metrum Research Group, Tariffville, CT, USA 3. Global Clinical Pharmacology, Pfizer Inc, Groton, CT, USA

2 Make a fist Draw a circle 7 June 2012 Name current month 2

3 ADAS-cog Assessment Cognitive subscale of Alzheimer s Disease Assessment Scale Cognitive assessment including broad range of sub-tests e.g., 7 June 2012 Make a fist Draw a circle Name current month LAKE CLOCK FOREST ANIMAL Name finger Name object Remember those words Foldput in envelopaddress stamp Ability to speak Ability to understand Rosen et al

4 ADAS-cog Assessment Cognitive subscale of Alzheimer s Disease Assessment Scale Cognitive assessment including broad range of sub-tests e.g., 7 June 2012 Make a fist Draw a circle Name current month Name finger Foldput in envelopaddress stamp Name object Ability to speak LAKE CLOCK FOREST ANIMAL Remember those words Ability to understand Primary outcome Range 0-70 Σ AD AD severity ADAS-cog Score Σ AD Rosen et al

5 ADAS-cog Assessment Cognitive subscale of Alzheimer s Disease Assessment Scale Cognitive assessment including broad range of sub-tests e.g., 7 June 2012 Make a fist Draw a circle Name current month Name finger Foldput in envelopaddress stamp Name object Ability to speak LAKE CLOCK FOREST ANIMAL Remember those words Ability to understand Σ AD Primary outcome Range 0-70 Σ AD AD severity ADAS-cog Score Σ AD t Rosen et al

6 Score Properties Tasks have varying difficulty e.g., construction or drawing task Non-linear scale Imputation necessary if subject refuses task or physician omits it Bias Object to Draw 0% 80% Fraction of Subjects Failing Task ADAS-cog in study A ADAS-cog in study B Different test versions (ADAS-cog11, ADAS-cogmod, ADAS-cog13, ADAS-cogMCI ) Hard to pool data 6

7 Cognitive Disability 7 June 2012 COGNITIVE DISABILITY LAKE CLOCK FOREST ANIMAL 7

8 Cognitive Disability 7 June 2012 COGNITIVE DISABILITY LAKE CLOCK FOREST ANIMAL 8

9 Cognitive Disability 7 June COGNITIVE DISABILITY LAKE CLOCK FOREST ANIMAL 9

10 Item Response Theory Georg Rasch Assumption: Paul Lazarsfeld Statistical framework to score tests or surveys consisting of several dichotomous (or polytomous) responses Developed around 1950 by Rasch and Lazarsfeld Individual responses for each item depend on a hidden variable (trait or ability) Describes the probability of a certain test outcome as the function of a person s ability Directly estimates the most likely ability, instead of summary scores Used in psychometrics for the development of high-stakes tests 10

11 Project Outline Assumption: Outcome of each test in the ADAS-cog assessment depends on unobserved variable cognitive disability Approach: 1. Develop IRT model for ADAS-cog assessment using data from clinical trial databases 2. Apply ADAS-cog IRT model to longitudinal clinical trial data 3. Investigate benefits of IRT model 11

12 Baseline Model 12

13 Data Observational study with normal, mild cognitively impaired (MCI) and mild AD subjects Baseline ADAS-cog data 819 subjects Database with placebo arm data from clinical trials First visit ADAS-cog data from 6* CAMD studies (Phase II & III) 1832 subjects > data entries in total adni-info.org *Studies with item level data as of November

14 Model 7 June 2012 COGNITIVE DISABILITY LAKE CLOCK FOREST ANIMAL 14

15 Model 7 June 2012 LAKE CLOCK FOREST ANIMAL 15

16 P(not Drawing Cube Correctly) P(not Drawing Cube Correctly) P(not Drawing Cube Correctly) Model Test specific Binary Subject specific 80% 60% 40% 20% 80% 60% 40% 20% Parameter Value Cognitive Disability Parameter Value Cognitive Disability 80% 60% 40% 20% Parameter Value Cognitive Disability

17 Model Binary (x 39) Binomial (x 3) Ordered Categorical (x 5) Generalized Poisson (x 1) 167 Parameters 166 fixed effect 1 random effect 17

18 Ordered Categorical P(Y ji =k) Binomial P(Failed) Binary P(Failed) Results Commands Construction Ideational Praxis Orientation Naming Word Recall Delayed Word Recall Word Recognition Number Cancellation Comprehension Concentration Remembering Word Finding Spoken Language

19 Longitudinal Model 19

20 Data Placebo arm of Phase III study with mild to moderate AD patients 18 month with 6 ADAS-cog assessments 322 subjects observations in total 20

21 Model Binary (x 39) Binomial (x 3) Ordered Categorical (x 5) Generalized Poisson (x 1) 21

22 Model Binary (x 39) Binomial (x 3) Ordered Categorical (x 5) Generalized Poisson 5 Parameters 2 fixed effect 2 random effect 1 covariance (x 1) 22

23 Results All parameters estimated precisely (assessed through - Hessian of log-likelihood) Corresponding to baseline ADAS-cog value of 22.2 points and yearly increase of 3.5 points Parameter Value RSE Baseline θ % Slope θ % IIV Baseline ω % IIV Slope ω % Cor(η 1,η 2 ) % 23

24 Fraction ADAS-cog Score Fraction Failed Diagnostics Summary Score Commands Test Make a fist Point to ceiling Put pencil Put watch Tap shoulder Word Recall Test Time [days] Time [days] Y=0 Y=1 Y=2 Y=3 Y=4 Y=5 Y=6 Y=7 Y=8 Y=9 Y= Time [days] Repetition 1 Y = Number not recalled words

25 Benefits of the IRT Approach 25

26 Density Handle the True Nature of the Score Occasion 1 Occasion 2 Occasion Occasion 4 Occasion 5 Occasion 6 Observed Simulated ADAS-cog Score Bounded nature of each subcomponent is taken into account Summary score distribution is more natural 26

27 Power Increased Power Method: 1. Simulation from longitudinal IRT model with disease modifying drug effect of 20 % (n=500) 2. Estimation with full and reduced IRT model 3. Estimation with full and reduced Summary Score model 80% 60% 40% Increased Power when using IRT model Approach Number of Individuals Summary Score IRT 27

28 Power Improved Clinical Trial Simulations ADAS-cog 11 ADAS-cog 13 Approach delivers test & subject specific parameters Simulate different populations & different ADAS-cog assessments 90 % 80 % 70 % 84 % 85 % 86 % 87 % Mild AD MCI *600 Subjects (300/300), 20% drug effect DM 28

29 Integrating Information Across Trials Combination of data across trials easily possible Other cognitive tests like MMSE can be related to same hidden variable MMSE assessments become additional observations ADAS-cog13 ADAS-cog11 * Work in Progress ADAS-cog COGNITIVE DISABILITY * MMSE ADAS-cogmod ADAS-cogmci 29

30 Advanced Optimal Trial Design COGNITIVE DISABILITY Each response function is dependent on D Calculate Fisher Information for each item: Measure of information content in each item 30

31 Information Item Information Commands Comprehension Concentration Construction Delayed Word Recall Ideational Praxis Naming Orientation Remembering Spoken Language Word Finding Word Recall Word Recognition Cognitive Disability

32 Information Information for a MCI Study Commands Comprehension Concentration Construction Delayed Word Recall Ideational Praxis Naming Orientation Remembering Spoken Language Word Finding Word Recall Word Recognition Cognitive Disability

33 Component Ranking for MCI Study Test Information Delayed Word Recall Word Recall Orientation Word Recognition Naming Number Cancellation Construction Word Finding Allows adaptation of the test to a specific population Test can be performed quicker with little change in information content Ideational Praxis Concentration Remembering Comprehension Commands Spoken Language

34 Summary Extension: UPDRS IRT Model Parkinson s Model ACR IRT Model Drug Effect Model Advantages Treat true nature of data (better simulation properties) Increased drug effect detection power More flexible clinical trial simulations Possibility to optimize test design Implicit mechanism for missing sub-scores Rheumatoid Arthritis Model Drug Effect Model 34

35 Acknowledgements Colleagues in Uppsala Pfizer colleagues in Groton 35