The Rest is Noise. The Promise of ecoa in Increasing Signal Detection

Medavante Research White paper The Rest is Noise The Promise of ecoa in Increasing Signal Detection By Christopher Randolph, PhD, ABPP-CN Medical Director of Neuropsychology, Loyola University Medical Center VP of Neurocognition, MedAvante

3 The Rest is Noise: The Promise of ecoa in Increasing Signal Detection Inconclusive clinical trials keep treatments from patients who need them most: a problem that is especially prevalent in central nervous system (CNS) disorders. For all neurological conditions combined, researchers estimate the Phase 3 success rate to be approximately 5. 1 This is due, in part, to imprecise endpoint measurements that add noise and reduce signal detection, decreasing the chances of a successful trial. Alzheimer s disease (AD) provides a particularly striking example of the steep road to trial success: between 2002 and 2012, Phase 3 trials in AD had a failure rate estimated at 99.6%. 2 What s behind these daunting odds, and what can sponsors and contract research organizations (CROs) do to improve them? A recent study that I led with colleagues at MedAvante, a company dedicated to improving signal detection in clinical trials, suggests that the use of paper-based assessments could be an important factor in failed or inconclusive trials.3 Why? Paper increases the possibility of rater and administrative errors in an already difficult process. This effect is often compounded with the challenges posed by subjectively reported, subjectively scored assessments. Trial sponsors and CROs, then, should be aware of the limitations of paper in CNS trials and understand that digital alternatives offer significant benefits. The human and financial stakes are too high not to use all available means to achieve conclusive results. 1 Hay M, Thomas DW, Craighead JL, Economides C, Rosenthal J. Clinical development success rates for investigational drugs. Nat Biotechnol. 2014;1:40 51. doi: 10.1038/nbt.2786. 2 Cummings JL, Morstorf T, Zhong K. Alzheimer s disease drug-development pipeline: few candidates, frequent failures. Alzheimer s Research & Therapy. 2014;6(4):37. PMID: 4095696. The study examined trials on Clinicaltrials.gov. The authors write, In the decade of 2002 through 2012, 244 compounds were assessed in 413 trials for AD. Of the agents advanced to Phase 3 (and excluding those currently in Phase 3), one was advanced to the FDA and approved for marketing (1.8%). Excluding the 14 compounds currently in Phase 3, the overall success rate for approval is 0.4% (99.6% attrition). This is among the lowest for any therapeutic area. 3 Negash S, Böhm P, Steele S, Sorantin P, Randolph C. Investigative Study Platform Minimizes Scoring Discrepancies to Improve Signal Detection. MedAvante Inc.; Loyola University Medical Center. Poster presentation, 14th Annual Athens/Springfield Symposium on Advances in Alzheimer Therapy, March 2016. Risks for imprecision in CNS trials A number of factors contribute to the high level of imprecision in CNS trials. First, traditional paper-based clinical outcome assessments are prone to high rates of error. For example, the total score of the Clinical Dementia Rating scale (CDR) requires calculation, either by the rater using the Sum of Boxes score, or by using an online tool in the case of the CDR Global Score. Then, in an additional step, score data must be transcribed manually into an electronic data capture (EDC) system and later be monitored for source data verification. Attempts to implement central oversight for data quality assurance typically involve scanning a large volume of paper, as well as uploading audio or video files. All these steps create the possibility of more errors being introduced as well as adding to the heavy administrative burden on trial sites, perpetuating significant obstacles in an already complex undertaking.4 Reviewing assessments based on source documents and recordings may reveal scoring discrepancies between raters and reviewers. Discrepancies take time to resolve and compromise data quality by calling ratings into question. A better option? Given the disadvantages of paper administration as well as the clinical challenges of neuropsychiatric assessments, an electronic clinical outcome assessment (ecoa) system that removes paper from the equation presents obvious advantages. That was the thinking that led to the development of the Investigative Study Platform, which reduces administrative burden and enables the real-time detection of calculation errors and discrepant scores. Such advantages point to a greater likelihood of a conclusive trial. ecoa can save time and increase accuracy for raters and study coordinators at the study sites, while also providing ready-to-analyze data to principal investigators, sponsors, and CROs far more quickly than paper collection. 4 Cummings JL, et al. Alzheimer's disease drug-development pipeline: few candidates, frequent failures. Alzheimers Res Ther. 2014 Jul 3;6(4):37.

4 The Rest is Noise: The Promise of ecoa in Increasing Signal Detection For trial raters who administer multiple assessments throughout a day, the tablet-based platform can: eliminate calculation errors (by performing any calculations and auto-scoring) flag missing data capture all data input once and forever ease the process of central oversight by simultaneously audio- or video-recording the administration of assessments, facilitating review without creating extra steps for the site eliminate the EDC transfer step and associated SDV monitoring Equally important, ecoa addresses the problem of subjectivity. It provides raters with real-time clinical guidance developed by expert clinicians right on the screen of a tablet used to administer the assessment. Links to scoring anchors, item descriptions and study-specific guidelines help maintain consistency across the assessment s various domains. Imagine a rater administering the CDR who enters, say, a 0 or 0.5 (no or very mild impairment) in the memory domain after an informant had responded rarely to the question Can the patient recall recent events? A flag can then pop up on screen pointing out that this is a possible inconsistency and ask the rater if this was the intended scoring. Error rates in AD trials drop with the use of ecoa My colleagues at MedAvante and I examined data from our work in major clinical trials to evaluate the effectiveness of an ecoa solution to alleviate the root cause of inconclusive trials.5 We found that using MedAvante s tablet to administer neurocognitive assessments could significantly reduce error rates in AD trials. Our study compared error rates in the administration of clinical outcome assessments in AD trials between the traditional paper-and-pencil method and the tablet. 5 Negash S, et al. Investigative Study Platform Minimizes Scoring Discrepancies to Improve Signal Detection. Poster presentation, Athens/Springfield Symposium, March 2016. We compared paper-administered assessments from a recent clinical trial of mild cognitive impairment (MCI) due to AD with tablet administrations of the same assessments from two other, separate AD MCI trials. All trials were Phase II/III multinational trials. We examined the following four commonly used rating scales: the Alzheimer s Disease Assessment Scale-Cognition (ADAS-Cog) the Alzheimer s Disease Cooperative Study-Daily Life Inventory-Mild Cognitive Impairment (ADCS-ADL-MCI) the Mini Mental State Examination (MMSE) the Clinical Dementia Rating (CDR) A cohort of Central Reviewers examined the first 150 administrations of each assessment for the paper-and-pencil trial and then compared them with the first 150 administrations done with the tablet. This same cohort of reviewers then used audio recordings and worksheets to identify errors in both modes of administration. Comparing the paper-based and administrations, the study quantified the percentage of reviewed assessments with the following kinds of errors: one (different score found by rater and reviewer) two or more discrepancies The results? The study demonstrated that compared to paper-and-pencil administration, substantially reduced scoring- errors for all four AD rating scales. s reduction of errors shows just how effective its guidance is in leading raters to provide accurate scores. Figures 1a-1d show the magnitude of this reduction. The percentage of assessments with one declined by: 26% (CDR) 43% (ADAS-Cog) 78% (ADCS-ADL-MCI) 71% (MMSE)

5 The Rest is Noise: The Promise of ecoa in Increasing Signal Detection Fig 1a: CDR Fig 1b: ADAS-Cog 5 47% 35% 24% 5 44% 25% 27% 2 1 7% more discrepancies 2 1 11% more ANOVA: F(1,298) = 15.0 p<.001 ANOVA: F(1,298) = 8.6 p =.004 Fig. 1c: ADCS-ADL-MCI Fig. 1d: MMSE 5 2 1 41% 9% 16% 1% more discrepancies 5 2 1 28% 7% 8% 3% more discrepancies ANOVA: F(1,298) = 38.4 p<.0001 ANOVA: F(1,298) = 16.9 p<.0001

6 The Rest is Noise: The Promise of ecoa in Increasing Signal Detection At the same time, the percentage of assessments with two or more discrepancies declined by: Fig. 2b: ADAS-Cog 71% (CDR) 59% (ADAS-Cog) 94% (ADCS-ADL-MCI) 57% (MMSE) Such reductions are notable when considering that multi- assessments likely account for a greater amount of noise. 6 An item-by-item look at discrepancies provides further detail (Figures 2a-2d). Worth noting was s effectiveness in reducing discrepancies in the orientation domains of both the CDR and the ADAS-Cog, as well as in items that are particularly hard to score, such as the memory domain in the CDR.7 Here, paper-and-pencil administration resulted in a 19% rate compared to 9% with. Fig. 2a: CDR Orientation 23 % 10 % Memory 19 % 9 % Judgment and Problem Solving 18 % 9 % Community Affairs 13 % 8 % Home and Hobbies 11 % 7 % Personal Care 5 % 2 % Platform 6 Analysis of variance (ANOVA) showed that the rates of for the esource system were statistically significantly lower on all four types of assessments than for paperand-pencil assessment administration (all p values <.001). 7 Tractenberg RE, Schafer K, Morris JC. (2001). Interobserver disagreements on clinical dementia rating assessment: interpretation and implications for training. Alzheimer Dis Assoc Disord. 2001 Jul-Sep;15(3):155-61. PMID: 11522933 Platform Orientation Place 11 % 2 % Orientation Year 11 % 0 % Orientation Day of the Week 11 % 0 % Naming Total Number Correct 9 % 2 % Constructional Praxis Cube 9 % 3 % Delayed Word Recall Score 4 % 2 % Delayed Word Recall Number Recalled 4 % 2 % Spoken Language Score 3 % 0 % Word Recall 1 Total Not Recalled 3 % 1 % Word Recall 3 Total Not Recalled 3 % 5 % Constructional Praxis Rhombus 3 % 3 % Constructional Praxis Overlapping Rectangles 2 % 0 % Comprehension Score 2 % 0 % Word Finding Difficulties Score 1 % 0 % Constructional Praxis Circle 1 % 1 % Commands Score 0 % 0 % Word Recall 2 Total Not Recalled 0 % 3 % Ideational Praxis Score 0 % 0 % Orientation Date 0 % 0 % Orientation Time of Day 0 % 1 % Orientation Season 0 % 1 % Orientation Name 0 % 0 % Orientation Month 0 % 0 % Fig. 2c: ADCS- ADL- MCI Platform Appointments 2 % 0 % At home 1 % 0 % At home (< 1 hour) 1 % 5 % Balance banking 3 % 0 % Balance banking Performance 5 % 0 Cleaning Performance 1 % 0 Discuss (<1 hour) 5 % 0 % Discuss show 0 % 1 % Discuss show (24 hours) 2 % 0 % Family, Friends or Neighbors 1 % 0 % Finding Belongings Performance 3 % 0 Laundry Performance 5 % 0 Make meal/snack Performance 2 % 0 Pastime, Hobby or Game 2 % 0 % Regional, national or International 0 % 0 % Select clothes 1 % 0 % Select clothes Performance 2 % 0 Select or Ask for show 1 % 0 % Selection of items 1 % 0 % Shopping 1 % 0 % Travel Performance 1 % 0 Use of Telephone Performance 12 % 0 Watch television 1 % 0 %

7 The Rest is Noise: The Promise of ecoa in Increasing Signal Detection Fig. 2d: MMSE Orientation to place Building type 7 % 5 % Repetition No ifs, ands, or buts 3 % 3 % Drawing Pentagons 4 % 1 % Orientation to place County 3 % 0 % Writing Sentence 3 % 0 % Orientation to time Date 3 % 0 % Attention 72 1 % 1 % Orientation to place City/town 3 % 0 % Orientation to place Floor 3 % 0 % Recall Penny Recall 1 % 1 % Recall Apple Recall 1 % 1 % Attention 65 1 % 1 % Orientation to time Season 2 % 0 % Attention 86 1 % 1 % Recall Table Recall 1 % 1 % Orientation to time Month 1 % 0 % Naming Watch 0 % 1 % Attention 93 1 % 0 % Orientation to time Day of week 1 % 0 % Orientation to place State 1 % 0 % Comprehension Put on floor 0 % 0 % Registration Apple 0 % 0 % Comprehension Fold in half 0 % 0 % Registration Penny 0 % 0 % Comprehension Right hand 0 % 0 % Registration Table 0 % 0 % Naming Pencil/pen 0 % 0 % Attention 79 0 % 0 % Orientation to time Year 0 % 0 % Reading Close your eyes 0 % 0 % Platform What do the results of this study suggest for raters, investigators, CROs, sponsors, regulatory agencies and, ultimately, patients? Conclusion Our study was the first to examine the actual error reduction outcomes that derive from using a tablet-based ecoa system such as. Directly comparable data sets from multinational clinical trials in MCI due to AD were available, and error rates were examined in a central review performed by the same cohort of calibrated raters. Error reduction was substantial, from approximately 5 to over 8 by scale, and highly statistically significant. This is a compelling demonstration of the clinical utility of the Investigative Study platform and its meaningful advantages over paper-and-pencil administration of assessments in clinical trials. The use of ecoa leads to fewer errors and discrepancies. This reduction in site-based scoring errors is likely due to the tablet s functions of displaying clinical guidance, performing auto-calculations, and offering consistency checks to raters in real-time. These advantages can increase signal detection and boost the chances of a successful trial. An ecoa system also has substantial benefits for sponsors and CROs, reducing administrative burden and costs and speeding trials. Such a system shows strong results with fewer errors and discrepancies, improved data quality, clearer signals, and standardized, accurate studies. The ultimate beneficiaries of clearer signals and reduced administrative burden are of course CNS patients and their families. Their quality of life depends on medical advances advances that improvements in the clinical trial process have vast potential to accelerate. They confirm the perception that traditional paper-and-pencil administration of COAs in AD trials is characterized by high error rates that contribute to error variance, which has the potential to degrade signal detection. Medavante 100 American Metro Boulevard #106 Hamilton, NJ 08619, USA +1 609 528 9400 medavante.com