RARE DISEASE WORKSHOP SERIES Improving the Clinical Development Process Disclaimer: Presentation slides from the Rare Disease Workshop Series are posted by the Kakkis EveryLife Foundation, for educational purposes only. They are for use by drug development professionals and statisticians, and are not to be used to guide the prescribing or use of any of the drugs mentioned in the slides. To obtain information on a particular drug, refer to the drug labeling. Do not reproduce or distribute the slides (full set or any portion of) without the permission of the author.
KAKKIS Foundation Clinical Evaluation of Rare Disease Treatments June 14-15, 2011 Washington DC James Witter MD, PhD Chief Science Officer NIH/NIAMS www.nihpromis.org Dynamic Tools to Measure Health Outcomes from the Patient Perspective
The Tower of Babel (Brueghel, 1563)
Vision-Mission: Vision The Patient-Reported Outcomes Measurement Information System (PROMIS), funded by the National Institutes of Health, aims to provide clinicians and researchers access to efficient, precise, valid, Click and responsive to edit adult- Master and title child-reported style measures of health. Mission PROMIS uses measurement science to create an efficient state-of-the-art assessment system for selfreported health.
Overall Framework: Assessments Dimension Objectiveness Clinical Measure (Some Non-objective element Involved) Biomarker Who measures Patient Clinician Observer Instrument Relationship to Tangible Clinical Benefit Direct Indirect Direct Indirect Direct Indirect Indirect How obtained Naturalistic Artificial Procedure VAS for pain intensity; NONE Possible PANSS for schizophrenia symptoms NONE Possible Observation of seizures for epilepsy outcomes NONE Possible HbA1c for cardiovascular outcomes; endocrine stimulation tests for endocrine organ function) 2 nd Annual Critical Path Institute-PRO Consortium: FDA Workshop March 15, 2011 Silver Spring, MD
NIH Assessment Tools NIH Toolbox www.nihtoolbox.org/default.aspx Assessment of Neurological and Behavioral Function Neuro-QOL www.neuroqol.org/default.aspx Quality of Life in Neurological Disorders The EXAMINER http://examiner.ucsf.edu Neurobehavioral Evaluation and Research Phen-X www.phenx.org Consensus Measures for Phenotypes and Exposures PROMIS www.nihpromis.org Patient-Reported Outcomes Measurement Information System
Patient-Reported Outcomes (PROs) FDA and NIH recognize PROs as important outcomes to answer clinical questions translating the efficacy-effectiveness and safety of therapeutics In development (clinical research) In the real world (clinical care) Click to edit Master title style Patient burden is important aspect of PRO assessment in both these settings
PROMIS: Present Click to edit Master title style
FDA/C-PATH PRO Process FDA/C-Path meeting: Dec. 4, 2008
Domain development/qualification-validation lifecycle -Maturity Model- Fully test domain Industry sponsored Secondary endpoints Primary endpoints National surveys CDC NIH phase II-III studies establish MID Clinical care Click to edit Master title style Psychometrically focused testing Clinically focused testing Domain: Selection Content Expert Review Healthy People-Patient Centered: Cross Sectional Cognitive testing Focus groups Fully develop domain Items administered to representative sample and calibrated for widespread use: Web-based (large numbers) Patient groups
PROMIS Domain Framework
Items from Instrument A Items from Instrument B Items from Instrument C New Items Item Pool Probability of Response 1.0 0.8 0.6 0.4 0.2 0.0 Content Expert Review Category Response Short Form Instruments -3-2 -1 0 1 2 3 Theta Focus Groups Questionnaire administered to large representative sample Psychometric Testing Cognitive Testing Item Bank v. 1 Domain (IRT-calibrated items, reviewed for reliability-validity-sensitivity) 2.5 Information 2.0 1.5 1.0 0.5 0.0 DIF (2 0 )Analysis -3-2 -1 0 1 2 3 Theta CAT Item Information
PROMIS Wave 1 Samples Site (n=329) Demo. short form Global health General population (n=329) UNC=236 Full Bank Testing (N=7,005) Forms A-G Legacy Polimetrix (n=6,676) Stanford=93 Demo. short form Global health General population (n=6,676) Forms A-G Legacy Co-morbidity Site (n=1,203) General population (n=400) UNC=304 Stanford=96 Block Testing (N=14,128) Demo. extended form Global health Forms H-W Co-morbidity Polimetrix (n=12,925) Demo. extended form Global health Forms H-W Co-morbidity Clinical sample (n=803) General population (n=5,845) Clinical sample (n=7,080) Pittsburgh=252 Stanford=150 Duke=401
The PROMIS Validation Process Qualitative Research and Item Writing Item Testing General Population Clinical Samples Click to edit Master title style Bank Analysis Interpretation Refining
PROMIS combines: Item Response Theory (IRT) and Computer Adaptive Testing (CAT) Together, IRT and CAT provide precise measurement Click to edit of Master individual title style symptoms
Item Response Theory In psychometrics- body of theory that: Describes application of mathematical models to how people respond to items» At person and item level Serves as basis for measuring Click to edit Master title style» Abilities» Attitudes Based on probability of endorsing a particular item category is function of latent trait or ability Ability to improve reliability of instrument
Definitions Items are questions that have: Context, stems, responses Parameters (Difficulty, Discrimination) Important for analyses, calibration Banks Click to edit Master title style Collection of items that have been calibrated on a common metric difficulty and discrimination have been estimated Define common concept-domain Domains Allow computerized adaptive testing (CAT) Define latent traits/abilities
A domain is the specific feeling, function, or perception you want to measure. Examples Fatigue Pain Anxiety
A domain is the specific feeling, function, or perception you want to measure. Examples Physical function Erectile function
A domain is the specific feeling, function, or perception you want to measure. Examples Satisfaction with Social Participation General Health Perceptions
An item bank is a large collection of items measuring a single domain. Any and all items can be used to provide a score for that domain.
IRT models Latent Traits: People and Items are Represented on the Same Scale Are you able to get out of bed. Item Difficulty Are you able to run 5 miles Low Low High High Persons Levels of Physical Function 23
PROMIS Current Physical Health Banks
PROMIS Current Mental Health Banks
PROMIS Current Social Health Banks
PROMIS Adult Banks: v1.0 Items in Domains Items in Bank Short Form Emotional Distress Anger 29 8 Emotional Distress Anxiety 29 7 Emotional Distress Depression 28 8 Fatigue 95 7 Click to edit Master title style Pain Behavior 39 7 Pain interference 41 6 Physical Function 125 10 Satisfaction with Discretionary Social Activities 12 7 Satisfaction with Social Roles 14 7 Sleep Disturbance 27 8 Wake Disturbance (sleep related impairment) 16 8 Global Health 10
PROMIS II Expanded Focus on Children 4 of the 12 research sites are devoted to child-reported outcome measurement
Child-Adult Linkage Studies Render child and adult editions comparable (i.e., same scale) Result: life course outcome assessment
Pediatric PROMIS Item Banks PROMIS I 1. Physical functioning/mobility 2. Physical functioning/upper extremity 3. Pain interference 4. Fatigue 5. Depressive symptoms 6. Anxiety 7. Anger 8. Peer relationships 9. Asthma impact PROMIS II 1. Pain behavior 2. Pain quality 3. Physical activity/mobility 4. Physical activity/sedentary 5. Experiences of stress 6. Physical distress 7. Life satisfaction 8. Positive affect 9. Meaning and purpose 10. Family belonging 11. Family involvement
PROMIS Profile Short Forms (29-43-57 items) Mental Anxiety 29 Depression 28 4 6 8 Fatigue 95 Physical Pain Impact 41 Sleep Disturbance 27 Physical Function 86 Social Satisfaction with Roles 14 32 32
Value of PROMIS : Comparability Validity and Reliability Flexibility Inclusiveness Click to edit Master title style
Comparability Chemotherapy trial Osteoarthritis trial Heart failure trial
Reliability and Validity
Physical Functioning: Improved Precision vs. Legacy SF-36 10 items P re reliability = 0.90 c i s i o n Standard error HAQ 20 items 10 item CAT SE = 0.32 reliability = 0.95 SE = 0.22 SF-36 full bank Disabled rheumatoid arthritis patients normed theta values 10 item PROMIS CAT US General Population mean High physical functioning
Flexibility
Linking PROMIS and EHRs SNOMED and LOINC codes
Inclusiveness
Computerized Adaptive Testing (CAT) Integrates IRT with computers to administer a PRO instrument selects questions on the basis of a patient s response to previously administered questions Click to edit Master title style measurement is adapted to individual skips uninformative items to minimize response burden allows determination of person s standing on a domain without a loss in measurement precision.
CAT: Estimate gets more precise with every question Q1: Q2: SEM after each question Q3: Q4: Q5: -3.0-2.0-1.0 0.0 1.0 2.0 42
What is Assessment Center? Web-based research management tool Primary mechanism for distribution of PROMIS instruments Display of item- and instrument-level statistics Allows Click creation to edit Master of study-specific title style URLs for data collection Enabled to administer CATs Administration of PROMIS and other instruments FREE!!
Select CATs Click to edit Master title style
CAT Report Graph Click to edit Master title style
Effect Size: PROMIS Pain Behavior 1.6 vs. Roland-Morris 1.4 1.2 Pain Behavior (4 (4-4) items) Roland-Morris (24 items) effect size 1 0.8 0.6 0.4 0.2 0 1 2 3 time point
Mean Difference: PROMIS Pain Interference vs. BPI Interference (pain) Mean difference from baseline 8 7 6 5 4 3 2 1 0 1 2 3 time point PROMIS Pain Interference (4 (4-12) items) BPI Interference (7 items)
PROMIS Integration Options Study-specific URL in Assessment Center PROMIS and other self-reported data: email-able link to respondents Online consent and assent Standalone study or run as sub-study Downloadable modules for local installation Redcap: includes PROMIS short-forms Short-forms can be added to SurveyMonkey, epro, etc. PROsetta Stone linking measures to common PROMIS metric Electronic Health Records & Personal Health Records LOINC and SNOMED codes have been assigned to PROMIS measures
Advancing Patient-Centered Outcomes PROMIS: A Common Source of PROs Clinical Practice Click to edit Master title style Surveys (CDC) Clinical research Clinic Hospital NIH Industry FDA
Patient-Reported Outcomes: PROs PROs - Biomarkers FDA guidance: Qualification of Drug Development Tools (Oct. 2010) http://www.fda.gov/downloads/drugs/guidancecomplianceregulatoryinformation/guida nces/ucm230597.pdf PROs to dissect mechanisms Knee OA Pain: AC&R July 2010 Click to edit Master title style neuropathic pain descriptors Burning, tingling, numbness, pins & needles PROs to describe disease SLE: Self-Assessed Lupus Damage Instrument AC&R: April 2010 RA: Patient-based Disease Activity Score AC&R: Feb 2008
Enhancing Pediatric Pain Pediatric Rheumatology and Rehabilitation Research Dr. Esi Morgan DeWitt-CCHMC Demonstrate validity and responsiveness in conditions that impair physical function and HRQOL Chronic pain conditions (140 children: ages 8-18) Juvenile idiopathic arthritis (120 children) Click Cerebral to palsy edit (60 children, Master 120 title parents) style Study content validity (interviews) Longitudinal study Three assessments: includes pain interference CAT and SF Develop & Test Pain Behavior and Quality Item Banks 450 children chronic: MS pain, JIA, sickle cell disease Facilitate Linking studies (Lifecycle) with adult banks
Patient-Reported Outcomes: PROs PROs measure symptoms which reflect biopsychosocial mechanisms of disease Items/questions capture symptoms Items capture mechanisms of disease PROs add to: Objective-biomarkers of disease Physician opinion regarding disease Engage patient in their care and treatment facilitate personalized care
DOMAINS vs. DISEASES Diseases (common & rare) are combinations of different mechanisms which impact domains Capturing multiple domains may be optimal way to assess diseases Core-common PRO domains are universally applicable across diseases, ages and ethnicities Core PRO domains can link common and rare diseases
A model for PROs in RA Pain Physical Function Morning stiffness Fatigue Sleep Disturbance Depression/anxiety Sexuality Satisfaction with social roles Capacity to work Family belonging Self efficacy
Rare Diseases Domain Validation-Qualification Numbers/Types of Trials? Clinically focused testing 0 Psychometrically focused testing
PROMIS Options -public private partnerships- Clinical, Translational, & CE Research Registries Population Surveillance Clinical Practice