DEVELOPING PROMs TO HAVE MORE IMPACT. Copyright 2015 by the International Consortium for Health Outcomes Measurement. All rights reserved.

Size: px

Start display at page:

Samson Richardson
5 years ago
Views:

2 Introduction & Session overview Topic Developing PROMs to have more impact Panelists 1. Konrad Pesudovs, PhD - Foundation Chair of Optometry and Vision Science at Flinders University, Adelaide, Australia 2. Matthias Rose, MD, PhD - Head of the Psychosomatic Medicine Centre at Medical Clinic of the Charité, Universitätsmedizin Berlin, Germany 3. James Willig, MD, MSPH - Associate Professor at the Department of Medicine, Division of Infectious Diseases at University of Alabama at Birmingham, Alabama, USA 4. Helen Burstin, MD, MPH, FACP - Chief Scientific Officer of The National Quality Forum, Washington DC, USA Agenda Introduction (5 minutes) Panel presentations (~10-12 minutes each) Discussion, Q&A (35 minutes) Summary and closing (2 minutes) Copyright 2015 by the International Consortium for Health Outcomes Measurement. All rights reserved. 2

4 Psychometrics concerns itself with turning qualitative measures into quantitative data I am in pain Intelligence Instrument 1, 2, 3, 4, Ability to ambulate Copyright 2015 by the International Consortium for Health Outcomes Measurement. All rights reserved. 4

5 Psychometrics concerns itself with turning qualitative measures into quantitative data Construct, domain, trait, patient-reported outcome PROM Score I am in pain Intelligence Instrument 1, 2, 3, 4, Ability to ambulate Copyright 2015 by the International Consortium for Health Outcomes Measurement. All rights reserved. 5

6 Psychometrics concerns itself with turning qualitative measures into quantitative data Construct, domain, trait, patient-reported outcome PROM Score I am in pain Intelligence Instrument 1, 2, 3, 4, Ability to ambulate PRO-PM - PRO-based Performance Measure Copyright 2015 by the International Consortium for Health Outcomes Measurement. All rights reserved. 6

7 Important to differentiate between ordinal and interval scales, health outcomes are mostly measured on ordinal scales Data Scale Examples Attributes can be rank-ordered (sorted), but distances between attributes have no meaning Ordinal Level of education, IQ, ability to ambulate, health outcomes, What we (might) have is ordinal scales Distance between attributes have meaning (averages can be computed), but ratio s have no meaning Interval Temperature ( o C), date What we want to have are interval scales A meaningful absolute zero exists, a meaningful ratio can therefore be calculated Ratio Weight, mass, length, temperature (K) There is no rational basis for adding up a set of ordinal Likert scores, unless they have been shown to scale * *Source: Matthew Hankins, Senior lecturer in Public Health at University of Southampton, 2011 presentation Copyright 2015 by the International Consortium for Health Outcomes Measurement. All rights reserved. 7

8 Item response theory is the field of work concerning the translation of ordinal health outcomes scores to interval scores Data Scale Examples Attributes can be rank-ordered (sorted), but distances between attributes have no meaning Distance between attributes have meaning (averages can be computed), but ratio s have no meaning Ordinal Interval Level of education, IQ, ability to ambulate, health outcomes, Temperature ( o C), date Item Response Theory A meaningful absolute zero exists, a meaningful ratio can therefore be calculated Ratio Weight, mass, length, temperature (K) Copyright 2015 by the International Consortium for Health Outcomes Measurement. All rights reserved. 8

Determining a questionnaire s psychometric properties is an important step in its development Rasch analysis is one of many possible Item Response Theory models which results in the anchoring of

9 Determining a questionnaire s psychometric properties is an important step in its development Rasch analysis is one of many possible Item Response Theory models which results in the anchoring of questions to a scale Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 Q10 Q11 Q12 Q13 High difficulty Low difficulty Q9 Q11 Q6 Q5 Q1 Q3 Q13 Q8 Q2 Q10 Q7 Q4 Q12 Copyright 2015 by the International Consortium for Health Outcomes Measurement. All rights reserved. 9

Optometry and Vision Science, Vol. 87, No.

11 Developing PROMs to have more impact Konrad Pesudovs, PhD Item Banking: A Generational Change in Patient-Reported Outcome Measurement. Optometry and Vision Science, Vol. 87, No. 4, April 2010 Copyright 2015 by the International Consortium for Health Outcomes Measurement. All rights reserved. 11

Item Banking Copyright 2015 by the International

15 The Eye-tem Bank Project To develop the most comprehensive and technologically advanced patient-reported outcome measurement system for eye disease- the Eye-tem Bank To develop, validate, and implement an item banking and CAT system to assess ophthalmic quality of life (QOL) across all eye diseases Eye-tem Bank modules Glaucoma Diabetic retinopathy.age-related macular degeneration Retinal detachment Other vitreo-retinal Cataract and Corneal opacities Corneal diseases Refractive error Strabismus and Amblyopia Uveitis spectrum of diseases Ocular inflammation other than uveitis Ocular-surface and Lacrimal Neuro-ophthalmic Eye-tem Bank QOL domains Activity limitation Mobility.Ocular comfort symptoms Visual symptoms General symptoms Emotional well-being Health concerns Convenience Social participation and well-being Economic (work and finance) Copyright 2015 by the International Consortium for Health Outcomes Measurement. All rights reserved. 15

Developing PROMs to have more impact Matthias Rose, MD, PhD Copyright 2015 by the

Medicine and Dermatology Charité Universitätsmedizin Berlin, Germany

18 Developing PROMs to have more impact Harmonisation ICHOM London 2016 Matthias Rose Department of Psychosomatic Medicine Center for Internal Medicine and Dermatology Charité Universitätsmedizin Berlin, Germany Department of Quantiative Health Sciences University of Massachusetts, USA

19 What is wrong with today s instruments?

20 What is Wrong with Today s Measurement? not precise measurement precision does not meet required standards for individual clinical decision making restricted measurement range clinical instruments do not provide information for the general population CAT Konrad Pesudovs talk! not comparable same constructs are often measured with different instruments without being comparable

21 Not Comparable Psychometric Utility Related Concepts and Characteristics SIP HIE NHP QLI COOP QLQ MOS MOS QWB EURO HUI SF-6D FWBP SF-36 -QOL CONCEPTS Physical functioning Social functioning Role functioning Psychological distress Health perceptions (general) Pain (bodily) Energy/fatigue Psychological well-being Sleep Cognitive functioning Quality of life Reported health transition Financial Impact SIP = Sickness Impact Profile (1976) HIE = Health Insurance Experiment surveys (1979) NHP = Nottingham Health Profile (1980) QLI = Quality of Life Index (1981) COOP = Dartmouth Function Charts (1987) QLQ-C30 = EORTC MOS FWBP = MOS Functioning and Well-Being Profile (1992) MOS SF-36 = MOS 36-Item Short-Form Health Survey (1992) QWB = Quality of Well-Being Scale (1973) EUROQOL = European Quality of Life Index (1990) HUI = Health Utility Index (1996) SF-6D= SF-36 Utility Index (Brazier, 2002) adapted from Ware, Medical Care 1995

22 What is Wrong with Today s Measurement? not precise measurement precision does not meet required standards for individual clinical decision making restricted measurement range clinical instruments do not provide information for the general population CAT not comparable same constructs are often measured with different instruments without being comparable not anchored score values have no intuitive meaning

23 Anchoring and Standardization water boils e.g. temperature 212 body temperature water freezes absolute zero Fahrenhei t 1724 Reaumur 1731 Celsius 1742 Kelvin 1848 Rankine 1859

24 Can IRT item banks also solve those problems?

25 Representative Samples low Item Bank - Depression Q happy 11 established instruments Q Wahl et al. J Clin Epi 2014 relaxed Questionaire A Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q Q QQ Q Q Q Q Q lost of Q Q interest Q Q Q Q Q Q difficulties concentrate feeling down feeling depressed Patients with Depression 7 Clinical Sites / 12 Health Centers total n > 33, Theta Q feeling guilty suicidal ideation Q Questionnaire B Q high

26 Standardized Metric Beck Depression Inventory Hospital Anxiety Depression Scale Well-Being Index Depression Cut-Off Depression Mean Representative Sample BDI 1961 HADS WHO-5 Theta Q

Standardized Metric Mercury Quartz Infrared Temperature 39.

27 Standardized Metric Mercury Quartz Infrared Temperature Cut-Off Fever Mean Representative Sample Celsius Q

28 Comparability HbA1c Depression 10 % 95%CI Theta Q 75 95%CI Cut off 8 17 Depression 6 Upper limit Week 1 Week 6 Week 1 Week 3 Week 6 HPCL Methode (IFCC) Standard Depressions tool (HADS) CAT HADS BDI Fliege et al. QLR 2005, Rose et al. Psychosomatic Med 2012, Devine et al. JCE 2015

29 Fisher et al. 2014

30 Are patient-reported health parameters disease specific?

31 Phenotype of Depressivity Hasselmann et al. Multiple Sclerosis Journal 2016

something g-globulin but is not unique to

32 Health Parameters Nephrology? Hematology hemoglobin kidney failure creatinine plasmacytoma σύμπτωμα (symptoma) = it falls together with something g-globulin but is not unique to it disease specific measures = compilation of health parameters fatigue

33 For which constructs do harmonized measurements make sense?

composite scores, preference instruments, treatment satisfaction, e.g.

34 Suitable Constructs main health constructs interval scales, e.g.: Depression Anxiety Fatigue Pain Shortness of breath Physical functioning Cognitive functioning Sleep functioning composite scores, preference instruments, treatment satisfaction, e.g.: Asthma control Questionnaire Anti-Clot Treatment Scale Alzheimer s Disease Care Giver Performance Questionnaire Ascites Impact Measure Comprehensive Assessment of Satisfaction with Care Diabetes Foot Ulcer Scale Social role functioning

35 Conclusion

36 Conclusion I II III Item banks allow to move away from an instrument to a construct defined measurement system Item banks provide a common metric for existing tools measuring the same construct Item banks make sense for key health indicators, and will allow to treat subjective health assessments similar to other clinical markers

Developing PROMs to have more impact James Willig, MD, MSPH Copyright 2015 by the

38 PROMs in Clinical Care Settings James H. Willig, MD, MSPH University of Alabama at Birmingham

40 1917 Clinic Implementation

42 One person monitoring completion real time

45 1917 Palliative Care Clinic Symptoms Improved Pain Pain Anxiety QOL PRO Implementation Selection of instruments Sequence Internal logic Frequency Alerts Depression Notification Triggered Clinical: SI, IPV (starts clinic response protocol) Study enrollment

46 Settings Social Services 1 2,610 Palliative 1 1,208 Signs & Symptoms 1 52,249 Viral Hepatitis 1 1,213 Supportive Care 2 1,376 Breast Health iengage 3 & BA2C Pediatric Neurosurgery 65 Sessions (1) 1917 Clinic; (2) TKC; (3) Research (2) As of 3/31/2016

Developing PROMs to have more impact Helen Burstin, MD, MPH, FACP Copyright 2015 by

48 Developing PROMs to Have More Impact Helen Burstin, MD, MPH, FACP Chief Scientific Officer, NQF ICHOM 2016 Annual Conference May 2016

49 What is NQF? NQF is an independent, nonprofit, membership organization that brings together all stakeholders working to improve health and healthcare through quality measurement. 49

50 U.S. Policy Environment: From Volume to Value 50

51 Measuring Value 51

52 Patient Focused Episodes Population at risk Acute Phase Post- Acute/ Rehab Secondary Prevention Functional Status Quality of Life Shared decision-making Clinical outcomes & PROs Costs 52

Selection and Use of PROs in Measurement Guiding principles for PRO selection: psychometric soundness, person-centered, meaningful, actionable, implementable Challenges to use PROs for

53 Selection and Use of PROs in Measurement Guiding principles for PRO selection: psychometric soundness, person-centered, meaningful, actionable, implementable Challenges to use PROs for accountability and performance improvement: Frequently used in research, but not in clinical use Aggregation of patient-reported information to measure provider performance challenging Proprietary tools 53

54 The Pathway from PROs to PRO-PMs PRO Identify the quality performance issue (include broad input) Identify outcomes meaningful to target population and amenable to change Determine whether (PRO) is the best way to assess the outcome of interest PROM Identify existing PROMs for measuring the outcome in the target population Select PROM suitable for use in performance measurement (e.g., reliable, valid, feasible) Use the PROM in real world with the intended target population and setting PRO-PM Specify the outcome performance measure (e.g., average change, percentage improved or meeting a benchmark) Test PRO-PM for reliability, validity, and threats to validity (e.g., measure exclusions; missing data; poor response rate; risk adjustment; discrimination of performance; equivalence of results across PROMs) 54

55 The Pathway from PROs to PRO-PMs PRO (concept) PROM (instrument, tool, single-item measure) PRO-PM (PRO-based performance measure) Symptom: Depression PHQ-9 standardized tool to assess depression Percentage of depressed patients with remission or progress toward remission in PHQ-9 scores at 6 months and at 12 months 55

56 NQF Measure Incubator: Getting to Quality Measures that Matter Topic Developer Data Funding Novel Measures emeasures Outcome Measures Patient-reported Outcome Measures Cost/Efficiency/ Value Measures Improved Patient Care and Outcomes 56

57 Helen Burstin, MD, MPH, 57

For more information: Quality of Life. World Health Organization Definition of Health

For more information: Quality of Life. World Health Organization Definition of Health Health-Related Quality of Life - 10 John E. Ware, Jr., PhD, Professor and Chief Measurement Sciences Division, Department of Quantitative Health Sciences University of Massachusetts Medical School, Worcester,