Patient Reported Outcomes

Transcription

1 Patient Reported Outcomes INTRODUCTION TO CLINICAL RESEARCH A TWO-WEEK INTENSIVE COURSE, 2010 Milo Puhan, MD, PhD, Associate Professor

2 Key messages Patient-reported outcomes (PRO) is a broad group of outcomes that directly reflect the patients perspective on symptoms, functional impairment and health-related quality of life Patient reported outcomes for comparative effectiveness research should be selected based on: Good responsiveness Established test-retest reliability Knowledge about their minimal importance difference Existing literature 2

3 What do patient report outcomes (PRO) measure?

4 Definition of patient-reported outcomes Information about a feature of health/disease that is obtained directly from the patient (with no interpretation by anyone else) 4

5 PROs: from simple to very complex measurements Multi-item Mutiple domains Measurement Complexity Pain Fatigue Symptom Index Physical Function Social Activities Health-related quality of life Activities of Daily Living Dyspnea during daily activities Pain during exercise Single Item Dyspnea when walking Abstractness of outcome

6 Patient-reported outcomes: Symptoms Uni-dimensional Dyspnea associated with physical activity Medical Research Council dyspnea scale Grade Degree of breathlessness related to activities 1 Not troubled by breathlessness except on strenuous exercise 2 Short of breath when hurrying or walking up a slight hill Walks slower than contemporaries on level ground because of breathlessness, or has to stop for breath when walking at own pace Stops for breath after walking about 100m or after a few minutes on level ground Too breathless to leave the house, or breathless when dressing or undressing 6

7 Patient-reported outcomes: Symptoms Multi-dimensional Dyspnea in different situations Chronic Respiratory Questionnaire dyspnea domain Extremely short of breath Very short of breath Quite a bit short of breath Moderate shortness of breath Some shortness of breath A little shortness of breath Not at all short of breath Not Done 1 Feeling emotional such as angry or upset 2 Taking care of your basic needs (bathing, showering, eating or dressing) 3 Walking 4 Performing chores (such as housework, shopping, groceries) 5 Participating in social activities Summary dyspnea domain score 7

8 Patient-reported outcomes: Health-related quality of life Security Symptoms Economy Health Quality of life Health-related quality of life Political stability Infrastructure Social environment Physical functioning Mental functioning Example: Mercer-Studies Example: SF-36 8

9 Patient-reported outcomes: Health-related quality of life Symptoms Pain Dyspnea Fatigue Physical functioning House keeping Walking Doing sports Health-related quality of life Mental functioning Depression Anxiety Questions do not address directly quality of life Abstract construct derived from underlying questions 9

10 Patient-reported outcomes: Health-related quality of life Uni-dimensional Please indicate on the scale from 0 to 100 how your health state was during last 7 days. Best imaginable health state Worst imaginable health state 5 Full health Death 10

11 Patient-reported outcomes: Health-related quality of life SF-36 8 domains Physical function 36 questions Role physical Bodily pain General Health Vitality Social function 2 summary scores Physical component score Mental component score Role emotional Mental Health 11

12 Patient-reported outcomes: Health-related quality of life Disease-specific instruments: Chronic Respiratory Questionnaire 4 domains Dyspnea 20 questions Fatigue Emotional function Mastery Total score 12

13 PROs: from simple to very complex measurements Multi-item Mutiple domains Measurement Complexity Pain Fatigue Symptom Index Physical Function Social Activities Health-related quality of life Activities of Daily Living Dyspnea during daily activities Pain during exercise Single Item Dyspnea when walking Abstractness of outcome

14 Development of PROs

15 The development of PROs is iterative

16 Is there a specific conceptual framework? A valid PRO should be based on a conceptual framework, which includes: What is the target population? What should be measured? Any disease? Specific disease? Disease severity? Clinical or general population? Symptoms? Functional impairment? Health-related quality of life? What s the purpose of the instrument? Treatment evaluation? Comparison of patients? Prediction of outcomes? evaluative discriminative predictive 16

17 The Chronic Respiratory Questionnaire Conceptual framework Disease: COPD Severity: Moderate to severe Concept COPD-specific healthrelated quality of life Population: Clinical Purpose: Evaluative Domains Dyspnea Emotional function Fatigue Mastery Items Dyspnea when doing chores Answer options: Likert-type 1-7 Scores: 1-7 per domain Recall period: 2 weeks 17

18 Steps in the development of PROs Conceptual framework A priori considerations Item and domain identification Item selection Questions Answer options Revision Adaptation Pilot testing Testing of measurement properties (validation) Translations Validation on other populations 18

19 Translation of PROs Testing of measurement properties (validation) Translations Forward Translation 1 Forward Translation 2 Professional interpreters Pilot testing Translated version1 Consensus meeting Professional interpreters Clinicians Methodologists Back translation Professional interpreter Revision Translated version 2 Validation Pilot testing 19

20 Measurement formats and properties

21 Reliability and validity Poor reliability and validity Good reliability but poor validity Good reliability and validity 21

22 Test-retest reliability First administration Typically 1-2 weeks) No change (treatments, exacerbations) Second administration r = 0.9 Intra-class correlation coefficient Between person variance Between + within person variance Inappropriate statistic Not bad, but does not correct for systematic differences Adequate 22

23 The validity challenge: No gold standard Face validity (expert knowledge) considered insufficient Known-group validity: Expected versus observed distributions Dyspnea Extreme Moderate No Mild Moderate Severe Severity of disease Construct validity: Expected versus observed correlations with external validation instruments 23

24 Construct validity: Expected versus observed correlations Instrument Chronic Respiratory Questionnaire Validation instruments Dyspnea Fatigue Mastery Emotional function MRC Dyspnea scale Self-reported physical activity Six-minute walk distance Exacerbations SF-36 physical functioning SF-36 mental functioning Hospital Anxiety and Depression Scale

25 Construct validity: Expected correlations Instrument Chronic Respiratory Questionnaire Validation instruments Dyspnea Fatigue Mastery Emotional function MRC Dyspnea scale Self-reported physical activity Six-minute walk distance Exacerbations SF-36 physical functioning SF-36 mental functioning Hospital Anxiety and Depression Scale High (>0.5) Moderate ( ) Moderate ( ) Moderate ( ) Moderate ( ) Low (<0.3) Low (<0.3) Moderate ( ) High (>0.5) Moderate ( ) Low (<0.3) Moderate ( ) Low (<0.3) Low (<0.3) Moderate ( ) Low (<0.3) Low (<0.3) High (>0.5) Low (<0.3) Moderate ( ) Moderate ( ) Low (<0.3) Low (<0.3) Low (<0.3) Moderate ( ) Low (<0.3) High (>0.5) High (>0.5)

26 Construct validity: Observed correlations Instrument Chronic Respiratory Questionnaire Validation instruments Dyspnea Fatigue Mastery Emotional function MRC Dyspnea scale 0.62 (>0.5) Self-reported physical activity 0.40 ( ) Six-minute walk distance 0.31 ( ) Exacerbations 0.38 ( ) SF-36 physical functioning 0.43 ( ) SF-36 mental functioning 0.20 (<0.3) Hospital Anxiety and Depression Scale 0.13 (<0.3) 0.43 ( ) 0.51 (>0.5) 0.44 ( ) 0.26 (<0.3) 0.47 ( ) 0.10 (<0.3) 0.11 (<0.3) 0.33 ( ) 0.21 (<0.3) 0.24 (<0.3) 0.47 (>0.5) 0.14 (<0.3) 0.54 ( ) 0.51 ( ) 0.15 (<0.3) 0.06 (<0.3) 0.22 (<0.3) 0.26 ( ) 0.09 (<0.3) 0.58 (>0.5) 0.81 (>0.5)

27 Responsiveness: Expected versus observed change Change expected First administration Second administration Treatment of known effectiveness Dyspnea Extreme No Baseline Follow-up Exacerbation (recovery) Dyspnea Extreme No Beginning Peak Exacerbations Recovery

28 Measures of responsiveness Treatment of known effectiveness Dyspnea Extreme No Baseline Follow-up Examples Comparison baseline and follow-up t-tests Wilcoxon Depends (too much) on sample size!! Cohen s effect size Mean change SD baseline Effect sizes Standardized response mean Mean change SD change 0.2 to <0.5 = small effect 0.5 to <0.8 = moderate effect 0.8 = large effect Many other measures 28

29 Generic vs. disease-specific instruments 281 COPD patients from four sites Respiratory rehabilitation of 12 weeks 179 patients at follow-up (102 lost because of SARS outbreak and other reasons) Generic instruments COPD-specific instruments SF-36 3 utility instruments Chronic Respiratory Questionnaire St. Georges Respiratory Questionnaire Resp Med 2007; 101,

30 Disease-specific instruments are more responsive COPD-specific instruments Standardized response mean CRQ, 4 domains SGRQ, 3 domains Generic instruments SF-36, 8 domains utility instruments Resp Med 2007; 101,

31 Conclusions responsiveness Look at responsiveness data Responsiveness disease-specific > disease-specific >> generic > generic 31

32 Minimal important difference

33 How should we interpret these data? Respiratory rehabilitation vs usual care for COPD Study Difference CRQ (95% CI) Simpson K ( ) Goldstein ( ) Güell ( ) Wijkstra PJ ( ) Cambach ( ) Hernandez MT ( ) Troosters T ( ) White ( ) Oh Eg ( ) Singh V ( ) Faager ( ) Combined 0.54 ( , p<0.001) favours usual care favours rehabilitation total score 33

34 The p-value is often not useful!! Imagine the following results Difference CRQ (95% CI) Combined 0.54 ( , p<0.001) Combined 0.44 ( , p<0.001) Combined 0.34 ( , p<0.001) Combined 0.24 ( , p=0.01) Combined 0.14 ( , p=0.05) Combined 0.04 ( , p=0.25) favours usual care favours rehabilitation total score We need a quantitative interpretation 34

35 The minimal important difference the smallest difference in the outcome of interest that informed patients or their proxies perceive as important and that may lead to a change in the management Related terms: Minimal clinically important difference (MCID) Clinically important difference Health services research 2005, 40(2):593 35

36 The MID of the CRQ is 0.5 Study Effects of inhaled drugs Difference CRQ (95% CI) Simpson K ( ) Goldstein ( ) Güell ( ) Wijkstra PJ ( ) Cambach ( ) Hernandez MT ( ) Troosters T ( ) White ( ) Oh Eg ( ) Singh V ( ) Faager ( ) Combined 0.54 ( , p<0.001) favours usual care favours rehabilitation total score 36

37 Methods to determine the MID Consensus-based Distribution-based Anchor-based Experts reach a consensus (e.g. Delphi methods) Statistical criteria: How strong is signal compared to noise Standardized response mean 0.5 Cohen s effect size 0.5 Standard error of measurement External instrument (anchor) with known MID used to estimate MID 37

38 MID of the Hospital Anxiety and Depression Scale in COPD patients Health and Quality of Life Outcomes 2008, 6:46 38

39 The HADS MID for COPD patients Between 1.41 and 1.68 with anchor-based method Between 1.17 and 1.40 with distribution-based method 1.5 on scale from 0-21 seems fair estimate Always use different methods since none of them is perfect 39

40 Use of the MID Interpretation of treatment effects for superiority AND non-inferiority trials Non-inferiority analysis for interval exercise trial Favours continuous exercise (n=41) CRQ Domain Favours Interval exercise (n=44) Difference Dyspnea 0.07 (-0.53 to 0.38) Fatigue 0.02 (-0.41 to 0.32) Emotional function 0.08 (-0.48 to 0.32) Mastery 0.01 (-0.42 to 0.45) Total score (-0.42 to 0.32) Differences between improvements Sample size calculations detection of important difference Annals of Internal Medicine 2006;145(11):816 40

41 Selection of PRO instruments

42 PROs: What are you interested in? Multi-item Mutiple domains Measurement Complexity Pain Fatigue Symptom Index Physical Function Social Activities Health-related quality of life Activities of Daily Living Dyspnea during daily activities Pain during exercise Single Item Dyspnea when walking Abstractness of outcome

43 43

44 Have standard steps for the development been followed? Conceptual framework A priori considerations Item and domain identification Item selection Questions Answer options Revision Adaptation Pilot testing Testing of measurement properties (validation) Translations Validation on other populations 44

45 Choice of patient-reported outcomes for CER Symptoms, functioning or health-related quality of life? Ideally matches measures of existing trials Meta-analysis! Can be measured reliably Responsive to change disease-specific instruments Established MID using valid methods Is reasonably efficient to measure (budget considerations) 45

46 Planning of a CER study in COPD Trial of rehabilitation in patients with COPD Our considerations Interested in health-related quality of life Matches measures of existing trials Can be measured reliably Responsive to change Established MID Efficient to measure MRC dyspnea scale SF-36 CRQ St Georges Resp Q 46

47 Planning of a CER study in COPD Trial of rehabilitation in patients with COPD Our considerations Interested in health-related quality of life Matches measures of existing trials MRC dyspnea scale SF-36 CRQ St Georges Resp Q /- + + Can be measured reliably +/ Responsive to change /- Established MID +/- +/- + + Efficient to measure + - +/- - 47

48 Key messages Patient-reported outcomes (PRO) is a broad group of outcomes that directly reflect the patients perspective on symptoms, functional impairment and health-related quality of life Patient reported outcomes for comparative effectiveness research should be selected based on: Good responsiveness Established test-retest reliability Knowledge about their minimal importance difference Existing literature 48

49 Further reading Streiner DL, Norman GR. Health Measurement Scales. Oxford University Press, 2003, 3rd edition. ISBN13: Journals: Health and Quality of Life Outcomes Journal of Clinical Epidemiology Quality of Life Research 49