VALUE IN HEALTH 19 (2016) 834 843 Available online at www.sciencedirect.com journal homepage: www.elsevier.com/locate/jval PREFERENCE-BASED ASSESSMENTS Which Questionnaire Should Be Used to Measure Quality-of-Life Utilities in Patients with Acute Leukemia? An Evaluation of the Validity and Interpretability of the EQ-5D-5L and Preference-Based Questionnaires Derived from the EORTC QLQ-C30 Annemieke van Dongen-Leunis, PhD*, W. Ken Redekop, PhD, Carin A. Uyl-de Groot, PhD Institute for Medical Technology Assessment/Institute of Health Policy & Management, Erasmus University Rotterdam, Rotterdam, The Netherlands ABSTRACT Objectives: The aim of this study was to assess the validity and interpretability of different preference-based questionnaires (generic 5-level EuroQol five-dimensional questionnaire [EQ-5D-5L], cancerspecific Quality of Life Questionnaire Preference-Based Measure, and European Organization of Randomized Controlled Trials 8 Dimension [EORTC-8D]) in patients with acute leukemia. Methods: Patients who participated in Hemato-Oncologie voor Volwassenen Nederland (HOVON the Haemato Oncology Foundation for Adults in the Netherlands) clinical trials between 1999 and 2011 at a single hospital were invited to complete the questionnaires. Interpretability was evaluated by the frequency of incomplete data and highest and lowest possible scores. Content validity was evaluated by exploring the health-related quality-of-life domains included in the questionnaires. Construct validity was assessed using correlations with other qualityof-life scales (EQ-visual analogue scale score and global quality-of-life scale of the EORTC Quality of Life Questionnaire) and ability to distinguish between patients with different health statuses. Results: Questionnaires were returned by 89% (111 of 125) of the patients. Six to seven respondents did not return full questionnaires. Perfect health on the EQ-5D-5L was reported by 32 respondents and many of them (N ¼ 17) did report health problems on other questionnaires. All questionnaires were strongly correlated (range 0.61 0.78) with other quality-of-life scales and yielded substantially different utility values for patients with different health statuses. Nevertheless, the diseasespecific preference-based questionnaires showed greater discriminatory power. Conclusions: Although the Quality of Life Questionnaire Preference-Based Measure and the EORTC-8D appear to have better validity, this study does not provide any strong evidence against the use of the EQ-5D-5L for measuring quality-of-life utilities in acute leukemia. However, our findings need to be confirmed in larger longitudinal studies. Keywords: acute leukemia, EQ-5D-5L, disease-specific preferencebased instrument, validity. Copyright & 2016, International Society for Pharmacoeconomics and Outcomes Research (ISPOR). Published by Elsevier Inc. Introduction Many different questionnaires are currently available to estimate health-related quality of life for use in economic evaluations. These questionnaires include generic questionnaires such as the EuroQol five-dimensional questionnaire (EQ-5D), the sixdimensional health state short form (derived from the 36-item short form health survey), and health utilities index 3, as well as disease-specific preference-based measures [1]. It has been shown that the utility values of these questionnaires differ because of differences in the content of the questionnaires, valuation techniques, and study populations [1 7]. These differences limit the comparability of economic evaluations if different questionnaires are used to calculate quality-of-life utilities. Therefore, the National Institute of Health and Care Excellence in the United Kingdom has explicitly stated that the EQ-5D should be used to measure quality-of-life utilities, unless there is proof that the EQ- 5D is not valid in the target population [8]. According to the National Institute of Health and Care Excellence recommendation, evidence about a lack of validity of the EQ-5D is required to allow the use of disease-specific questionnaires for estimating quality-of-life utilities in specific patient populations. At this moment, that evidence is not available for patients with acute leukemia, but nevertheless, the EQ-5D has rarely been used to measure quality of life in these patients [9]. A more frequently used quality-of-life questionnaire in this patient * Address correspondence to: Annemieke van Dongen-Leunis, Institute for Medical Technology Assessment/iBMG, Erasmus University Rotterdam, PO Box 1738, 3000 DR Rotterdam, The Netherlands. E-mail: a.vandongen@bmg.eur.nl. 1098-3015$36.00 see front matter Copyright & 2016, International Society for Pharmacoeconomics and Outcomes Research (ISPOR). Published by Elsevier Inc. http://dx.doi.org/10.1016/j.jval.2016.05.008
VALUE IN HEALTH 19 (2016) 834 843 835 population is the Quality of Life Questionnaire for Cancer from the European Organization of Randomized Controlled Trials (EORTC QLQ-C30). At this moment, quality-of-life utilities can be derived from the EORTC QLQ-C30 from two different preference-based questionnaires, the Quality of Life Questionnaire-Preference-Based Questionnaire (QLQ-PBM) [4] and the European Organization of Randomized Controlled Trials-8 Dimensions (EORTC-8D) [10]. Insight into the validity of both the EQ-5D and the two diseasespecific preference-based questionnaires in patients with acute leukemia is needed to select the most appropriate questionnaire for estimating quality-of-life utilities in these patients. Although the validity of the EQ-5D has not been evaluated in patients with acute leukemia, evidence about its validity is available for other cancer types. These studies showed that the EQ-5D is a valid and reliable questionnaire to measure quality of life in several cancer populations, but a relatively high percentage of patients reported full health (ceiling effect) on the EQ-5D and small changes in health were difficult to identify [2,5,11 17]. Because these shortcomings were also observed in other disease areas, the EuroQol group has recently developed the 5-level EuroQol five-dimensional questionnaire (EQ-5D-5L), which has five instead of three levels [18]. The first evaluations of the psychometric properties of this adjusted questionnaire showed a smaller ceiling effect and improved discriminative power due to the addition of two more levels [19 21]. Although these studies indicate an improved validity of the EQ-5D-5L, assessment of its validity in patients with acute leukemia is needed to justify its use in economic evaluations for this patient population. Alternatives for estimating quality-of-life utilities for patients with acute leukemia are the QLQ-PBM and the EORTC-8D. Both questionnaires are shortened versions of the EORTC QLQ-C30 with some differences in the selected items (Table 1). Although the EORTC QLQ-C30 has been validated in several cancer populations [22 24], evidence about the validity of the shortened questionnaire is not yet available. Therefore, this study aimed to assess the validity of not only the EQ-5D-5L in patients with acute leukemia but also the QLQ-PBM and the EORTC-8D in this patient population. Methods Patient Selection This validation study was performed as a secondary analysis of a cross-sectional quality-of-life study in patients with acute leukemia [9]. Questionnaires were sent to all patients with acute leukemia who participated in clinical trials HOVON-29, HOVON-37, HOVON-42(A), HOVON-43, HOVON-70, HOVON-71, HOVON-81, and HOVON-92 [25 29] between 1999 and 2011 and received first-line treatment at Erasmus University Medical Center (Erasmus MC) in Rotterdam, the Netherlands (N ¼ 125). Patients were excluded if no follow-up visits were planned in 2012 at Erasmus MC. The main reasons for loss to follow-up were migration from the region (within or outside the Netherlands) and scheduled follow-up visits in other hospitals closer to their homes. Patients were informed that they would give permission to participate in the study by returning the questionnaire. The study was approved by the institutional medical ethics committee (Erasmus MC MEC- 2011-490). Sociodemographic and Clinical Data The questionnaire included questions about the age, sex, education, employment status, and marital status of the patient. Clinical data, including type of acute leukemia, date of diagnosis, last treatment received, and leukemia recurrence, were obtained via the HOVON Data Center. Quality-of-Life Questionnaires The EQ-5D-5L consists of the EQ-5D descriptive system and the EQ-visual analogue scale (EQ-VAS). The descriptive system includes five questions representing the dimensions mobility, self-care, usual activities, pain/discomfort, and anxiety/depression [30]. All questions have a five-level response scale [16,18]. The quality-of-life utilities were derived from both the UK and Dutch tariffs [31] to account for differences in preferences between countries in the comparison with disease-specific preference-based questionnaires. Because the QLQ-PBM and the EORTC-8D were valued by Dutch and UK general public, respectively, the Dutch tariff was used for comparison of the EQ-5D-5L with the QLQ-PBM and the UK tariff was used for comparison of the EQ-5D-5L with the EORTC-8D. The EQ-VAS is a rating scale ranging from 0 (worst imaginable health) to 100 (best imaginable health). Both the QLQ-PBM and the EORTC-8D were derived from the EORTC QLQ-C30 version 3.0 [32,33]. For both preference-based questionnaires, the 30 items of the EORTC QLQ-C30 have been Table 1 Content of the three preference-based questionnaires. HRQOL domain QLQ-PBM EORTC-8D EQ-5D-5L Physical functioning Trouble taking a long walk Trouble taking a long and short walk Problems in walking about Role functioning Limited in doing either work or other daily activities Limited in pursuing hobbies or other leisure activities Problems doing usual activities Self-care Problems with washing or dressing themselves Social functioning Physical condition or medical treatment interfered with social activities Physical condition or medical treatment interfered with social activities Emotional functioning Being worried Feel depressed Being anxious or depressed Cognitive functioning Difficulty in concentrating on things Pain Existence of pain Pain interferes with daily activities Having pain or discomfort Nausea Felt nauseated Felt nauseated Fatigue Being tired Being tired Constipation and diarrhea Constipated and/or having diarrhea EORTC-8D, European Organization of Randomized Controlled Trials 8 Dimension; EQ-5D-5L, five-level EuroQol five-dimensional questionnaire; HRQOL, health-related quality of life; QLQ-PBM, Quality of Life Questionnaire Preference-Based Measure.
836 VALUE IN HEALTH 19 (2016) 834 843 reduced to 8 items [4,10]. Health states from these shortened questionnaires were valued by the general public by means of a time trade-off method. Detailed information about the item selection and the valuation method for the QLQ-PBM and the EORTC-8D is reported by Versteegh et al. [4] and Rowen et al. [10], respectively. All respondents in this study were asked to fill in the full QLQ-C30 version 3.0, which enabled us to estimate utilities for both disease-specific preference-based questionnaires. The two shortened questionnaires differ with respect to the selected items (see Table 1) and the population in which the health states has been valued. Important to note is that the QLQ-PBM was derived from the EORTC QLQ-C30 version 2.0. This older version only asked whether patients had difficulties with taking a long walk without asking the extent of these problems. Consequently, the different levels of problems with taking a long walk on the QLQ-C30 version 3.0 were combined in one answer category (having problems) for the QLQ-PBM. Statistical Analysis The analysis of the psychometric properties of the two questionnaires included an analysis of the validity and interpretability of the questionnaires. Interpretability of the questionnaire is an important prerequisite for adequate measurement of difference in quality of life. Interpretability of the questionnaires is assessed by the percentage of respondents with incomplete questionnaires (i.e., missing or inconclusive scoring of one or more items) and by the percentage of respondents reporting the best or worst score (ceiling and floor effect) [34]. Incomplete questionnaires and/or large ceiling or floor effects may limit the ability to detect relevant changes in quality of life and therefore limit the feasibility of the questionnaire. The evaluation of the validity of the questionnaires was limited to an assessment of content and construct validity. Content validity was evaluated by exploring the different domains of health-related quality of life that have been included in the three preference-based questionnaires [34]. Spearman correlation coefficients were estimated between the domains of the EQ-5D-5L, the QLQ-PBM, and the EORTC-8D to evaluate differences in content between the questionnaires. The construct validity was evaluated in two ways. First, it was evaluated whether the utility scores of the preference-based questionnaires were strongly correlated (r 4 0.5) [35] with two other quality-of-life scales: the EQ-VAS and the global quality-of-life scale of the QLQ-C30 (QL scale). Second, it was evaluated whether the preference-based questionnaires were able to distinguish between patients with different health statuses [34]. Three different measures were used to distinguish patients with different health statuses. Both the EQ-VAS and the QL scale were used as proxies for health status. Quartile scores, as observed in this patient population, were used to categorize these variables into four subgroups. Furthermore, patients younger than 65 years were distinguished according to the self-reported ability to work. T tests and analysis of variance were used to test for significant differences in utility scores between the subgroups. Standardized effect sizes (ESs) (Cohen s d) were calculated by dividing the difference in quality-of-life utility of subsequent categories by the overall SD of the two categories. A Cohen s d of around 0.2 was defined as a small ES, around 0.5 as a moderate ES, and around 0.8 as a strong ES [35]. All analyses were performed in SAS 9.2 (SAS institute, inc). Results Patient Characteristics A total of 111 respondents returned the questionnaire (89%). The characteristics of the respondents are presented in Table 2. Table 2 Patients characteristics. Characteristic Value Age (y) Mean SD 51 13.4 Median (range) 52 (23 78) Time since diagnosis (y) Mean SD 6 2.7 Median (range) 6 (2 13) Ethnicity, n (%) European 97 (87) Other 12 (11) Unknown 2 (2) Sex, n (%) Female 53 (48) Male 58 (52) Level of education, n (%) Elementary school or secondary education 32 (29) Vocational school 49 (44) University 29 (26) Unknown 1 (1) Employment status, n (%) Paid job 48 (43) Retired 21 (19) Other 41 (37) Unknown 1 (1) Marital status, n (%) Living with a partner 84 (76) Living without a partner 27 (24) Leukemia type, n (%) Acute lympoblastic leukemia 19 (17) Acute myeloid leukemia 92 (83) Last treatment received, n (%) Chemotherapy 34 (31) Autologous HSCT 13 (12) Allogeneic HSCT 64 (58) Relapse, n (%) Yes 20 (18) No 91 (82) HSCT, hematopoietic stem cell transplantation. The respondents were aged between 23 and 78 years with a median age of 52 years. The time since diagnosis ranged between 2 and 13 years with a median of 6 years. About half the respondents were men. Most respondents had a European nationality and lived with a partner. About 60% of the respondents had a paid job or were retired; the remainder mainly consisted of patients who indicated to be unable to work because of cancer. Interpretability of the Questionnaires Incomplete data were found for six respondents for the EQ-5D-5L and the EORTC-8D and for seven respondents for the QLQ-PBM. Only one respondent had incomplete data for both the EQ-5D-5L and the cancer-specific preference-based questionnaires. None of the respondents had incomplete data for all items of the questionnaires. A fairly large proportion of patients reported no problems on the individual items of the EQ-5D-5L, the QLQ-PBM, and the EORTC-8D (Fig. 1). The largest ceiling effects were found for the self-care item of the EQ-5D-5L and the nausea and constipation and diarrhea item of the disease-specific preferencebased questionnaires. With respect to the items that differ
VALUE IN HEALTH 19 (2016) 834 843 837 Fig. 1. Floor and ceiling effects of the individual items and utility scores of (A) the EQ-5D-5L, (B) the QLQ-PBM, and (C) the EORTC-8D. EORTC-8D, European Organization of Randomized Controlled Trials 8 Dimension; EQ-5D-5L, five-level EuroQol five-dimensional questionnaire; QLQ-PBM, Quality of Life Questionnaire Preference- Based Measure. between the QLQ-PBM and the EORTC-8D, larger ceiling effects were reported on the items of the EORTC-8D. Overall, 30% of the respondents reported full health on the EQ-5D-5L compared with 14% and 16% for the QLQ-PBM and the EORTC-8D, respectively. Of the patients in full health according to the EQ-5D-5L, 50% to 91% reported less than perfect health on the other quality-of-life scales (Table 3); the actual percentage depended on the scale used. These patients most frequently reported fatigue and problems with physical functioning on the QLQ-PBM and the EORTC- 8D. None of the patients reported the worst imaginable health according to the three questionnaires, but some patients reported the worst score for individual items. The worst score was more frequently reported on items of the QLQ-PBM and the EORTC-8D than on items of the EQ-5D-5L. Validity Insight into the content of the different questionnaires showed a large overlap between the three questionnaires. The QLQ-PBM and the EORTC-8D included similar domains of quality of life except for cognitive functioning, which was included only in the QLQ-PBM, and constipation and diarrhea, which was incorporated only in the EORTC-8D. Furthermore, the operationalization of physical functioning, role functioning, emotional functioning, and pain differed between the questionnaires (Table 1). No conclusion can be drawn with respect to the question which of the two disease-specific preference-based questionnaire had the largest overlap with the EQ-5D-5L. For some items (cognitive functioning and role functioning), the domains between the
838 VALUE IN HEALTH 19 (2016) 834 843 Table 3 Quality of life for patients with perfect health according to the EQ-5D-5L (ceiling effect) (N ¼ 32). Quality of life scale No perfect health, N (%) Mean SD Median (range) QLQ-PBM utility score 17 (55) 0.94 0.06 0.93 (0.84 1.00) EORTC-8D utility score 17 (55) 0.94 0.06 0.95 (0.78 1.00) EQ-VAS 29 (91) 86.0 13.0 90.0 (35.0 100) Global quality-of-life scale (QL scale) 16 (50) 91.4 10.5 95.8 (58.3 100) Disease-specific quality-of-life domains Physical functioning QLQ-PBM 10 (31) EORTC-8D 10 (31) Role functioning QLQ-PBM 2 (6) EORTC-8D 3 (9) Social functioning (QLQ-PBM and EORTC-8D) 6 (19) Emotional functioning QLQ-PBM 5 (16) EORTC-8D 1 (3) Cognitive functioning (QLQ-PBM) 7 (23) Pain QLQ-PBM 1 (3) EORTC-8D 0 (0) Nausea (QLQ-PBM and EORTC-8D) 3 (10) Fatigue (QLQ-PBM and EORTC-8D) 13 (42) Constipation and/or diarrhea (EORTC-8D) 2 (6) EORTC-8D, European Organization of Randomized Controlled Trials 8 Dimension; EQ-5D-5L, five-level EuroQol five-dimensional questionnaire; EQ-VAS, EQ-visual analogue scale; QLQ-PBM, Quality of Life Questionnaire Preference-Based Measure. QLQ-PBM and the EQ-5D-5L had stronger correlations, whereas for others the correlations were stronger between the EORTC-8D and the EQ-5D-5L (physical functioning, pain, and emotional functioning) (Table 4). The disease-specific domains nausea and constipation and/or diarrhea had no substantial overlap with any of the domains of the EQ-5D-5L. It is expected that the weak to moderate correlations for these domains were partly caused by the small variance in responses. Two other disease-specific domains (fatigue and cognitive functioning) were strongly correlated with one of the EQ-5D-5L domains, but not with the utility score. The comparison of the average utility from the EQ-5D-5L with the utility from the disease-specific preference-based questionnaire differed for the UK and Dutch tariffs. The average Dutch utility of the EQ-5D-5L was significantly lower than the average utility of the QLQ-PBM (0.83 and 0.85, respectively, P ¼ 0.005), whereas the average UK utility of the EQ-5D-5L was significantly higher than the utility score of the EORTC-8D (Table 5). Subgroup analyses for patients with different health statuses showed a significant difference between the EQ-5D-5L and the diseasespecific preference-based questionnaires for only a few subgroups, especially in patients in poor health. As expected in advance, the utility scores of all three questionnaires were strongly correlated with the EQ-VAS and the QL scale (Table 4). Furthermore, all questionnaires were able to distinguish between patients with different health statuses, with higher utilities for poorer health status (Fig. 2 and Table 5). However, full health on the EQ-5D-5L was reported in all health status categories, whereas lower utility scores were reported by the QLQ-PBM and the EORTC-8D (Fig. 2). Although absolute differences in utilities between health status categories were smaller for the disease-specific preference-based questionnaires compared with the EQ-5D-5L, the ESs were larger for the QLQ- PBM and the EORTC-8D. Figure 2 also indicates that the QLQ-PBM and the EORTC-8D are better able to discriminate between different health status categories in the utility estimation. Nevertheless, it seems that the discriminative power of the EQ-5D-5L is sufficient because all ESs were categorized as moderate to strong ES (40.5), except for the ESs of detecting differences between the second and third quartiles of the EQ-VAS. Discussion This study showed some evidence for the validity and interpretability of the EQ-5D-5L, the QLQ-PBM, and the EORTC-8D in patients with acute leukemia. Insight into the content and the performance of the questionnaire in patients with acute leukemia is useful to select the most appropriate questionnaire to measure quality-of-life utilities in patients with acute leukemia. An important difference in descriptive content between the EQ-5D-5L, the QLQ-PBM, and the EORTC-8D is the focus of the questionnaires. Because the QLQ-PBM and the EORTC-8D are specifically focused on patients with cancer, more cancerrelevant domains are included such as fatigue and cognitive functioning. The lack of these domains in the EQ-5D is a wellknown problem [5,36] and research is ongoing to evaluate whether and how the EQ-5D can be extended with these and other relevant domains [15,37 41]. From a clinical perspective, the QLQ-PBM and the EORTC-8D might be more relevant because these indicate disease-specific problems of the quality of life. However, the question is whether the higher level of detail is required to identify relevant quality-of-life differences for use in economic evaluations. All three questionnaires had good interpretability with acceptably low percentage of missing data. The only concern about the interpretability is the high ceiling effect of especially the EQ-5D- 5L. In total, 30% of the respondents reported full health on the EQ-5D-5L compared with 14% and 16% for the QLQ-PBM and the EORTC-8D, respectively. The high ceiling effect of the EQ-5D-5L might imply that the EQ-5D-5L is not able to detect all relevant differences in health because 50% of the respondents with perfect health according to the EQ-5D-5L reported problems on at least one disease-specific domain. However, it can also be argued that
Table 4 Correlation coefficients between the quality-of-life instruments. QLQ-PBM Questionnaire EQ-5D-5L EQ- VAS Mobility Selfcare Daily Pain/ Anxiety/ Utility score Utility score activities discomfort depressed (NL) (UK) QL scale Physical functioning QLQ-PBM 0.47 0.17 0.47 0.41 0.37 0.47 0.47 0.60 * 0.60 * EORTC-8D 0.75 * 0.34 0.56 * 0.62 * 0.32 0.65 * 0.67 * 0.63 * 0.65 * Role functioning QLQ-PBM 0.70 * 0.43 0.74 * 0.67 * 0.46 0.76 * 0.77 * 0.68 * 0.67 * Cognitive functioning Emotional functioning Social functioning EORTC-8D 0.63 * 0.41 0.69 * 0.55 * 0.39 0.67 * 0.67 * 0.71 * 0.69 * QLQ-PBM 0.29 0.14 0.43 0.24 0.55 * 0.41 0.40 0.34 0.35 QLQ-PBM 0.28 0.14 0.56 * 0.30 0.58 * 0.42 0.43 0.55 * 0.62 * EORTC-8D 0.40 0.10 0.47 0.31 0.80 * 0.52 * 0.51 * 0.33 0.47 QLQ-PBM þ 0.53 * 0.33 0.64 * 0.49 0.40 0.60 * 0.62 * 0.49 0.56 * EORTC-8D Pain QLQ-PBM 0.64 * 0.30 0.60 * 0.78 * 0.42 0.72 * 0.74 * 0.53 * 0.49 EORTC-8D 0.69 * 0.41 0.69 * 0.75 * 0.43 0.81 * 0.81 * 0.55 * 0.55 * Nausea QLQ-PBM þ 0.15 0.16 0.18 0.07 0.26 0.15 0.16 0.24 0.31 EORTC-8D Fatigue QLQ-PBM þ 0.39 0.09 0.50 * 0.41 0.30 0.43 0.42 0.59 * 0.64 * EORTC-8D Constipaton and EORTC-8D 0.09 0.11 0.23 0.07 0.20 0.17 0.15 0.18 0.21 diarrhea Utility score QLQ-PBM 0.70 * 0.38 0.80 * 0.71 * 0.58 * 0.83 * 0.82 * 0.74 * 0.74 * EORTC-8D 0.68 * 0.39 0.74 * 0.67 * 0.55 * 0.77 * 0.77 * 0.72 * 0.78 * EQ-VAS 0.55 * 0.31 0.65 * 0.53 * 0.41 0.61 * 0.61 QL scale 0.54 * 0.37 0.65 * 0.55 * 0.49 0.65 * 0.65 * EORTC-8D, European Organization of Randomized Controlled Trials 8 Dimension; EQ-5D-5L, five-level EuroQol five-dimensional questionnaire; EQ-VAS, EQ-visual analogue scale; NL, the Netherlands; QL, global quality-of-life scale of the QLQ-C30; QLQ-C30, Quality of Life Questionnaire for Cancer; QLQ-PBM, Quality of Life Questionnaire Preference-Based Measure. * These correlations indicate strong correlations (i.e., correlations 4 0.5 ). The correlations between domain scores and overall quality of life are negative because a higher domain score indicates a worse quality of life. VALUE IN HEALTH 19 (2016) 834 843 839
840 Table 5 Utility scores for the different questionnaires, including the known subgroup analysis. EQ-5D-5L QLQ-PBM (Dutch tariff) EQ-5D-5L (UK tariff) EORTC-8D P value * P value * P value * P value * All patients Mean score SD 0.81 0.22 0.85 0.11 0.85 0.18 0.82 0.15 Median score (range) 0.87 (0.07 1) 0.87 (0.51 1) 0.90 (0.18 1) 0.86 (0.47 1) Severity by EQ-VAS, mean SD EQ-VAS Q1 (EQ-VAS: 0 65) 0.64 0.26 o0.0001 0.76 0.10 o0.0001 0.71 0.23 o0.0001 0.67 0.11 o0.0001 EQ-VAS Q2 (EQ-VAS: 65 80) 0.79 0.18 0.83 0.06 0.84 0.15 0.78 0.11 EQ-VAS Q3 (EQ-VAS: 81 90) 0.84 0.17 0.86 0.08 0.85 0.15 0.85 0.14 EQ-VAS Q4 (EQ-VAS: 92 100) 0.96 0.06 0.95 0.05 0.98 0.03 0.97 0.05 ES EQ-VAS Q2 vs EQ-VAS Q1 (95% CI) 0.67 (0.09 to 1.22) 0.92 (0.33 to 1.49) 0.69 (0.11 to 1.24) 1.04 (0.44 to 1.61) EQ-VAS Q3 vs EQ-VAS Q2 (95% CI) 0.28 ( 0.28 to 0.84) 0.39 ( 0.17 to 0.95) 0.25 ( 0.31 to 0.81) 0.53 ( 0.03 to 1.08) EQ-VAS Q4 vs EQ-VAS Q3 (95% CI) 0.96 (0.38 to 1.51) 1.28 (0.68 to 1.85) 0.97 (0.39 to 1.52) 1.12 (0.53 to 1.68) Severity by QL, mean SD QL Q1 (QL: 0.0 66.6) 0.65 0.26 o0.0001 0.76 0.11 o0.001 0.72 0.21 o0.0001 0.66 0.11 o0.0001 QL Q2 (QL: 66.7 83.3) 0.76 0.17 0.82 0.06 0.80 0.16 0.76 0.12 QL Q3 (QL: 83.4 91.6) 0.85 0.19 0.88 0.09 0.89 0.16 0.88 0.11 QL Q4 (QL: 91.7 100.0) 0.97 0.05 0.95 0.05 0.98 0.03 0.96 0.04 ES QL Q2 vs QL Q1 (95% CI) 0.52 ( 0.05 to 1.08) 0.66 (0.08 to 1.23) 0.47 ( 0.10 to 1.02) 0.86 (0.27 to 1.42) QL Q3 vs QL Q2 (95% CI) 0.52 ( 0.04 to 1.07) 0.79 (0.21 to 1.34) 0.57 (0.00 to 1.12) 1.00 (0.42 to 1.55) QL Q4 vs QL Q3 (95% CI) 0.80 (0.24 to 1.34) 0.99 (0.42 to 1.54) 0.78 (0.22 to 1.32) 1.08 (0.50 to 1.63) VALUE IN HEALTH 19 (2016) 834 843 Severity by ability to work, mean SD Not able to work 0.72 0.24 0.0008 0.81 0.11 0.0003 0.77 0.20 0.0004 0.73 0.15 o0.0001 Able to work 0.88 0.17 0.89 0.09 0.92 0.13 0.88 0.12 ES (95% CI) 0.79 (0.34 to 1.24) 0.84 (0.38 to 1.29) 0.86 (0.40 to 1.30) 1.15 (0.67 to 1.60) EORTC-8D, European Organization of Randomized Controlled Trials 8 Dimension; EQ-5D-5L, five-level EuroQol five-dimensional questionnaire; EQ-VAS, EQ-visual analogue scale; ES, effect size; QL, global quality-of-life scale of the QLQ-C30; QLQ-C30, Quality of Life Questionnaire for Cancer; QLQ-PBM, Quality of Life Questionnaire Preference-Based Measure. * Comparison of utility scores between severity subgroups. Significantly different from the QLQ-PBM utility score (α ¼ 0.05). Significantly different from the EORTC-8D utility score (α ¼ 0.05). Restricted to patients aged o65 y.
VALUE IN HEALTH 19 (2016) 834 843 841 Fig. 2. Distribution of utility scores per health status category. (A) EQ-5D-5L utilities (Dutch tariff) in the EQ-VAS quartiles. (B) QLQ-PBM utilities in the EQ-VAS quartiles. (C) EQ-5D-5L utilities (UK tariff) in the EQ-VAS quartiles. (D) EORTC-8D utilities in the EQ-VAS quartiles. (E) EQ-5D-5L utilities (Dutch tariff) in the global quality-of-life scale (QL) quartiles. (F) QLQ-PBM utilities in the QL quartiles. (G) EQ-5D-5L utilities (UK tariff) in the QL quartiles. (H) EORTC-8D utilities in the QL quartiles. (I) EQ-5D-5L utilities (Dutch tariff) according to ability to work. (J) QLQ-PBM utilities according to ability to work. (K) EQ-5D-5L utilities (UK tariff) according to ability to work. (L) EORTC-8D utilities according to ability to work. EORTC-8D, European Organization of Randomized Controlled Trials 8 Dimension; EQ-5D-5L, five-level EuroQol five-dimensional questionnaire; EQ-VAS, EQ-visual analogue scale; QLQ-PBM, Quality of Life Questionnaire Preference-Based Measure. the explicit focus on specific problems in the disease-specific questionnaires causes a focusing effect and thereby exaggerates the severity of these problems [42]. The focusing effect might occur in this study population because the population is relatively healthy. The lowest quartiles of both the EQ-VAS and the QL scale included scores up to 65 (on a 0 100 scale). Despite the relatively good health and the small differences in EQ-VAS and QL scores between the quartiles, the EQ-5D-5L was able to able to discriminate between patients with different health statuses as indicated by moderate to strong ES. Therefore, it is concluded that the high ceiling effect of the EQ-5D-5L is not problematic in this study population. Nevertheless, it should be noted that the discriminative power was larger for the two disease-specific questionnaires, especially for the EORTC-8D. The difference in discriminative power between the two diseasespecific questionnaires may be due to the operationalization of the physical functioning domain. The EORTC-8D incorporated five different levels of functioning, whereas only two levels were incorporated in the QLQ-PBM. The relatively good health seen among the patients in this study is probably a consequence of the selection of acute leukemia survivors [9]. It is assumed that survivors are relatively healthy compared with nonsurvivors because survivors were able to endure both the disease and its intensive treatment. Furthermore, survivors could have adapted themselves to their new situation. Consequently, they would report relatively good health because of changes in their standards and values (response shift) [43,44]. The response shift might differ between the EQ-5D-5L, the QLQ-PBM, and the EORTC-8D because of differences in the formulation of items. It is expected that a larger response shift would result in higher ceiling effects. Although this study provides some useful insight into some aspects of the validity of the EQ-5D-5L and two disease-specific preference-based questionnaires, it also has some limitations. First, because this study was not initially designed for validation purposes, we have not been able to assess all psychometric properties of the preference-based questionnaires. Patients completed the questionnaires only once, making it impossible to evaluate the responsiveness and reliability of the questionnaires. Another missing component is an explicit evaluation of the face validity of the questionnaires for acute leukemia. Furthermore, health status was defined only according to self-reported health
842 VALUE IN HEALTH 19 (2016) 834 843 status and ability to work. Although these subjective measures provide useful information about the construct validity of the questionnaires, these measures are not perfect in defining different levels of health status. It is worthwhile to assess the ability to distinguish between health statuses according to more objective measures, such as physician-reported Eastern Cooperative Oncology Group (ECOG) performance status or relapse versus remission. Another limitation of this study is a possible selection bias because the questionnaires were distributed only among acute leukemia survivors who previously participated in a clinical trial. Consequently, we may have selected a relatively healthy patient population who is open to participate in research, which limits the generalizability of our findings to a broader patient population. However, because one of the major concerns of the validity of the EQ-5D is the high ceiling effect, we expect that the discriminative power of the EQ-5D-5L would be better only in a population with more health problems. Furthermore, we have no reasons to assume that the willingness to participate in research will impact the validity of quality-of-life questionnaires. Quality of life depends mainly on the underlying disease and the intensity of the treatment and not on participation in clinical trials. In addition, because the treatment of acute leukemia is concentrated in specialized centers in the Netherlands, we assume that treatment and the resulting quality-of-life outcomes are more or less similar within and outside clinical trials. This study was limited to an assessment of the validity and interpretability of the EQ-5D-5L and two disease-specific preference-based questionnaires. In addition to disease-specific preference-based questionnaires, it is possible to estimate quality-oflife utilities from disease-specific questionnaires by means of mapping. Because it has been shown that mapping algorithms do not incorporate items that are not adequately captured by the EQ-5D [45], these algorithms are not considered a valid alternative in case the EQ-5D has proven to be invalid. Because of this reason, this study did not evaluate the validity of different mapping algorithms. According to the findings of this study, we would still recommend to use the EQ-5D-5L for measuring quality-of-life utilities in patients with acute leukemia. No problems with the validity of the EQ-5D-5L have been identified. The EQ-5D-5L was strongly correlated with other quality-of-life measures and it was able to distinguish between patients with different health states. However, other aspects of the validity and responsiveness and reliability need to be evaluated in a broader patient population to draw any definite conclusion about its appropriateness for measuring quality-of-life utilities in acute leukemia. Acknowledgments We thank Bronno van der Holt from the HOVON Data Center, Erasmus MC, and Bob Löwenberg from the Erasmus MC for providing clinical data of the patients who participated in this quality-of-life study. We thank Sarah Lonergan from the Erasmus MC for her help regarding the logistics of this study and Marjolein Schouten for her help with the data management. Furthermore, we thank Sander Arons for his valuable feedback on a draft version of this article at a LOLAHESG conference. Source of financial support: This work was supported by a grant from the Center for Translational Molecular Medicine (CTMM), project BioCHIP (grant no. 03O-102). The article has been approved for publication by the CTMM. REFERENCES [1] Brazier JE, Rowen D, Mavranezouli I, et al. Developing and testing methods for deriving preference-based measures of health from condition-specific measures (and other patient-based measures of outcome). Health Technol Assess 2012;16:1 114. [2] Pickard AS, Ray S, Ganguli A, Cella D. Comparison of FACT- and EQ-5Dbased utility scores in cancer. Value Health 2012;15:305 11. [3] Lamers LM, Bouwmans CAM, van Straten A, et al. Comparison of EQ-5D and SF-6D utilities in mental health patients. Health Econ 2006;15:1229 36. [4] Versteegh MM, Leunis A, Uyl-de Groot CA, Stolk EA. Condition-specific preference-based measures: benefit or burden? Value Health 2012;15:504 13. [5] Rowen D, Young T, Brazier J, Gaugris S. Comparison of generic, condition-specific, and mapped health state utility values for multiple myeloma cancer. Value Health 2012;15:1059 68. [6] Brazier J, Roberts J, Tsuchiya A, Busschbach J. A comparison of the EQ-5D and SF-6D across seven patient groups. Health Econ 2004;13:873 84. [7] Kontodimopoulos N, Pappa E, Papadopoulos A, et al. Comparing SF-6D and EQ-5D utilities across groups differing in health status. Qual Life Res 2009;18:87 97. [8] National Institute for Health and Care Excellence. Guide to the methods of technology appraisal 2013. 2013. Available from: http://publications. nice.org.uk/pmg9. [Accessed January 30, 2015]. [9] Leunis A, Ken Redekop W, Uyl-de Groot CA, Lowenberg B. Impaired health-related quality of life in acute myeloid leukemia survivors: a single-center study. Eur J Haematol 2014;93:198 206. [10] Rowen D, Brazier J, Young T, et al. Deriving a preference-based measure for cancer using the EORTC QLQ-C30. Value Health 2011;14:721 31. [11] Conner-Spady B, Cumming C, Nabholtz JM, Jacobs P, Stewart D. Responsiveness of the EuroQol in breast cancer patients undergoing high dose chemotherapy. Quality of life research : an international journal of quality of life aspects of treatment, care and rehabilitation 2001;10(6):479 86. [12] Kim S, Hwang J, Kim T, Hong Y, Jo M. Validity and reliability of the EQ- 5D for cancer patients in Korea. Supportive Care in Cancer 2012;20 (12):3155 60. [13] Krabbe PF, Peerenboom L, Langenhoff BS, Ruers TJ. Responsiveness of the generic EQ-5D summary measure compared to the disease-specificeortc QLQ C-30. Quality of life research : an international journal of quality of life aspects of treatment, care and rehabilitation 2004;13(7):1247 53. [14] Kvam AK, Fayers PM, Wisloff F. Responsiveness and minimal important score differences in quality-of-life questionnaires: a comparison of the EORTC QLQ-C30 cancer-specific questionnaire to the generic utility questionnaires EQ-5D and 15D in patients with multiple myeloma. European journal of haematology 2011;87(4):330 7. [15] Longworth L, Yang Y, Young T, Mulhern B, Hernandez Alava M, Mukuria C. Use of generic and condition-specific measures of healthrelated quality of life in NICE decision-making: a systematic review, statistical modelling and survey. Health technology assessment (Winchester, England) 2014;18(9):1 224. [16] Pickard AS, De Leon MC, Kohlmann T, Cella D, Rosenbloom S, et al., Psychometric comparison of the standard EQ-5D to a 5 level version in cancer patients. Medical care 2007;45(3):259 63. [17] Teckle P, Peacock S, McTaggart-Cowan H, van-der-hoek K, Chia S, Melosky B, et al., The ability of cancer-specific and generic preferencebased instruments to discriminate across clinical and self-reported measures of cancer severities. Health and quality of life outcomes 2011;9:106 7525-9-106. [18] Herdman M, Gudex C, Lloyd A, et al. Development and preliminary testing of the new five-level version of EQ-5D (EQ-5D-5L). Qual Life Res 2011;20:1727 36. [19] Janssen MF, Pickard AS, Golicki D, et al. Measurement properties of the EQ-5D-5L compared to the EQ-5D-3L across eight patient groups: a multi-country study. Qual Life Res 2012;22:1 11. [20] Kim S, Kim H, Lee S, Jo M. Comparing the psychometric properties of the EQ-5D-3L and EQ-5D-5L in cancer patients in Korea. Qual Life Res 2012;21:1065 73. [21] Scalone L, Ciampichini R, Fagiuoli S, et al. Comparing the performance of the standard EQ-5D 3L with the new version EQ-5D 5L in patients with chronic hepatic diseases. Qual Life Res 2012;22:1 10. [22] Fitzsimmons D, Gilbert J, Howse F, et al. A systematic review of the use and validation of health-related quality of life instruments in older cancer patients. Eur J Cancer 2009;45:19 32. [23] Luckett T, King MT, Butow PN, et al. Choosing between the EORTC QLQ- C30 and FACT-G for measuring health-related quality of life in cancer clinical research: issues, evidence and recommendations. Ann Oncol 2011;22:2179 90. [24] Osborne TR, Ramsenthaler C, Siegert RJ, et al. What issues matter most to people with multiple myeloma and how well are we measuring them? A systematic review of quality of life tools. Eur J Haematol 2012;89:437 57.
VALUE IN HEALTH 19 (2016) 834 843 843 [25] Löwenberg B, van Putten W, Theobald M, et al. Effect of priming with granulocyte colony-stimulating factor on the outcome of chemotherapy for acute myeloid leukemia. N Engl J Med 2003;349:743 52. [26] Löwenberg B, Pabst T, Vellenga E, et al. Cytarabine dose for acute myeloid leukemia. N Engl J Med 2011;364:1027 36. [27] Löwenberg B, Ossenkoppele GJ, van Putten W, et al. High-dose daunorubicin in older patients with acute myeloid leukemia. N Engl J Med 2009;361:1235 48. [28] Ossenkoppele GJ, Stussi G, Maertens J, et al. Addition of bevacizumab to chemotherapy in acute myeloid leukemia at older age: a randomized phase 2 trial of the Dutch-Belgian Cooperative Trial Group for Hematooncology (HOVON) and the Swiss Group for Clinical Cancer Research (SAKK). Blood 2012;120:4706 11. [29] HOVON the Haemato Oncology Foundation for Adults in the Netherlands. Available from: http://www.hovon.nl/trials/trials-by-type/aml.html. [Accessed April 29, 2013]. [30] Brooks R. EuroQol: the current state of play. Health Policy 1996;37:53 72. [31] Devlin N, Shah K, Feng Y, et al. Valuing health-related quality of life: an EQ-5D-5L value set for England. No. 16/01. Available at: https://www. ohe.org/publications/valuing-health-related-quality-life-eq-5d-5lvalue-set-england. [Accessed: July 1, 2016]. [32] Versteegh MM, Vermeulen KM, Evers SMAA, et al. Dutch tariff for the 5 level version of EQ-5D. Value Health 2016;19:343 52. [33] Aaronson NK, Ahmedzai S, Bergman B, et al. The European Organization for Research and Treatment of Cancer QLQ-C30: a qualityof-life instrument for use in international clinical trials in oncology. J Natl Cancer Inst 1993;85:365 76. [34] King MT. The interpretation of scores from the EORTC quality of life questionnaire QLQ-C30. Qual Life Res 1996;5:555 67. [35] Mokkink LB, Terwee CB, Patrick DL, et al. The COSMIN study reached international consensus on taxonomy, terminology, and definitions of measurement properties for health-related patient-reported outcomes. J Clin Epidemiol 2010;63:737 45. [36] Cohen J. Statistical Power Analysis for the Behavioral Sciences (2nd ed.). New Jersey: Erlbaum, New Jersey, 1988. [37] Garau M, Shah K, Mason A, et al. Using QALYs in cancer. Pharmacoeconomics 2011;29:673 85. [38] Yang Y, Rowen D, Brazier J, et al. An exploratory study to test the impact on three bolt-on items to the EQ-5D. Value Health 2015;18:52 60. [39] Swinburn P, Lloyd A, Boye KS, et al. Development of a disease-specific version of the EQ-5D-5L for use in patients suffering from psoriasis: lessons learned from a feasibility study in the UK. Value Health 2013;16:1156 62. [40] Yang Y, Brazier J, Tsuchiya A. Effect of adding a sleep dimension to the EQ-5D descriptive system: a Bolt-on experiment. Med Decis Making 2014;34:42 53. [41] Krabbe PFM, Stouthard MEA, Essink-Bot M, Bonsel GJ. The effect of adding a cognitive dimension to the EuroQol multiattribute healthstatus classification system. J Clin Epidemiol 1999;52:293 301. [42] Brazier J, Tsuchiya A. Preference-based condition-specific measures of health: what happens to cross programme comparability? Health Econ 2010;19:125 9. [43] Rapkin B, Schwartz C. Toward a theoretical model of quality-of-life appraisal: implications of findings from studies of response shift. Health Qual Life Outcomes 2004;2:14. [44] Sprangers MA, Schwartz CE. Integrating response shift into healthrelated quality of life research: a theoretical model. Soc Sci Med 1999;48:1507 15. [45] Versteegh MM, Leunis A, Luime JJ, et al. Mapping QLQ-C30, HAQ, and MSIS-29 on EQ-5D. Med Decis Making 2012;32:554 68.