Psychometric Validation of the Cancer Therapy Satisfaction Questionnaire

Transcription

1 Volume 11 Number VALUE IN HEALTH Psychometric Validation of the Cancer Therapy Satisfaction Questionnaire Peter C. Trask, PhD, MPH, 1 Crystal Tellefsen, BA, 2 Derek Espindle, MA, 3 Christine Getter, 1 Ming-Ann Hsu, MPH 1 1 Pfizer Inc., New London, CT, USA; 2 Mapi Values, Boston, MA, USA; 3 Private Consultant, Beverly, MA, USA ABSTRACT Purpose: To assess the domain structure and to evaluate the psychometric properties of the Cancer Therapy Satisfaction Questionnaire (CTSQ) and the relation between the CTSQ and health-related quality of life (HRQOL). Methods: Three hundred sixty-one individuals with breast, colorectal, lung cancer, or melanoma who had received in the last 6 months or were currently receiving more than one cycle of chemo, biological, or hormonal therapy completed the 21-item CTSQ, with a random subsample of 88 patients completing it again 1 week later. Participants also completed quality of life, treatment satisfaction, and other self-reported questions on each occasion. Demographics, ECOG performance status, and clinical information were collected. CTSQ responses were submitted to multitrait analyses and exploratory factor analysis. Psychometric properties and the correlations between the CTSQ and the Quality of Life Questionnaire Core 30 (QLQ-C30) were evaluated. Results: Analyses revealed three domains with good psychometric properties: Feelings about Side Effects, Satisfaction with Therapy, and Expectations of Therapy. Correlations with the QLQ-C30 domains were low to moderate. Conclusion: The CTSQ is a newly developed 16-item patient-reported measure with strong psychometric properties and constructs not captured by the QLQ-C30. It can be used to evaluate cancer patients experiences with intravenous and/or oral chemo, biological, and hormonal therapies. Keywords: cancer, psychometric validation, treatment satisfaction. Introduction The use of chemo, biological, and hormonal therapies is associated with a variety of side effects that have the potential to adversely impact health-related quality of life (HRQOL) [1 6]. Additionally, the act of administering these drugs can disrupt the day-to-day routines that were established before the cancer diagnosis. Although numerous studies have documented the impact that these treatments have on patient-reported outcomes (PROs) and specifically HRQOL, few studies have assessed cancer patients satisfaction with therapy. Patient satisfaction with treatment, which is closely linked to treatment decision-making and compliance, might reveal beneficial information about the patient s experience and constitute a compelling outcome that needs to be taken into account. The Cancer Therapy Satisfaction Questionnaire (CTSQ) was recently created to measure treatment satisfaction in individuals with cancer [7]. The CTSQ was developed based on extensive item-elicitation Address correspondence to: Peter C. Trask, WW Outcomes Research, Pfizer Inc., MS 6025-A3221, 50 Pequot Avenue, New London, CT 06320, USA. Peter.C.Trask@Pfizer.com /j x interviews and focus groups with cancer patients, clinicians, and nurses in the United States, United Kingdom, and France. The CTSQ was developed for use in a wide range of cancer types and stages and was specific to patients receiving cancer therapy. In particular, the instrument was intended for use in studies where patients are taking intravenous (IV) and/or oral anticancer therapies. The face and content validity of the CTSQ was assessed through cognitive-debriefing interviews with adult patients with cancer. The initial CTSQ was multidimensional in its assessment of satisfaction with cancer therapy, being composed of six hypothesized domains (Expectations of Therapy [ET], Feelings about Side Effects [FSE], Oral Therapy Compliance [OTC], Convenience [CON], Satisfaction with Therapy [SWT], and Stopping Therapy [ST]). In addition, one item used a check-all-that-apply format to assess reasons for noncompliance among patients taking oral therapy reporting that they do not always take therapy as directed [7]. To determine whether the CTSQ could be a useful measurement tool for research, a psychometric validation study was required. This article documents that study as a follow-up to the original manuscript. The objectives of the current study were: 1) to assess the 2008, International Society for Pharmacoeconomics and Outcomes Research (ISPOR) /08/

2 670 Trask et al. domain structure and evaluate the psychometric properties of the CTSQ; and 2) to evaluate the strength of association between the CTSQ domains and HRQOL as measured by the European Organization for Research and Treatment of Cancer Quality of Life Questionnaire Core 30 (EORTC QLQ-C30). Methods Patients and Procedure A total of 361 participants, across a range of cancer types, were recruited for the study from investigators practicing in 14 community clinical practices throughout the United States (average number of patients/ site = 26, range 1 43). A subsample of 88 participants completed follow-up questionnaires 1 week after their baseline assessment. Patients were eligible if they: provided written informed consent, aged 18 years or older, could read and write in English, were available for follow-up evaluation, and were actively receiving more than one cycle of first- or second-line chemo, biological, or hormonal therapy for early (Stage I or II) or advanced (Stage II or IV) melanoma, breast, colorectal, or lung cancer. Patients were excluded if, in the opinion of the investigator, they did not have the mental or physical capacity to participate in the trial, were currently undergoing third-line therapy for any tumor type, or were treatment-free for 6 months or more. The study was conducted as a 1-week prospective study. Study participants were randomly assigned to complete either the baseline assessment only or both the baseline and follow-up assessments on a 4:1 ratio. All consenting patients were included in the initial visit with current clinical status noted. Participants randomized for follow-up visits were provided with a mailing packet containing a set of questionnaires to be returned to the investigator in 7 days. Participants withdrawing consent before the completion of the study were replaced based on their randomization assignment. Sample Size The sample size for the baseline assessment was driven by the need to perform multivariate analyses involving many questionnaire items, including multitrait and exploratory factor analysis. The baseline sample size of 361 subjects for a questionnaire containing 21 items (more than 17 subjects per item) meets or exceeds most recommendations, and allows for some sample size reduction because of missing item data [8 11]. In addition, the sample size of 85 patients that completed the follow-up assessment and the stable subsample of these patients (as discussed below) exceeds the sample size of at least 50 that has been recommended when evaluating test retest and/or interrater reliability [12]. Patient-Completed Questionnaires In addition to providing demographic and background information, study participants also completed the following measures. CTSQ. Of the prevalidation version of the CTSQ (21 items) [7], 19 items were spread across five tentative multi-item domains: ET (6 items), FSE (2 items), OTC (2 items), CON (3 items), and SWT (6 items). In addition, there was one tentative single-item domain (ST) and one check-all-that-apply item (Reasons for Noncompliance with Oral Therapy). All items were scored on a 1 5 scale, with a value of 1 associated with the worst response and a value of 5 representing the best response. Six of the items were reverse-coded. For each domain, if the minimum item completion standard was met (as described in the Results section), the domain was scored using the formula: (mean of completed item scores - 1) 25; otherwise, a missing value was assigned. This linear transformation resulted in a score ranging from 0 to 100 for each domain, with a higher score associated with the best outcome on each domain. EORTC QLQ-C30 version 3.0. The core module EORTC QLQ-C30 is a cancer-specific HRQOL instrument with demonstrated psychometric properties [13]. It consists of 30 items and incorporates five functional scales (physical, role, cognitive, emotional, and social), a global health status/quality-of-life scale, three multi-item symptom scales (fatigue, pain, nausea, and vomiting), and a number of single items assessing additional symptoms and difficulties commonly reported by cancer patients (dyspnea, loss of appetite, insomnia, constipation, and diarrhea), and perceived financial impact of the disease. Each of the QLQ-C30 domains is scored on a scale, with higher scores on the functional scales being indicative of better HRQOL, whereas higher scores on the symptom scales are reflective of worse symptoms. Treatment Satisfaction Questionnaire for Medication (TSQM) version 1.4. The TSQM is a generic PRO instrument developed to assess patient satisfaction with therapy [14]. It consists of 14 items and assesses effectiveness, side effects, convenience, and global satisfaction. The TSQM is a psychometrically sound, validated measure scored on a scale with a higher score associated with a better outcome. This questionnaire s psychometric properties, however, have not been validated in cancer subjects. Additional questions. Additional questions assessed perceived improvement or worsening in general health, cancer, and satisfaction with cancer therapy over the

3 Psychometric Validation of the CTSQ 671 past 2 weeks. Questions also assessed whether or not a change in cancer therapy occurred, whether or not a hospitalization or visit to the emergency room occurred (due to cancer, a side effect of cancer therapy, or for some other reasons) during the same time frame, and whether and to what degree subjects received assistance from others when completing the assessments. Investigator-Completed Questionnaires The following information was completed by the investigator: diagnosis (type and stage of cancer), current line of therapy, medical history, anticancer therapies received in the last 3 months, hospitalizations in the last 3 months due to cancer-related events, and ECOG performance status. Investigators also provided information on the patients current anticancer drug treatment regimen (including those used in the past 3 months). Statistical Analyses Multitrait analysis was first performed on the hypothesized CTSQ domains from the 21-item questionnaire to evaluate the item-convergent and -discriminant validity [11,15]. Multitrait analysis takes a confirmatory approach to evaluating the hypothesized domain structure of an instrument, and involves examining the matrix of correlations between each item and each domain, with each item s correlation with its own hypothesized domain corrected for overlap. In a multitrait analysis, item-convergent validity is supported if items have correlations = 0.40 with their own hypothesized domain, and item-discriminant validity is considered to be supported if items correlate more highly with their own hypothesized domain than they do with other ( competing ) domains in the instrument. If convergent validity was not supported, exploratory factor analysis was performed using maximumlikelihood extraction and oblique promax rotation to determine the most psychometrically sound domains. Internal consistency reliability was evaluated using Cronbach s coefficient alpha [16], and test retest reliability was evaluated using the intraclass correlation coefficient (ICC) [17]. For both Cronbach s alpha and the ICC, a value of 0.70 or greater is considered desirable for measures intended for use in group-level research [18,19]. Known-groups validity comparisons of group means were conducted using one-way ANOVA after square-root transformation to improve the normality of the dependent variable distributions. Based on Cohen s d [20], effect sizes for group differences were calculated and interpreted using Cohen s guidelines (small effect = 0.2; medium effect = 0.5; large effect = 0.8). Estimates of the minimally important difference (MID) for each of the CTSQ domains were calculated using the approach of 0.5 SD [21] and 1 standard error of measure (SEM) [22 24]. In addition, betweengroup differences for appropriate known-groups validity criteria were examined to provide useful interpretive information about the meaningfulness of differences of various sizes. Except for test retest reliability analyses, analyses were conducted using baseline data. Unless otherwise noted, correlational analyses used Pearson correlation coefficients. Where specific significance tests were used, the threshold for statistical significance was P < 0.05 for each test. SAS software version for Windows (SAS Institute Inc., Cary, NC, USA) was used for all analyses except for multitrait analysis. The Multitrait Multiitem Analysis Program Revised (MAP-R) software [11] running on top of SAS version 6.12 for Windows(r) (SAS Institute Inc.) was used for multitrait analysis. Results Patient Characteristics Table 1 summarizes the demographic and clinical characteristics of the participants. Although attempts were made to obtain approximately equal numbers of breast, colorectal, lung, and melanoma cancer patients, changes in treatment practices occurring between the study conceptualization and initiation resulted in significantly fewer melanoma patients being accrued. There were overall a greater proportion of female cancer patients because of a high ratio of breast cancer enrollment. Analyses of Tentative Domain Structure Examination of the OTC scale revealed that the majority of patients had correctly skipped the items because they were not receiving oral therapy, and the majority of those who had answered it had chosen the best possible category. As a result of these findings, the OTC scale was excluded from further analyses. The multitrait analysis of the hypothesized scale structure based on the initial study [7] (excluding the OTC items and the check-all-that-apply item [Reasons for Non-compliance with Oral Therapy]) revealed that while the concepts measured by each domain were largely intact, there were several scaling issues which suggested that slight modifications to the scale structure might improve the validity of the instrument. As a result, exploratory factor analyses were conducted and several subsequent revisions were made to the CTSQ domain structure. These revisions expanded the FSE domain to four items, reduced the CON domain to two items, and eliminated the single-item ST domain by moving the item onto the SWT domain.

4 672 Trask et al. Table 1 Sociodemographic and clinical characteristics Overall sample* (N = 361) Full (n = 85) Test retest reliability analysis subsamples* Stable (n = 53) Age (year) Mean (SD) 60.7 (12.36) 59.2 (11.75) 58.2 (12.29) Median Min, Max 23.2, , , 79.8 Sex, n (%) Male 133 (36.8) 35 (41.2) 19 (35.9) Female 228 (63.2) 50 (58.8) 34 (64.2) Ethnicity, n (%) Asian 5 (1.4) 0 (0.0) 0 (0.0) Black 22 (6.1) 5 (5.9) 3 (5.7) Hispanic 6 (1.7) 3 (3.5) 1 (1.9) White 328 (90.9) 77 (90.6) 49 (92.5) Highest level of education, n (%) High school diploma/ged or less 172 (47.6) 33 (38.8) 24 (45.3) Vocational school cert. or some year of college 90 (24.9) 30 (35.3) 17 (32.1) College/university degree (2- or 4-year) 59 (16.3) 13 (15.3) 6 (11.3) Graduate or professional degree 36 (10.0) 7 (8.2) 4 (7.5) Other 4 (1.1) 2 (2.4) 2 (3.8) Living status, n (%)* Alone 57 (15.8) 14 (16.5) 10 (18.9) With significant other/spouse 252 (69.8) 59 (69.4) 35 (66.0) With children 99 (27.4) 18 (21.2) 9 (17.0) With relatives (other than spouse 25 (6.9) 10 (11.8) 7 (13.2) With others (other than family) 13 (3.6) 4 (4.7) 4 (7.5) Other 16 (4.4) 3 (3.5) 3 (5.7) Missing (nothing checked) 1 (0.3) 0 (0.0) 0 (0.0) Cancer type, n (%) Breast 137 (37.9) 30 (35.3) 20 (37.7) Colorectal 121 (33.5) 32 (37.6) 17 (32.1) Lung 91 (25.2) 20 (23.5) 14 (26.4) Melanoma 12 (3.3) 3 (3.5) 2 (3.8) Cancer stage, n (%) I 22 (6.1) 6 (7.1) 5 (9.4) II 91 (25.2) 18 (21.2) 11 (20.8) III 90 (24.9) 21 (24.7) 11 (20.8) IV 158 (43.8) 40 (47.1) 26 (49.1) Line of therapy, n (%) First-line 176 (48.8) 46 (54.1) 27 (50.9) Second-line 101 (28.0) 19 (22.4) 14 (26.4) Adjuvant 84 (23.3) 20 (23.5) 12 (22.6) Stage and line of therapy, n (%) Early stage (I & II) 113 (31.3) 24 (28.2) 16 (30.2) Advanced first-line & adjuvant 161 (44.6) 42 (49.4) 23 (43.4) Advanced second-line 87 (24.1) 19 (22.4) 14 (26.4) Physician-reported ECOG performance status, n (%) Grade 0 (Fully active) 150 (41.6) 36 (42.4) 20 (37.7) Grade 1 (Restricted in physically strenuous activity but ambulatory) 181 (50.1) 42 (49.4) 29 (54.7) Grade 2 (Ambulatory and capable of all self-care but unable to 28 (7.8) 7 (8.2) 4 (7.5) carry out any work activities) Grade 3 (Capable of only limited self-care) 2 (0.6) 0 (0.0) 0 (0.0) Hospitalization in the past 3 months due to cancer-related event, n (%) Yes 85 (23.5) 21 (24.7) 12 (22.6) No 276 (76.5) 64 (75.3) 41 (77.4) *May add to greater than 100% in each column because subjects were allowed to check all living situations that applied to them. All eligible patients who completed both assessments. No change in CT treatment, follow-up interval between 3 and 14 days, and patient filled out questionnaires without help at both visits. Analyses of Revised Domain Structure Multitrait analysis. The revised 18-item CTSQ domain structure was resubmitted to multitrait analysis to assess the impact of the revisions on itemconvergent and -discriminant validity. The revised domain structure with the items noted (Table 2) showed substantial improvements in item-convergent and discriminant validity, with all 18 items (100%) correlating 0.40 with their own hypothesized domain, and 53 out of the 54 (98.1%) discriminant validity comparisons showing desirable results (higher or significantly higher item correlations with their own hypothesized domain than with competing domains),

5 Psychometric Validation of the CTSQ 673 Table 2 Multitrait item-scale correlations for the revised CTSQ domain structure (N = 336) Item no. Description ET FSE CON SWT Expectations of Therapy (ET) 1 CT would help you to return to a normal life 0.71* CT would get rid of the cancer 0.74* CT would help prevent the cancer from coming back 0.77* CT would stop the cancer from spreading 0.75* CT would help you live longer 0.50* Feelings about Side Effects (FSE) 5R CT limited your daily activities * R Upset about the side effects * R Overall, was taking CT as difficult as expected * Overall, were side effects of CT as expected * Convenience (CON) 13 Overall, how inconvenient was it for you to take your CT * R Overall, how bothered were you by the amount of time it took to take your CT * 0.34 Satisfaction with Therapy (SWT) 7 CT was worth taking even with side effects * 9R How often did you think about stopping CT * 15 Overall, how worthwhile was your CT * 17 Overall, how well did the benefits of CT you re your expectations * 19 How satisfied were you with the form of your CT * 20 How satisfied were you with your most recent CT * 21 If given the choice again, would you decide to take this CT treatment * *Item correlation with own scale, corrected for overlap. R following item number indicates that item was reverse-coded by subtracting the original coded CRF value from 6 so that for each item, a response value of 5 is associated with the best possible response and a response of 1 is associated with the worst possible response. CT, cancer therapy; CTSQ, Cancer Therapy Satisfaction Questionnaire. the only exception being that item 8 (How often felt that cancer therapy would help you live longer?) had a correlation of 0.50 with its own domain (ET) but a slightly higher correlation of 0.55 with the SWT domain. Internal consistency reliability. Table 3 shows the final estimates of internal consistency reliability (Cronbach s alpha) for the CTSQ domains. The revised ET, FSE, and SWT domains consistently met the target of 0.70 or above, with each showing improvement. Cronbach s alpha for the revised CON domain (0.60), however, was still below 0.70 and actually decreased slightly because the revised domain had only two items. As such, a decision was made to remove the CON domain from the CTSQ, leaving the final version of the CTSQ with 16 items and three domains (ET, FSE, and SWT). CTSQ Scoring For each of the revised domains, sequential itemremoval testing was conducted to establish a conservative estimate of the minimum number of completed items needed for Cronbach s alpha to remain above The following minimum item completion standards were established for scoring each of the revised domains: ET, minimum three out of five items complete; SWT, five out of seven; FSE, four out of four (i.e., all items must be complete). CTSQ Validation Score distributions and floor and ceiling effects. Overall, except for FSE, the domains were positively skewed, with high mean scores and substantial ceiling effects on ET and SWT (Table 4). Floor effects were negligible for all domains. Of note, the minimum observed score at baseline on the SWT domain was 21.4 (0 being the lowest possible score). Scores on the revised FSE domain were much less skewed than scores on the other domains, with a substantially lower mean and median, and a negligible ceiling effect of 3.9%. Table 3 Internal consistency and test retest reliability of revised CTSQ domains CTSQ domain Internal consistency reliability (Cronbach s alpha) Test retest reliability (ICC) Baseline (n = 361) Full sample* (n = 85) Stable sample (n = 53) Expectations of Therapy Feelings about Side Effects Satisfaction with Therapy Convenience 0.60 *All eligible patients who completed both assessments. No change in CT treatment, follow-up interval between 3 and 14 days, and patient filled out questionnaires without help at both visits. CTSQ, Cancer Therapy Satisfaction Questionnaire; ICC, intraclass correlation coefficient.

6 674 Trask et al. Table 4 Summary statistics for revised CTSQ domains at baseline CTSQ domain n Mean (SD) Median Observed range (Min, Max) Floor effect n (%) Ceiling effect n (%) Expectations of Therapy (19.18) , (0.3) 74 (20.5) Feelings about Side Effects (21.99) , (0.8) 14 (3.9) Satisfaction with Therapy (13.76) , (0.0) 47 (13.1) CTSQ, Cancer Therapy Satisfaction Questionnaire. Interdomain correlations. The interdomain Spearman correlations among ET, FSE, and SWT were small to moderate in magnitude: ET-FSE, r = 0.18; ET-SWT, r = 0.42; and FSE-SWT, r = Test retest reliability. Three of the 88 subjects randomized to follow-up completed the follow-up assessment on the same day as the baseline assessment and were subsequently excluded from further analyses. Table 3 shows the test retest reliability estimates (ICCs) both for the overall sample of 85 eligible patients and for a stable subsample of 53 participants who met all of the following criteria: 1) at the follow-up assessment reported no change in cancer treatment over the past 2 weeks; 2) completed the follow-up assessment between 3 and 14 days after the baseline assessment; and 3) reported that they filled out both questionnaires without assistance of any kind from anyone else. Although the study design is intended to minimize change and differences in testing conditions between baseline and follow-up, in any study some real, unplanned change or variation in testing conditions will occur, and conducting an analysis of test retest reliability after limiting the analysis sample to a stable subsample based on external criteria is a standard practice with many examples available in the literature [25 29]. FSE and SWT exceeded the desired test retest reliability ICC value of 0.70 in both the overall sample and the stable subsample. ET fell somewhat short in the overall sample (0.56), but showed a substantial improved result in the stable subsample. Concurrent Validity Concurrent validity was assessed by correlating the CTSQ domains with corresponding domains of the TSQM. It was expected that domains measuring the most similar constructs would have the highest correlations, and this is in fact what was found (Table 7). The CTSQ FSE domain correlated most highly with the Side Effects domain of the TSQM (r = 0.52), and the CTSQ SWT domain with the Global Satisfaction domain of the TSQM (r = 0.66). Although there is no directly analogous domain in the TSQM, the CTSQ ET domain correlated most highly with the Global Satisfaction domain of the TSQM (r = 0.43). Known-Groups Validity Known-groups validity comparisons were made for the CTSQ domains in relation to cancer stage, perceived change in cancer, presence/absence of medication side effects, and ECOG performance status (Table 5). In particular, it was expected that ET and SWT would be related to perceived change in cancer, and FSE would differ depending on the reported presence or absence of side effects. It was also expected that ET would be related to cancer stage (disease severity). Cancer stage. Individuals with Stage I disease had the highest mean score (85.1) on the ET domain, with Stage IV subjects having the lowest (72.3). The effect size associated with this difference of 12.8 points between the Stage I and Stage IV groups in this analysis was d = Effect sizes for the differences between adjacent groups were as follows: Stages I II, d = 0.19; Stages II III, d = 0.17; Stage III IV, d = No significant differences were found among cancer stages on the FSE or SWT domains. Perceived change in cancer. The ET domain also showed statistically significant differences among four groups of patients according to their degree of selfreported perceived change in cancer over the past 2 weeks (much better, somewhat better, about the same, or somewhat worse). Mean scores were ordered in a logical manner for the four groups, with higher mean scores for groups with better perceived change in their cancer. The difference in ET scores between perceived change of much better and somewhat worse was 16.2 points (d = 0.84). Effect sizes for the differences between adjacent groups were as follows: Much better Somewhat better, d = 0.38; Somewhat better About the same, d = 0.21; About the same Somewhat worse, d = Differences among the mean scores on the SWT domain according to the level of perceived change in cancer were also statistically significant and logically ordered (difference of 9.6 points between Much better and Somewhat worse, d = 0.70). Effect sizes for the differences between adjacent groups were as follows: Much better Somewhat better, d = 0.39; Somewhat better About the same, d = 0.14; About the same Somewhatworse, d = There were no significant differences in FSE among the groups.

7 Psychometric Validation of the CTSQ 675 Table 5 Known-groups validity: one-way anova score comparisons (N = 361) Description CTSQ Expectations of Therapy CTSQ Feelings about Side Effects CTSQ Satisfaction with Therapy n Mean (SD) P-value (effect size) n Mean (SD) P-value (effect size) n Mean (SD) P-value (effect size) Cancer stage (0.67*) I (16.09) (21.61) (14.48) II (17.06) (23.11) (15.05) III (18.82) (22.49) (13.95) IV (19.95) (21.11) (12.84) Perceived changed in cancer (0.84 ) (0.70 ) (compared to 2 weeks ago) Much better (15.71) (24.34) (12.73) Somewhat better (18.86) (18.85) (10.91) About the same (19.74) (21.94) (14.55) Somewhat worse (19.30) (21.78) (17.62) Much worse (18.93) (9.55) (5.05) Any side effects of medication? < (0.59) (0.28) Yes (have side effects) (19.42) (20.76) (14.29) No (no side effects) (18.57) (22.15) (12.21) ECOG performance status (0.46 ) Grade (18.07) (22.08) (12.74) Grade (18.82) (21.75) (14.42) Grade (21.82) (21.45) (13.39) Grade (3.54) (4.42) (15.15) *Effect size for difference between Stages I and IV. Effect size for difference between Much better and Somewhat worse. Effect size for difference between Grades 0 and 1. Group not included in one-way ANOVA due to extremely small group size (n < 5). P-values are from one-way ANOVAs performed on square-root transformed scores; mean, SD, and ES calculated based on original scores. Effect sizes shown only where one-way ANOVA was significant (P < 0.05). CTSQ, Cancer Therapy Satisfaction Questionnaire.

8 676 Trask et al. Table 6 Estimates of meaningful differences on CTSQ domains Revised domain or item 0.5 SD of baselines cores Minimally important difference estimates 1 SEM (using internal consistency reliability) 1 SEM (using test retest reliability)* Known-groups differences Expectations of Therapy Feelings about Side Effects Satisfaction with Therapy Perceived change in cancer over the past 2 weeks: difference of 7.3 points between Much better and Somewhat better, 4.1 points between Somewhat better and Almost the same, average difference of 5.7 points Self-reported presence of any side effects due to medication: difference of 12.9 points between No (no side effects) and Yes (have side effects) Perceived change in cancer over the past 2 weeks: difference of 5.4 points between Much better and Somewhat better, 1.9 points between Somewhat better and Almost the same, average difference of 3.7 points *Test retest reliability estimates based on stable subsample were used in calculating SEM. CTSQ, Cancer Therapy Satisfaction Questionnaire; SEM, standard error of measure. Presence versus absence of any side effects of medication. Question 4 on the TSQM ( As a result of taking this medication, do you experience any side effects at all? ) was used to define groups of patients reporting (n = 245) or not reporting (n = 110) side effects of medication/cancer therapy. Logical significant differences existed between these two groups on the FSE domain (lower mean score for those with side effects; difference of 12.9 points, d = 0.59). A small but statistically significant and logically ordered difference was also observed between the two groups on the SWT domain, with a lower mean score for those with side effects (difference of 3.9 points, d = 0.28). Mean scores on the ET domain also appeared to be logically ordered for both groups (lower scores for the group who reported having side effects), although the difference was not statistically significant. ECOG performance status. There were very few subjects rating an ECOG grade of 2 or worse (n = 28 for Grade 2, and n = 2 for Grade 3). Although generally showing lower scores across all CTSQ domains, these two Grade 3 subjects were excluded from the comparisons. Significant differences were detected in the ET domain scores among ECOG Grades 0, 1, and 2. No significant differences were found on the FSE or SWT domains. On the ET domain, the differences between ECOG Grades 0 and 1 were in a logical direction (Grade 0 with higher scores), but the score differences between ECOG Grades 1 and 2 were small and in the opposite direction (scores slightly higher for Grade 2). Follow-up pair-wise comparisons revealed that only the difference of 8.8 points (d = 0.46) between Grades 0 and 1 was statistically significant. Minimally Important Differences For each CTSQ domain, Table 6 shows estimates of the MID calculated as 0.5 SD of baseline scores and 1 SEM, as well as the size of differences between group mean scores using the most appropriate known-groups validity criteria. For calculating the SEM based on the test retest reliability estimate, the ICC estimates for the restricted (stable) subsample were used. Perceived change in cancer over the last 2 weeks was chosen as most appropriate known-groups criterion for examining the size of group differences for the ET and SWT domains. Because of the small number of respondents in other categories on the question about perceived change in cancer, only the group differences between Much better and Somewhat better and Somewhat better and Almost the same were examined, in addition to the average of those group differences. For the FSE domain, the known-group criterion chosen for examining the size of group differences was TSQM Question 4. Correlation of CTSQ Domains with HRQOL Table 7 shows the correlations of the CTSQ domains with the EORTC QLQ-C30 domains for 352 of the 361 patients with useable data at baseline. The FSE domain correlated more highly with the quality of life, functioning, and symptom domains measured by the EORTC QLQ-C30 than the other CTSQ domains. Overall, the correlations between the CTSQ domains and QLQ-C30 domains were low to moderate. Correlations with the symptom domains of the QLQ-C30 tended to be negative because higher scores on those QLQ-C30 domains are associated with worse symptoms. Discussion The current study provides information on the psychometric properties of the CTSQ in a mixed oncology sample. Initially conceived following focus groups and interviews with patients, nurses, and physicians as a 21-item instrument measuring six domains, the psychometric validation performed in the current study has refined it to a 16-item instrument measuring three domains: ET 5 items, FSE 4 items, and SWT 7

9 Psychometric Validation of the CTSQ 677 Table 7 Correlations of CTSQ with TSQM and EORTC QLQ-C30 domains CTSQ domains Expectations of Therapy Feelings about Side Effects Satisfaction with Therapy TSQM domains (n = 346) Effectiveness Side Effects Convenience Global Satisfaction EORTC QLQ-C30 domains (n = 352) Physical Functioning Role Functioning Emotional Functioning Cognitive Functioning Social Functioning Global Health Status/Quality of Life Fatigue Nausea and Vomiting Pain Dyspnea Insomnia Appetite Loss Constipation Diarrhea Financial Difficulties Note: Spearman correlations. Only subjects with scores for all CTSQ and TSQM or QLQ-C30 domains were included.the expected highest correlations of CTSQ domains with TSQM domains are in bold. Correlations of CTSQ domains with EORTC QLQ-C30 domains with a magnitude of 0.40 or larger are in bold. CTSQ, Cancer Therapy Satisfaction Questionnaire; EORTC QLQ-C30, the European Organization for Research and Treatment of Cancer Quality of Life Questionnaire Core 30;TSQM,Treatment Satisfaction Questionnaire for Medication. items. It can be used in group-level research to evaluate patients experiences with chemo, biological, and hormonal therapies. The three domains eliminated from the final domain structure were done so for different reasons. The ST domain was originally hypothesized to contain only one item assessing how often the patient thought about stopping their cancer therapy. The multitrait and factor analyses, however, clearly showed that this item belonged to the SWT domain, a finding that makes sense since the frequency with which patients think about stopping therapy should both depend on and be an indicator of how satisfied they are with therapy. Although potentially important in the future as cancer medications are increasingly made in oral forms, the OTC items applied to only a minority of the patients in the current study. As such, there was insufficient data for validation. Moreover, for those who did answer the questions, the responses were extremely positively skewed, indicating that complying with oral therapy was not a problem for most patients (at least according to their self-report). Therefore, the OTC domain was of limited use. The CON domain suffered from having too few items available to achieve an acceptable level of internal consistency. Although not reported, it did exhibit good concurrent validity, indicating that it was clearly measuring what it purported to measure. Currently, other instruments are available that measure convenience of medication (such as the TSQM), and may be of use if a measure of convenience is desired. A decision was made to keep item 8 (How often did you think that cancer therapy would help you live longer?) in the ET domain despite the fact that it tended to correlate somewhat more highly with the SWT domain. This decision was made for two reasons. First, the wording of item 8 parallels the wording of other items in the ET domain clearly asking the patient for their thoughts about the future outcome of their therapy. Second, it is the only item in the domain covering the issue of life extension, an important outcome of therapy. Thus, removing it would negatively affect the content validity of the ET domain. Although there was some variation in the size of MID estimates of the CTSQ domains when calculated using different approaches, overall there appeared to be more agreement than disagreement. As preliminary estimates of the MID for the ET and SWT domains, we suggest simply using the average of the four methods shown in Table 6. This yields average MID estimates of 8.3 points for the ET domain and 5.9 points for the SWT domain. For the FSE domain, we suggest using the average result of the three methods of estimating the MID (10.3). This excludes the difference of 12.9 points between patients with side effects and those without, which represents a larger semantic difference than could be considered to represent an MID. Overall, the results strongly support the internal consistency, test retest reliability, concurrent validity, and known-groups validity of the FSE and SWT domains. The ET domain also has very good internal consistency reliability, concurrent validity, and knowngroups validity, and had acceptable test retest reliability results if the sample is restricted to a stable subgroup using criteria to eliminate unplanned variation between tests. The overall results, including substantial differ-

10 678 Trask et al. ences in scores depending on stage of cancer and perceived change in cancer, suggest that the ET domain is reliable and valid and may be particularly sensitive to both clinical changes and changes in the context of administration. Like many satisfaction measures, the score distributions on the ET and SWT domains are positively skewed with substantial ceiling effects, which should be taken into account when powering future studies. Nevertheless, we believe that the CTSQ can be used to assess several aspects of chemo, biological, and hormonal therapies heretofore formally untested from a patient perspective. Moreover, the generally moderate correlations of the CTSQ domains with the quality of life and functioning domains of the QLQ-C30 indicate that the therapy satisfaction-related domains measured by the CTSQ are unique constructs that are not captured by HRQOL measures. Although the CTSQ was developed for use in group-level research and the domains do not have the much higher levels of reliability (>0.90) recommended for patient-level interpretation of scores, it is possible that examining patients CTSQ responses might enhance communication between the patient and the health-care team. The utility of the CTSQ for this purpose will be examined in future studies. The authors would like to thank Kathleen Yost, PhD, for her insightful comments during the analyses of the study and for her helpful review of earlier versions of this manuscript. Source of financial support: Pfizer Inc. References 1 Fallowfield LJ, Harper P. Health-related quality of life in patients undergoing drug therapy for advanced non-small-cell lung cancer. Lung Cancer 2005; 48: Goodwin PJ, Black JT, Bordeleau LJ, Ganz PA. Health-related quality-of-life measurement in randomized clinical trials in breast cancer taking stock. J Natl Cancer Inst 2003;95: Trask PC, Paterson AG, Esper P, et al. Longitudinal course of depression, fatigue, and quality of life in patients with high risk melanoma receiving adjuvant interferon. Psychooncology 2004;13: Conroy T, Blazeby JM. Health-related quality of life in colorectal cancer patients. Expert Rev Anticancer Ther 2003;3: Gunnars B, Nygren P, Glimelius B. Assessment of quality of life during chemotherapy. Acta Oncol 2001;40: Trask PC. Quality of life and emotional distress in advanced prostate cancer survivors undergoing chemotherapy. Health Qual Life Outcomes 2004;2: Abetz L, Coombs JH, Keininger DL, et al. Development of the Cancer Therapy Satisfaction Questionnaire: item generation and content validity testing. Value Health 2005;8(Suppl. 1):S Fayers PM, Machin D. Quality of Life: Assessment, Analysis and Interpretation. New York: John Wiley & Sons, Gorsuch RL. Factor Analysis (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum Associates, Heinemann K, Ruebig A, Potthoff P, et al. The Menopause Rating Scale (MRS) scale: a methodological review. Health Quality Life Outcomes 2004;2: Ware J, Harris W, Gandek B, et al. MAP-R for Windows: Multitrait. Multi-Item Analysis Program Revised User s Guide. Boston, MA: Health Assessment Laboratory, Baumgartner TA, Chung H. Confidence limits for intraclass reliability coefficients. Meas Phys Educ Exerc Sci 2001;5: Fayers P, Aaronson N, Bjordal K, et al., on behalf of the EORTC Quality of Life Study Group. EORTC QLQ-C30 Scoring Manual (2nd ed.). Brussels: EORTC Quality of Life Group, Atkinson MJ, Sinha A, Hass SL, et al. Validation of a general measure of treatment satisfaction, the Treatment Satisfaction Questionnaire for Medication (TSQM), using a national panel study of chronic disease. Health Quality Life Outcomes 2004;2: Campbell D, Fisk D. Convergent and discriminant validation by the Multitrait Multi-method Matrix. Psychol Bull 1959;56: Cronbach L. Coefficient alpha and the internal structure of tests. Psychometrika 1951;16: Shrout PE, Fleiss JL. Intraclass correlations: uses in assessing rater reliability. Psychol Bull 1979;86: Nunnally J, Bernstein I. The Assessment of Reliability. Psychometric Theory. New York: McGraw-Hill, Chassany O, Sagnier P, Marquis P. Patient-reported outcomes: the example of health-related quality of life a European guidance document for the improved integration of health-related quality of life assessment in the drug regulatory process. Drug Inf J 2002;36: Cohen J. Statistical Power Analysis for the Behavioral Sciences (2nd ed.). Hillsdale, NJ: Lawrence Erlbaum, Norman GR, Sloan JA, Wyrwich KW. Interpretation of changes in health-related quality of life: the remarkable universality of half a standard deviation. Med Care 2003;41: Wyrwich KW, Nienaber NA, Tierney WM, Wolinsky F. Linking clinical relevance and statistical significance in evaluating intra-individual changes in health-related quality of life. Med Care 1999;37: Wyrwich KW, Tierney WM, Wolinsky FD. Further evidence supporting an SEM-based criterion for identifying meaningful intra-individual changes in healthrelated quality of life. J Clin Epidemiol 1999;52: Wyrwich KW, Tierney WM, Wolinsky FD. Using the standard error of measurement to identify important changes on the Asthma Quality of Life Questionnaire. Qual Life Res 2002;11:1 7.

11 Psychometric Validation of the CTSQ McHorney CA, Ware JE Jr, Lu JF, Sherbourne CD. The MOS 36-item short-form health survey (SF-36): III. Tests of data quality, scaling assumptions, and reliability across diverse patient groups. Med Care 1994;32: Davidson M, Keating JL. A comparison of five low back disability questionnaires: reliability and responsiveness. Phys Ther 2002;82: Gummesson C, Atroshi I, Ekdahl C. Performance of health-status scales when used selectively or within multi-scale questionnaire. BMC Med Res Methodol 2003;3:3. 28 Terwee CB, Gerding MN, Dekker FW, et al. Test retest reliability of the GO-QOL: a disease-specific quality of life questionnaire for patients with Graves ophthalmopathy. J Clin Epidemiol 1999;52: Jones CA, Feeny D, Eng K. Test retest reliability of health utilities index scores: evidence from hip fracture. Int J Technol Assess Health Care 2005;21: