How Response Shift May Affect the Measurement of Change in Fatigue

12 Journal of Pain and Symptom Management Vol. 20 No. 1 July 2000 Original Article How Response Shift May Affect the Measurement of Change in Fatigue Mechteld R.M. Visser, PhD, Ellen M.A. Smets, PhD, Mirjam A.G. Sprangers, PhD, and Hanneke J.C.J.M. de Haes, PhD Department of Medical Psychology, Academic Medical Center, University of Amsterdam, Amsterdam, The Netherlands Abstract If patients experience extreme fatigue during treatment, they may judge the level of fatigue following this experience differently from how they would have judged it before. This change in internal standard is referred to as a response shift. We explored whether a response shift might have occurred in patients receiving radiotherapy (n 199). Fatigue was assessed before and after radiotherapy. Following completion of the post-test, a thentest was administered where patients had to provide a renewed judgment of their pre-treatment level of fatigue. Response shift was assessed by the mean difference between the pre-test and thentest scores. Comparing the thentest with the pretest scores, patients retrospectively minimized their pre-treatment level of fatigue. The thentest post-test difference was significant, whereas the conventional pretest post-test difference was not. These results are in line with the occurrence of a response shift. Additional hypotheses regarding response shift were partially supported. It is concluded that the potentially large implications of response shift justify further research. J Pain Symptom Manage 2000;20:12 18. U.S. Cancer Pain Relief Committee, 2000. Key Words Response shift, radiotherapy, fatigue, measurement, thentest Introduction Interest in cancer-related fatigue is increasing, as can be deduced from the growing number of publications on this topic in recent years. This interest is fostered by the growing awareness of the high prevalence of fatigue and its potential detrimental effect on patient well-being. As a result, researchers interested in quality-of-life issues as related to cancer and Address reprint requests to: E.M.A. Smets, PhD, Department of Medical Psychology, Academic Medical Center, University of Amsterdam, P.O. Box 22700, 1100 DE Amsterdam, The Netherlands. Accepted for publication: August 9, 1999. its treatment increasingly incorporate fatigue as a measure of outcome in their studies. Fatigue is a complex symptom. There is nevertheless some general understanding that, like pain, fatigue is a subjective experience. 1 Similar to a well-accepted definition of pain, 2 fatigue can thus be regarded to be what the experiencing person says it is, and exists whenever he or she says it does. Because there are no objective indicators of fatigue, one has to rely almost exclusively on self-report to investigate this symptom. Self-report instruments of fatigue developed in recent years for use in cancer patient populations include the Piper Fatigue Scale, 3 the Multidimensional Fatigue Inventory, 4 Functional Assessment of Cancer Therapy Fatigue scale, 5 and the Fatigue Symptom Inventory. 6 U.S. Cancer Pain Relief Committee, 2000 0885-3924/00/$ see front matter Published by Elsevier, New York, New York PII S0885-3924(00)00148-2

Vol. 20 No. 1 July 2000 Measuring Change in Fatigue 13 In employing self-report measures in longitudinal study designs, two assumptions are made, often implicitly. The first assumption is that the individuals under study have an internalized standard for judging their current level of the subject of investigation. Secondly, it is assumed that this internalized standard remains stable over time, at least for the duration of the investigation. 7 In other words, it is believed that a common metric exists between the assessments. The latter assumption, regarding the stability of an internalized standard for fatigue, is the subject of this study. When evaluating an intervention by means of a standard pretest post-test design, the degree of change is usually measured by comparing pretest with post-test scores. However, the intervention may affect the internalized standards of judgment of the study participants. For example, if a patient experiences extreme fatigue as a consequence of radiotherapy, he or she may judge any level of fatigue following this experience quite differently from what he or she would have done before. This change in internalized standard is referred to as response shift. 8 When applied to the area of quality of life, response shift is defined as a change in the meaning of one s self-evaluation of quality of life, or an aspect thereof, as a result of: (a) a change in the respondent s internal standards of measurement (recalibration, in psychometric terms); (b) a change in the respondent s values (i.e., the importance of component domains constituting quality of life); or (c) a redefinition of life quality (i.e., reconceptualization). 9 This paper focuses on the first possible cause of a response shift, that is, a change in the patient s internal standard regarding the measurement of fatigue. When fatigue increases substantially, for example as a consequence of treatment, one can adapt to the circumstances by regarding this new level of fatigue as normal. If one perceives the former level of fatigue as the normal condition, this might lead to recurrent frustration. From this perspective, response shift is the consequence of a desirable adaptation mechanism. Thus, to the extent that cancer and its treatment adversely affect patients quality of life, it may be adaptive for these patients to change their internal standard of measurement. Although a shift in internalized standard can be functional for patients, for investigators it can be troublesome because it raises methodological issues. 10,11 A shift in internal criteria may render assessments completed over time incomparable. Because the units of comparison have changed, the comparison itself has lost its meaning. In the context of treatment evaluation, response shift may thus attenuate treatment effects as patients adapt to treatment toxicities or disease progression over time. To quote Longo 12 : toxicity to which the patient has accommodated still is toxicity (p.3716). Assessing response shift is therefore of paramount importance to further quality-of-life research. Only by integrating measures of changes in internal standards into study designs, can response shift effects be explicitly measured and taken into account when treatments or interventions are being evaluated. From a theoretical perspective, integrating response shift into quality-of-life research would allow a better understanding of how quality of life is affected by changes in health status. 9 Response shift might also explain counterintuitive findings in quality-of-life research. These involve stable quality-of-life scores despite severe disease or intensive treatment 13,14 and quality-of-life scores of patients not being inferior to scores of healthy populations. 10 Additionally, health care providers and significant others tend to underestimate patients quality of life as compared to patients evaluations of their own quality of life. 15,16 All these findings suggest that response shift may play an important yet not explicitly investigated role in adapting to illness. The Thentest An attempt to measure change in internal standards has been made in the area of educational training interventions. To distinguish the effects of an intervention on outcome from a shift in standard of judgement, Howard and colleagues 8 recommended an extension of the pretest post-test design with a thentest. The thentest is a test administered immediately following the post-test. Respondents are asked to recall the point in time at which the conventional pretest was administered and to give a renewed judgment of their pretest level of functioning. Thus, respondents are asked how

14 Visser et al. Vol. 20 No. 1 July 2000 they perceive themselves at post-test and how they now perceive themselves to have been at pretest. It is assumed that by administering the post-test and thentest in close proximity, the comparison of post-test and thentest would provide an estimate of the treatment effect, corrected for response shift. The difference between conventional and thentest scores indicates the magnitude and direction of the response shift. Although research into response shift originates from studies evaluating educational interventions, 8,17,18,19 the phenomenon is receiving increased attention in the field of health psychology. 10,20,21 In a study investigating response shift and quality of life, fatigue, in particular, was found to be sensitive to a shift in internalized standard. 22 As knowledge about the phenomenon of response shift in health psychology is still quite limited, the present study intends to extend this knowledge by examining whether there are indications of the occurrence of response shift with respect to fatigue in cancer patients. This study is part of a larger investigation concerning radiotherapy-related fatigue. 23 Because the investigation was not primarily designed to examine changes in internal standards, the study is explorative by nature. The following three hypotheses were tested. First, a response shift in self-reported fatigue was expected to occur in radiotherapy patients, as indicated by a mean difference between the thentest scores and the conventional pretest scores. Secondly, it was hypothesized that patients who are tired for a relatively long period of time have adapted their internal standards before receiving radiotherapy. Consequently, their standards would not change, or to a lesser degree, as a result of fatigue during radiotherapy. Thus, a response shift was expected to occur more clearly in those patients whose fatigue increased recently. The third hypothesis was that a response shift would occur only in patients who reported their level of fatigue to have changed over treatment. Thus patients were expected to retrospectively minimize their pre-treatment fatigue level, if fatigue had exacerbated during treatment. Conversely, if their level of fatigue had decreased, it was expected that they would retrospectively judge their initial level of fatigue to be worse than at pre-treatment itself. We did, however, expect to find few patients whose fatigue would decrease over treatment. Methods Patients and Procedure The radiation oncologist introduced the study at the first consultation of consecutive cancer patients attending for radiotherapy at the Academic Medical Centre in Amsterdam. Patients were later contacted by telephone by the researchers to ask for consent. Patients were eligible when they were 18 years or older and native Dutch. They had to receive treatment on an outpatient basis for cure or control of cancer, rather than for palliation. Patients were excluded if they had a malignancy in the central nervous system or when they additionally received chemotherapy. Of the 308 eligible patients, 250 (81%) agreed to participate. Participants were interviewed at their homes approximately two weeks before the start of radiotherapy and two weeks after completion of treatment. At post-treatment, 216 of the original 250 patients (86%) were still on study; 9 patients (4%) had declined further participation and 25 patients (10%) were not included in the second assessment, either for medical reasons, such as receiving additional chemotherapy, or because they could not be interviewed within the time-limit of one month post-treatment. For 17 of the 216 patients, the fatigue ratings were incomplete, leaving the data of 199 patients available for analyses. Instruments Diagnosis was obtained from the patients medical records. The patients prognosis in terms of five-year survival probability was classified by the Dutch Cancer Registration Office as either less than 20%, 20 40%, 40 60%, 60 80%, or greater than 80%. The presence of at least one comorbid condition was assessed at the first interview. For the measurement of fatigue, a one-item numerical rating scale asking for tiredness during the last few days was used, ranging from 0 (not tired at all) to 10 (worst tiredness imaginable). Instructions for the conventional pre- and post-test were identical. For the thentest, the interviewer first asked the patient to recall the time of the first interview using retrieval cues such as Can you remember when you were first interviewed by

Vol. 20 No. 1 July 2000 Measuring Change in Fatigue 15 me. Can you remember how you felt; maybe you were anxious, maybe you were not. The actual instruction was read aloud. I would like to ask you to provide a new judgment about how tired you were at the time you were interviewed for the first time. Thus I would like to ask you to answer this question as you now perceive yourself to have been during the last few days prior to the first interview. Can you indicate with a number between 0 and 10 how tired you have been the last few days before our first interview? 0 is not tired at all and 10 is the worst tiredness you can imagine. It was emphasized that patients were not asked to recall their responses but rather to provide a new judgment. To test the hypotheses two interview questions were used to form mutually exclusive subgroups. The first question, Were you tired in the period prior to the diagnosis of cancer (response options being yes or no) was addressed during the first part of the pre-treatment interview, prior to the question about the degree of fatigue (pretest). The second question was Do you find the fatigue of the last few days worse, better, or the same as the fatigue in the days prior to our first interview? The response options were much worse, worse, the same, better, much better. This question was addressed at the post-treatment interview, following the thentest. For statistical analyses, the response categories were reduced to deterioration, stable, and improvement. Statistical Analyses Mean differences between pre- and post-test (conventional treatment effect), the conventional pretest and thentest (response shift), and between thentest and post-test (treatment effect corrected for response shift) were investigated using dependent t-tests. For testing the additional hypotheses, ANOVAs for repeated measures were used, with pretests (conventional and retrospective) as within subject factors. The between subject factor had two levels in the first and three in the second analysis. Results Sample The average age of participating patients was 64 years (SD 13). The group comprised 116 men and 83 women, of whom 157 were married. Their diagnoses, prognoses and comorbidity are presented in Table 1. Treatment Effect and Response Shift The conventional mean difference between the scores on the pretest (mean 3.58, SD 2.86) and post-test (mean 3.77, SD 2.75) was nonsignificant (t(198) 0.90, n.s.), indicating no change in fatigue over time. However, as hypothesized, scores on the conventional pretest (mean 3.58, SD 2.86) were higher as compared to those on the thentest (mean 3.13, SD 2.61; t(198) 2.47, P 0.01). On average, patients judged their pretreatment fatigue retrospectively as less severe than they did before treatment. The difference between the retrospective pretest and post-test reached significance (t(198) 2.95, P 0.005). Based on this difference, patients reported to be more fatigued after treatment than they did before the start of treatment. As hypothesized, an interaction effect was found between the patients reporting fatigue before the diagnosis of cancer and the occurrence of a significant mean pre-thentest difference (F(1,197) 13.70, P 0.001). However, the direction of the interaction was contrary to expectation. Patients who reported to have been tired before their diagnosis had higher scores on the conventional pretest than on the thentest. Conversely, patients who reported Table 1 Sample Characteristics (n 199) n % Range of total radiation dose, Gy a Diagnosis Head and Neck 12 6 60 66 Gastrointestinal 11 6 45 60 Gynecological 25 13 40 70 Lung 20 10 50 60 Breast 35 18 50 75 Genitourinary tract 76 38 26 70 Hematological malignancies 15 7 40 Miscellaneous 5 2 40 70 Five year survival probability b 20% 22 11 20 40% 12 6 40 60% 21 11 60 80% 61 31 80% 43 22 Comorbidity 107 54 a Variation in dose schemes within the tumor groups is due to variation in indications; e.g. postoperative adjuvant vs. primary radiotherapeutic treatment. b Necessary information was not available for all patients.

16 Visser et al. Vol. 20 No. 1 July 2000 Table 2 Average Fatigue Scores on Numerical Scale (0 10) for Patients Who Reported Being Tired or Not Tired Before the Diagnosis of Cancer (Mean SD) Tired before diagnosis? yes (n 70) no (n 129) Conventional pretest 5.53 2.86 2.53 2.48 Thentest 4.17 2.32 a 2.56 2.56 Posttest 5.00 2.65 3.10 2.57 a Significant difference between thentest and conventional pretest at post-hoc comparison. not to have been tired prior to their diagnosis showed no differences between the conventional pretest and thentest (Table 2). The final analysis concerned the hypothesized relation between response shift and the change in fatigue during the treatment period. A significant interaction was found (F(2,193) 12.26, P 0.001) in the direction expected. Patients who reported their fatigue to have increased had lower scores on the thentest as compared to the conventional pretest. The same, although to a lesser degree, applied to patients who reported their fatigue to have remained stable. An opposite effect was observed for those patients who reported a decrease in fatigue over the period of treatment. They retrospectively judged their fatigue before treatment to be worse than at pre-treatment itself (Table 3). Discussion This study sought to examine whether a response shift, in terms of a change in internal standard of measurement, would occur with respect to fatigue in radiotherapy patients. The results were mostly in line with the theoretical predictions. At post-treatment, patients retrospectively judged their fatigue before treatment as less intense than at pre-treatment itself. When the conventional pretest was compared to the posttest, no differences were found. This might lead to the conclusion that radiotherapy does not affect fatigue. However, when the thentest was used as the measure of fatigue prior to treatment, the conclusion would be that patients experience more fatigue as a consequence of radiotherapy treatment. The question then arises which of these two conclusions is most valid. Two additional hypotheses were formulated which, if supported by the data, would lend credence to the conclusion that a response shift occurred. The first hypothesis stated that a response shift was expected to occur more clearly in patients who had only recently become tired, and who were expected not to have adapted yet to this symptom. Contrary to expectation, in patients who were already tired before their diagnosis, a difference between conventional and thentest scores was found, whereas in patients who reported no fatigue prior to diagnosis no such difference was found. Consequently, one might conclude that the difference between the conventional and thentest is not the result of a change in standard of judgement. However, closer inspection of the data showed that those patients who reported to be without fatigue before diagnosis also reported much lower fatigue scores at pre- and post-treatment. This led us to believe that it concerns a group of patients who remain relatively unaffected by radiotherapy. In such a group, a shift in standard is less likely to occur. The second hypothesis concerned the assumption that a change in internalized standard would only occur in patients whose fatigue actually changed over the period of radiotherapy. As expected, patients who reported an increase in their level of fatigue ret- Table 3 Average Fatigue Score on Numerical Scale (0 10) for Patients Whose Fatigue Deteriorated, Remained Stable or Improved Over the Period of Radiotherapy Treatment (Mean SD) Deterioration n 88 Change in fatigue Stable n 57 Improvement n 51 Conventional pretest 3.52 2.90 2.84 2.98 4.53 2.60 Thentest 2.43 1.91 a 2.07 2.49 a 5.51 2.31 a Posttest 5.49 2.18 1.98 2.46 2.88 2.22 a Significant difference between thentest and conventional pretest at post-hoc comparison.

Vol. 20 No. 1 July 2000 Measuring Change in Fatigue 17 rospectively judged their fatigue to be less intense than at pre-treatment itself. Patients who reported their fatigue to have decreased during treatment retrospectively judged their level of pre-treatment fatigue as worse. These results are in line with the hypotheses concerning the occurrence of a response shift. As described in the introduction, shifts in internal standards may be relevant both from a theoretical and a methodological perspective. The methodological implications are especially relevant for quality-of-life trials and cost-effectiveness studies. 23 Response shifts may reduce or inflate treatment effects. Moreover, differences in quality of life across treatment arms may be obscured when response shift affects the treatment groups differently. With respect to cost-effectiveness evaluations, it has been found that the valuation of health states is dependent on the health status of the patient whose preferences are being elicited. This implies that the cost-effectiveness of an intervention will depend on who is rating it. Obviously, this may have major policy implications. Given the potentially large consequences of response shift, further research into this area is indicated. 24 Clearly, sound methods for the assessment of response shift are needed. Although the thentest appears to be reasonably valid in research on the evaluation of tutorial courses, its validity in self-reported health needs to be further established (see Sprangers et al. 25 for a full discussion of the strengths and weaknesses of the thentest approach). For example, thentests, like all self-report measures, are susceptible to confounding effects, such as social desirability, cognitive dissonance, response-style effects, and recall bias. If patients undergo a medical or psychosocial intervention aimed to improve their health condition, they may feel inclined to retrospectively adjust their initial level of functioning, and as such report improvement, in order to justify their invested effort (cognitive dissonance), or to please the clinician (social desirability). We are still uncertain about the extent to which these potential confounders may operate in health-related research. It also is important to clarify the methodology that could distinguish response from recall bias. One way to proceed is to include a memory test of subjects initial responses. The concordance between the responses to the memory test, pretest and thentest can then be examined. Clearly, methods other than the thentest approach are also required to assess response shift, not only because of the limitations inherent in the thentest, but also because of the need to assess response shifts that result from changes in values and conceptualization of life quality. Such new methods have recently been suggested by Schwartz and Sprangers. 11 They describe and evaluate individualized methods, preference based methods, design approaches, statistical approaches and qualitative approaches. These authors recommend triangulation of methods as a useful direction for further research. This study gives no decisive answer as to whether the observed mean difference between conventional and thentest scores can indeed be attributed to a shift in internal standard of judgement. The limitations of this study do not warrant such a firm conclusion. First, a considerable number of patients remained stable over time with respect to reported fatigue (n 57). Consequently, the selection of the patient sample (heterogeneous with respect to diagnoses and stage of disease) and the choice of the intervention (i.e., radiotherapy with curative intent) may have been suboptimal in inducing the change in fatigue needed for a response-shift to occur, at least in a considerable number of patients. Second, in this study the thentest was used to get an estimate of the treatment effect, corrected for response shift. However, as indicated, the usefulness and validity of assessing changes in internal standards in health-related quality of life have yet to be critically examined. Third, because a gold standard for the level of fatigue does not exist, given the definition thereof, this study lacked an objective measurement of change in fatigue. Only a self-reported transition score was used. Moreover, the question concerning change was posed immediately following the thentest. Awareness of a difference between posttest and thentest ratings might have led some patients to the determination that their fatigue had either improved or deteriorated over time. Finally, in this study, a oneitem numerical rating scale for the assessment of fatigue was used. Generally, this is not the recommended way to assess fatigue. Rather, a more comprehensive instrument should be used to do justice to the complexity of this symptom. However, to have patients provide a

18 Visser et al. Vol. 20 No. 1 July 2000 retrospective judgment on each item of such an instrument was considered a too time consuming task within the context of the larger study. To conclude, research involving the concept and measurement of response shift is in its infancy, although work addressing these issues is in progress. 9,20,21 Future empirical research has to demonstrate whether the concept may be helpful to further our understanding of change and adaptation in chronically ill patients. Acknowledgment This study was funded by the Dutch Cancer Society. References 1. Smets EMA, Garssen B, Schuster-Uitterhoeve ALJ, de Haes JCJM. Fatigue in cancer patients. Br J Cancer 1993;68:220 224. 2. McCaffery M. Nursing management of the patient with pain. Philadelphia, PA: Lippincot, 1979. 3. Piper BF, Lindsey AM, Dodd MJ, Ferketich S, Paul SM, Weller S. The development of an instrument to measure the subjective dimension of fatigue. In: Funk SG, Tornquist EM, Campagne MT, Archer Gopp L, Wiese RA, editors. Key aspects of comfort. Management of pain fatigue and nausea. New York: Springer Publishing Company, 1989. 4. Smets EMA, Garssen B, Bonke B, de Haes JCJM. The Multidimensional Fatigue Inventory (MFI); Psychometric qualities of an instrument to assess fatigue. J Psychosom Re 1995;39:315 325. 5. Yellen SB, Cella DF, Webster K, Blendowski C, Kaplan E.. Measuring fatigue and other anemiarelated symptoms with the Functional Assessment of Cancer Therapy (FACT) measurement system. J Pain Symptom Manage 1997;13:63 74. 6. Hann DM, Jacobson PB, Azzarello LM, Martin SC, Curran SL, Fields KK, Greenberg H, Lyman G. Measurement of fatigue in cancer patients: development and validation of the Fatigue Symptom Inventory. Qual Life Res 1998;7:301 310. 7. Cronbach LJ, Furby L. How we should measure change - or should we? Psychol Bull 1970;74:68 80. 8. Howard GS, Ralph KM, Gulanick NA, Maxwell SE, Nance SW, Gerber SK. Internal validity in pretest-posttest self-report evaluations and a re-evaluation of retrospective pretests. Appl Psychol Measure 1979;3:1 23. 9. Sprangers MAG, Schwartz CE. Integrating reponse shift into health related quality of life research: a theoretical model. Soc Sci Med 1999;48: 1507 1515. 10. Breetvelt IS, van Dam FSAM. Underreporting by cancer patients: the case of response-shift. Soc Sci Med 1991;32:981 987. 11. Schwartz C, Sprangers MAG. Methodological approaches for assessing response shift in longitudinal health-related quality-of-life research. Soc Sci Med 1999;48:1531 1548. 12. Longo DL. Interferon toxicity worse in retrospect; impact on Q-TWIST? (Letter) J Clin Oncol 1998;16:3716. 13. Andrykowski MA, Brady MJ, Hunt JW. Positive psychosocial adjustment in potential bone marrow transplant recipents: cancer as a psychosocial transition. Psycho Oncology 1993;2:261 276. 14. Bach J, Tilton MC. Life satisfaction and wellbeing measures in ventilator assisted individuals with traumatic teraplegia. Arch Phys Med Rehabil 1994;75:626 632. 15. Friedland J, Renwick R, McColl M. Coping and social support as determinants of quality of life in HIV/AIDS. AIDS Care 1996;8:15 31. 16. Sneeuw KCA, Aaronson NK, Sprangers MAG, Detmar SB, Wever LDV, Schornagel JH. The value of caregiver rating in evaluating the quality of life of patients with cancer. J Clin Oncol 1996;15:1206 1217. 17. Bray JH, Howard GS. Methodological considerations in the evaluation of a teacher-training program. Br J Educ Psychol 1980;72:62 70. 18. Hoogstraten J. Influence of objective measures on self-reports in a retrospective pretest-posttest design. J Exp Edu 1985;53:207 210. 19. Sprangers MAG. Response shift and the retrospective pretest: On the usefulness of retrospective pretest-posttest designs in detecting training related response shifts. Dissertation. S Gravenhage: Het instituut van Onderzoek van het Onderwijs S.V.O., 1988. 20. Norman P, Parker S. The interpretation of change in verbal reports: implications for health psychology. Psych Health 1996;11:301 314. 21. Allison PJ, Locker D, Feine JS. Quality of life: a dynamic construct. Soc Sci Med 1997;45:221 230. 22. Sprangers MAG. Response-shift bias: a challenge to the assesment of patients quality of life in cancer clinical trials. Cancer Treat Rev 1996;22:55 62. 23. Smets EMA, Visser MRM, Willems AFMN, Garssen B, Oldenburger F, van Tienhoven G, de Haes JCJM. Fatigue and Radiotherapy: (a) experience in patients undergoing treatment. Br J Cancer 1998; 78:899 906. 24. Sprangers MAG, Schwartz CE. The challenge of response shift for quality-of-life-based Clinical Oncology research.(editoral) Ann Oncol 1999;10:747 749. 25. Sprangers MAG, van Dam FSAM, Broersen J, Lodder L, Wever LDV, Visser MRM, Oosterveld P, Smets EMA. Acta Oncol 1999;38:709 718.