Reliability of a Chronic Fatigue Syndrome Questionnaire

Transcription

1 Reliability of a Chronic Fatigue Syndrome Questionnaire Caroline Hawk, PhD Leonard A. Jason, PhD Susan Torres-Harding, PhD ABSTRACT. Background: A diagnostic instrument, the CFS Questionnaire, was developed for clinicians and researchers to administer to their patients as a screening instrument for CFS. The CFS Questionnaire is comprehensive, covering the inclusionary and exclusionary self-report criteria of the current U.S. case definition (4). The instrument also assesses past and current activity levels, and symptoms of post-exertional malaise to ensure these items are adequately assessed. Objectives: The goal of the present study was to evaluate the diagnostic reliability of an experimental measure for assessing chronic fatigue syndrome (CFS). Methods: This instrument was administered to 15 persons with CFS, 15 persons with major depressive disorder (MDD), and 15 controls. Using the Fukuda et al. (4) diagnostic criteria, raters independently reviewed participants CFS Questionnaire responses and rated whether each study participant met criteria for chronic fatigue syndrome. Results: This instrument demonstrated good inter-rater reliability. Further, this instrument demonstrated adequate classification accuracy, with a 9.3 positive likelihood ratio and a.08 negative likelihood ratio. Overall, the CFS Questionnaire demonstrated good test-retest reliability, with Caroline Hawk is affiliated with Hines VA Hospital, Spinal Cord Injury Service (128), P.O. Box 5000, Hines, IL Leonard A. Jason and Susan Torres-Harding are affiliated with Center for Community Research, 990 West Fullerton, Room 3100, Chicago, IL. Address correspondence to: Leonard A. Jason, 990 West Fullerton, Suite 3100, Chicago, IL ( ljason@depaul.edu). Journal of Chronic Fatigue Syndrome, Vol. 13(4) 2006 Available online at by The Haworth Press, Inc. All rights reserved. doi: /j092v13n04_05 41

2 42 JOURNAL OF CHRONIC FATIGUE SYNDROME intra-class correlation coefficients and kappa coefficients at.70 or higher for most items. Lower test-retest reliability coefficients were found for some items assessing temporal symptoms or items requiring an estimate of time. Conclusion: The present study suggests that the CFS Questionnaire is a reliable diagnostic tool. Use of the CFS Questionnaire should promote higher levels of diagnostic reliability because it allows for accurate classification of individuals with CFS. doi: /j092v13n04_05 [Article copies available for a fee from The Haworth Document Delivery Service: HAWORTH. address: <docdelivery@haworthpress.com> Website: < by The Haworth Press, Inc. All rights reserved.] KEYWORDS. Chronic fatigue syndrome, depression, symptomatology, diagnostic criteria INTRODUCTION When diagnostic categories lack reliability and accuracy, the validity (i.e., usefulness) of a diagnostic category is inherently limited by its reliability. Therefore, to the extent to which a diagnostic category is unreliable, a limit is placed on its validity for any clinical research or administrative use (1). The reliability of clinical diagnosis is also crucial when conducting assessment studies. If there is limited reliability of the diagnostic groups studied, the results of any study using such diagnostic categories are likely to be unreliable or invalid. Issues concerning reliability of clinical diagnosis, therefore, have important research and practical implications. Low reliability of routine diagnostic procedures is a critical problem in the assessment of fatigue, affecting both clinical work and research efforts to improve the management, treatment, and care of patients (2). The assessment and diagnosis of chronic fatigue syndrome (CFS) is a difficult and complex process that is complicated by a number of factors. Perhaps one of the greatest challenges clinicians face when considering a diagnosis of CFS is the vague wording of the diagnostic criteria and the lack of operational definitions and guidelines (3). An additional challenge that clinicians face is the absence of objective clinical or laboratory diagnostic markers for CFS. Without clear diagnostic markers, case identification for CFS depends primarily on information obtained through clinical interviews (4). Together, these two challenges create circumstances that are likely to produce low levels of diagnostic reliability.

3 Original Research 43 When evaluating CFS, it is important to consider sources of diagnostic unreliability, technically known as variance, that lead to disagreement among clinicians regarding diagnostic decisions (4). Sources of variance can be divided into the following five categories: subject variance, occasion variance, information variance, observation variance, and criterion variance (5). Subject variance occurs when patients have different conditions at different times. For example, a patient may have acute alcohol intoxication on admission to the hospital but develop delirium tremens several days later. Occasion variance occurs when patients are in different stages of the same condition at different times. An example would be a patient with multiple sclerosis who was in remission during one period of illness and symptomatic during another. Information variance occurs when clinicians have different sources of information about their patients. For example, one clinician may regularly question patients about areas of functioning and symptoms about which another does not. Observation variance occurs when clinicians presented with the same stimuli differ in what they notice. An example would be disagreement among clinicians as to whether a patient is irritable. Criterion variance occurs when there are differences in the formal inclusion and exclusion criteria that clinicians use in reaching diagnostic conclusions. An example would be disagreements as to whether difficulty concentrating is necessary for the diagnosis of fatigue. Studies have shown that criterion variance, differences in the formal inclusion and exclusion criteria used by clinicians to classify patients data into diagnostic categories, accounts for the largest source of diagnostic unreliability (5). Therefore, improvement in diagnostic reliability is primarily dependent on reducing criterion variance as a source of unreliability. Criterion variance is most likely to occur when operationally explicit criteria do not exist for diagnostic categories (1). Diagnostic criteria should specify what diagnostic instrument to use, which informants to use, and how to rate for presence and severity of the criteria (6). For example, one needs to specify that a certain number and type of symptoms should be present in order to make a particular diagnosis. In addition to the importance of the number and type of symptoms, definitions of fatigue should also include specific guidelines pertaining to the importance of symptom severity in the diagnostic procedure. Given the high variability in symptom severity among persons with fatigue, standardized procedures should be employed to determine whether or not a particular symptom is severe enough to qualify as occurrence of one of symptoms required for the diagnosis of CFS.

4 44 JOURNAL OF CHRONIC FATIGUE SYNDROME Standardized instruments should ideally have adequate sensitivity and specificity in order to accurately rule in or rule out CFS. Sensitivity refers to a positive result on a diagnostic test when a condition is present and specificity refers to a negative result on a test when a condition is not present (7). For a measure to be useful in aiding diagnosis of a condition, both sensitivity and specificity should be present in order to accurately detect a condition when it is present, and to minimize misclassifications of individuals without a condition receiving a positive result (false positive) and individuals with a condition receiving a negative result (false negative).the likelihood ratio combines information regarding test sensitivity and specificity to summarize how many times more likely patients with the disease are to have the result than patients without the disease (8). While there have been few scales developed to measure symptoms of CFS, several scales have been developed to measure the construct of fatigue. Friedberg and Jason (9) reviewed some of the more common and well-validated fatigue rating scales, and more recently, Dittner, Wessely, and Brown (10) provided a comprehensive review of 30 published fatigue scales. Clearly, at this time, there is no gold standard of fatigue severity available. Friedberg and Jason (9) found that most fatigue scales measure fatigue intensity and fatigue/function; whereas other fatigue scales measure affect constructs. Wagner et al. (11) developed a CDC CFS Symptom Inventory to assess the presence and severity of the symptomatology required for the definition of CFS. The CDC CFS Symptom Inventory consisted of 19 items assessing fatigue and illness-related symptoms during the previous month. These symptoms include the eight definitional symptoms and other symptoms believed to be associated with CFS, including diarrhea, fever, chills, sleeping problems, nausea, stomach or abdominal pain, sinus or nasal problem, shortness of breath, sensitivity to light, and depression. Individuals rated the presence, frequency, and intensity of these symptoms. A Case Definition score and Other Symptoms score were derived by multiplying the frequency by intensity score on each subscale. Wagner et al. (11) found that this instrument had good convergent validity, as defined by correlations with the Multidimensional Fatigue Inventory, Chalder Fatigue scale, and SF-36 subscales; and also found that this instrument discriminated between the subgroups classified as to fatigue status (CFS, not fatigued, unexplained fatigue not meeting CFS criteria, and CFS in remission). Regarding reliability, a Cronbach s alpha coefficient of.88 indicated high internal consistency. While this instrument appears to be useful in assessing the symptoms of CFS, the goal of the study was to test the psychometric properties of the instrument, not to evaluate

5 Original Research 45 the sensitivity and specificity of this illness when diagnosing CFS. So, while it is likely that this instrument may aid in the accurate diagnosis of fatiguing illnesses, it is unknown whether this instrument would lead to higher diagnostic accuracy when classifying people with CFS in a clinical setting. Jason, Ropacki et al. (12) conducted a study assessing the reliability of a CFS Screening Questionnaire for epidemiological research. Four groups (i.e., CFS, lupus, multiple sclerosis, and healthy controls), each containing 15 subjects, were recruited and interviewed twice over a two-week period of time using this screening questionnaire. The screening questionnaire included the Fatigue Scale (13), demographic questions, and questions about symptoms commonly found in individuals with CFS, such as those that conform to the Fukuda et al. (4) criteria. The total score on the Fatigue Scale achieved high interrater reliability and test-retest reliability. Reliability of the symptom questions was indicated by an interrater agreement of % and a test-retest agreement of 76-92%. However, this scale was developed and validated for use within a research population. There exists a need to update and appropriately revise this scale so that it can be used for both research and clinical purposes. The present study sought to evaluate the reliability and diagnostic utility of an experimental diagnostic instrument designed for clinicians and researchers to use when assessing patients presenting an unexplained chronic fatigue. It was hypothesized that this modified scale would be a reliable instrument for researchers and clinicians to use when assessing the presence of chronic fatigue syndrome in patients presenting with chronic fatigue. It was also hypothesized that this instrument would demonstrate adequate sensitivity, specificity, and a high positive likelihood ratio in accurately diagnosing individuals as having or not having CFS. Participants METHODS Forty-five participants (15 with CFS, 15 with major depressive disorder (MDD), and 15 healthy controls) were recruited from the greater Chicago area. Demographic characteristics of the three groups of participants were reported in King and Jason (14). There were no significant

6 46 JOURNAL OF CHRONIC FATIGUE SYNDROME differences among the groups with respect to gender, race, age, socioeconomic status (SES), education, marital status, occupation, and work status. Fifteen participants with CFS were recruited from a local CFS support group in Chicago and previous research studies conducted at DePaul University. Participants were required to have been diagnosed with a current (active) case of CFS, using the Fukuda et al. (4) diagnostic criteria, by a board-certified physician. The participants were also required to have been seen by their physician in the past year. All participants were screened with the Structured Clinical Interview for the DSM-IV (SCID) to ensure that they did not have any exclusionary psychiatric illnesses as stipulated by the Fukuda et al. (4) case definition. Individuals who reported having uncontrolled/untreated medical illnesses (e.g., anemia, diabetes) were excluded from the study. 1 Fifteen participants with a diagnosis of MDD were recruited from a local chapter of the National Depressive and Manic Depressive support group in Chicago. Participants were required to have been diagnosed with major depression by a licensed psychologist or psychiatrist. All participants were screened with the SCID-IV to ensure that they met criteria for a current (active) case of major depression and to ensure they did not have any other current psychiatric illnesses. Individuals who had other current psychiatric conditions in addition to major depression were excluded. Individuals who reported having uncontrolled/untreated medical illnesses(e.g., anemia, diabetes) were also excluded from the study. Finally, 15 healthy control participants were recruited from the greater Chicago area. Individuals who did not report any medical illnesses or who did not report any uncontrolled/untreated illnesses (e.g., anemia, diabetes) were allowed to participate. All participants were screened with the SCID-IV to ensure that they did not have any current psychiatric illnesses. Individuals with current psychiatric conditions were excluded. Raters Two licensed clinical health psychologists with expertise in diagnosing and treating psychiatric disorders and chronic health conditions were recruited to review participants responses on the CFS Questionnaire and rate each participant as either having CFS or not having CFS (see Appendix A). Each rater was currently an active practitioner with at least two years post-licensure clinical experience. Each rater was educated about the clinical features of CFS and the current U.S. case definition for CFS (4).

7 In addition, each rater was given a copy of the Fukuda et al. (4) paper. They were also familiarized with the CFS Questionnaire and given the opportunity to ask questions. Measures Original Research 47 Demographic Variables. Basic demographic data were gathered, including age, ethnicity, marital status, gender, occupation, work status, SES, and educational level. The Structured Clinical Interview for the DSM-IV (SCID). The SCID is a valid and reliable semi-structured interview guide that closely resembles a traditional psychiatric interview (15). The SCID is designed to identify current, past, and lifetime (chronic or reoccurring, current and past) diagnoses for a majority of DSM-IV, Axis I psychiatric disorders. The CFS Questionnaire. The CFS Questionnaire is a revised version of the Jason et al. (12) CFS Screening Questionnaire. The items on the revised instrument (see Appendix) were designed to assess the diagnostic criteria for CFS as specified by Fukuda et al. (4). Items 1a, 1b, 15, and 23 were designed to assess for the presence of persisting or relapsing chronic fatigue (4, p. 956). Item 2 was created to assess whether the fatigue is of new or definite onset (4, p. 956). Items 3-6 and items 9-14 were designed to assess for fatigue that is not the result of ongoing exertion; is not substantially alleviated by rest; and results in substantial reductions in previous levels of occupational, educational, social, or personal activities (4, p. 956). Items 7-8e were designed to assess for post-exertional malaise. Items 16 through 22 were based on the exclusionary criteria of the Fukuda et al. (4) CFS case definition. Items 16a and 16b were designed to assess for (1) any active medical condition that may explain the chronic fatigue... (2) Any previously diagnosed medical condition whose resolution has not been documented beyond reasonable clinical doubt and whose continued activity may explain the chronic fatiguing illness... (4, p. 955). Items 16b, 17a, and, 17b were designed to further assess whether an ongoing illness reported in item 16a was actively being treated through medication. Items 18 through 21b were designed to assess for alcohol or other substance abuse within two years before the onset of chronic fatigue or at anytime afterward (4, p. 955). Items 22a and 22b were created to assess for the presence of an eating disorder. Items and items were designed to measure the presence of the eight minor symptoms (i.e., impaired memory or concentration, sore throat, tender lymph nodes, muscle pain, multi-joint pain, new head-

8 48 JOURNAL OF CHRONIC FATIGUE SYNDROME aches, unrefreshing sleep, and post-exertion malaise) as specified by the Fukuda et al. (4) case definition. Participants were also asked to indicate whether they had a number of somatic, cognitive, and emotional symptoms commonly experienced by people with CFS. These additional symptoms (e.g., nausea, fever and chills, muscle weakness, and sensitivity to alcohol) were taken from a variety of sources, including results of studies by Hartz et al. (16) and Komaroff et al. (17) that suggested the inclusion of new symptoms in the case definition. For each symptom, participants were asked to indicate if the symptom had been present for six months or longer, if the symptom began before the onset of their fatigue or health problems, and how often (never, seldom, often/usually, or always) the symptom is experienced. Participants were also asked to rate the intensity of each symptom they endorsed on a scale of 0 to 100, where 0 = No problem and 100 = The worst problem possible. This is a numerical rating scale(nrs), which has been shown to be consistently valid measures of symptom intensity, particularly for pain intensity (18). The primary advantages of using NRS scales are that they are easy to administer and score, they have demonstrated good evidence of construct validity, the compliance with the measurement task is high, and the data can be treated as ratio data (18). Procedure All individuals were initially screened by a trained interviewer to determine if they met the inclusionary and exclusionary criteria for the group condition they were being considered for (i.e., CFS, MDD, and healthy controls). As part of this screening process, the individuals were administered the SCID-IV to assess for psychiatric conditions. Individuals who met criteria for participation were asked to complete the CFS Questionnaire. Five days later, the participants were asked to complete this questionnaire a second time. No difficulties or problems were noted during the testing administration. Ratings. Using the Fukuda et al. (4) diagnostic criteria, raters independently reviewed participants CFS Questionnaire responses. Each of the raters was blinded as to the status of each participant (CFS, Depression, or Control). After reviewing each participant s response on the CFS Questionnaire, the rater indicated whether that person met criteria for having CFS. For each case that they rated, raters were asked to indicate how confident ( not at all confident, not very confident, fairly confident, and very confident ) they were with their diagnostic decision.

9 Original Research 49 RESULTS In regard to psychiatric co-morbidity, three (20%) participants in the CFS group met the DSM-IV diagnostic criteria for dysthymia. No other current diagnoses were detectedin the CFS group. In the MDD group, al1 15 (100%) participants met DSM-IV diagnostic criteria for major depressive disorder (MDD). None of the participants in the MDD group met criteria for MDD with catatonic, melancholic, psychotic, or atypical features. Participants in the MDD group did not meet criteria for any other Axis I disorders. None of the participants in the control group met criteria for any Axis I disorder. Regarding inter-rater reliability of CFS/not CFS classifications, the Kappa coefficient of.85 (p <.001) indicated very good inter-rater reliability. After independently reviewing the participants responses on the CFS Questionnaire, the two raters correctly diagnosed 14 of the 15 CFS participantswithcfs. TheoneCFS participantwho was notratedas having CFS was correctly excluded from a CFS diagnosis by both raters. Although this participant initially met full criteria for CFS when screened for the study, the participant only reported having three Fukuda et al. (4) minor symptoms on the CFS Questionnaire, thereby excluding her from a CFS diagnosis because she was required to list at least four symptoms. A follow-up contact with this participant confirmed that she did have four minor symptoms, but that she had mistakenly reported only three symptoms on the CFS Questionnaire. Two MDD participants and one healthy control participant were diagnosed by at least one of the raters as having CFS. The raters were brought together and asked to discuss their ratings on the three misclassified cases. After reviewing their ratings with each other, the raters recognized their classification errors. For one case, a rater had failed to recognize that the fatigue was due to over-exertion, an exclusionary condition. For the other two cases, one rater failed to recognize the presence of other exclusionary conditions. The CFS Questionnaire, therefore, demonstrated 90% specificity, and 93% sensitivity. This means that the responses from the questionnaire were used to correctly identify individuals with CFS group in 90% of cases, and the questionnaire was used to correctly exclude individuals without CFS in 93% of cases. The positive likelihood ratio was 9.3, which means that the odds of a positive result were 9.3 times higher for the CFS group than for the Depression and Control groups. The negative likelihood ratio was.08, indicating that the odds of a negative result were lower for individuals in a CFS group when compared with the odds of a

10 50 JOURNAL OF CHRONIC FATIGUE SYNDROME negative result for individuals in the Depression and Control groups. Likelihood ratios above 10 and below.1 are considered strong evidence to rule in or rule out disease, respectively (8). The positive likelihood ratio, at 9.3, approaches the benchmark. Similarly, the negative likelihood ratio of.08 exceeds the benchmark. Next, test-retest reliability for the CFS Questionnaire was evaluated. For categorical data, the kappa statistic was used to rate test-retest agreement. Intraclass correlation coefficients were calculated for continuous data. Kappa coefficients and intraclass correlation coefficients for each item on the CFS Questionnaire are shown in Table 1. For some items, kappa coefficients or intraclass correlation coefficients could not be computeddue to eithervery smallnumberof responses or no responses in the patient subgroups, or because of extremely restricted data variability that violated the test statistic assumptions. Overall, most items on the CFS Questionnaire demonstrated good to very good test-retest reliability, with the majority of kappa and intraclass correlation coefficients falling above.70. A few exceptions were noted for some of the individualitems: in the CFS group, the following 12 items had lower test-retest kappa or intraclass correlation coefficients: item 2 which measured fatigue onset; item 3c which measured work-related activities for the past month; item 4c which measured time spent on social activities prior to reducing activity levels; item 5a which measures whether fatigue improves with rest; 5b which measured how long needed to rest for the fatigue to go away; item 6 which measures restriction of activity levels; item 7 which measures how physical activity affects fatigue, item 8b which measured the duration of post-exertional malaise; items 9, 10, and 11, which respectively measured amount of perceived energy, amount of expended energy, and amount of fatigue in the past 24 hours; and item13whichmeasuredamountofexpendedenergyinthepastweek. In the MDD group, items 5a, 9, 11, and 13 were found to have lower test-retest reliability. In the control group, the following six items had lower kappa or intraclass correlation coefficients: item 3a which measured household related activities for the past month, item 8c which measures post-exertional malaise, item 8e measuring the reason given for not exercising,anditems9,10,12,and14whichmeasuredtheamountofperceived energy and the amount of fatigue in the past day and past week, respectively. Test-retest analyses were also performed for the numeric rating scales for the severity of eight symptoms of the Fukuda et al. (4) CFS case definition. Overall, most items for the numeric rating scales demonstrated very good levels of test-retest reliability, with a majority of intraclass correlation

11 Original Research 51 TABLE 1. Test-Retest Percent of Agreement and Correlation Coefficients for Individual Items on the CFS Questionnaire. CFS (n = 15) Depression (n = 15) Controls (n = 15) Total Sample (N = 45) (1a) Are you currently experiencing any problems 100% 100% 100% 100% with fatigue or tiredness? a (1b) If you replied Yes to 1a: When did the fatigue 86% 100% 100% 96% begin? (2) When your problem with fatigue began, did it develop: Rapidly within 24 hours; Over 1 week; Over 1 month; Over 2-6 months; Over 7-12 months; Over 1-2 years; Longer than 2 years; Had problems with fatigue since childhood or adolescence; Not having problem with fatigue. a 67% 86% 86% 80% (3) In the past month, how many hours a week have you spent doing: Household related activities? b r =.98** r =.99** r =.63* r =.93** Social-related activities? b r =.99** r =.99** r =.95** r =.98** Work-related activities? b r =.57* r =.99** r =.96** r =.90** (4a) In the past six months, have you had to reduce the number of hours you previously spent on occupational, social, or family activities because of your health or problems with fatigue? a 86% 100% 86% 91% (4b) If you replied Yes to 4a: Which activities and by how many hours per week have you cut back? Occupational activities? b r =.97** r =.99** c r =.98** Social activities? b r =.90** r = 1** c r =.88** Family activities? b r =.99** r = 1** c r =.99** (4c ) If you replied Yes to 4b: How many hours did you used to spend on: Occupational activities? b r =.95** r =.86** c r =.85** Social activities? b r =.58 r = 99** c r =.81** Family activities? b r =.89* r = 1** c r =.95** (5a) If you rest, does your fatigue go away entirely, partially, or does rest have no effect on your fatigue? a 80% 80% 67% 76% (5b) If you replied Entirely or Partially to 5a: r =.60 r =.99** r =.95**.59** How long do you have to rest for your fatigue goes away? b (5c) Will your fatigue return if you stop resting and start doing something? a 80% 93% 80% 84%

12 52 JOURNAL OF CHRONIC FATIGUE SYNDROME TABLE 1 (continued) CFS (n = 15) Depression (n = 15) Controls (n = 15) Total Sample (N = 45) (6) Do you restrict your activity levels 86% 100% 86% 91% to avoid experiencing severe fatigue? a (7) Does physical activity make you feel: Worse; 86% 93% 93% 91% Better; Has no effect. a (8a) In the past six months, how often have you experienced a persistent or recurrent problem with post-exertional malaise?: Never; Seldom; Often or Usually; Always. a 86% 93% 80% 87% (8b) If you replied Often or Usually or Always to 8a: How long does the post-exertional malaise for?: Less than 1 hour;1-3 hrs; 4-10 hrs; hrs; More than 13 hrs (specify);more than 24 hrs. a (8c) If you replied Never or Seldom to 8a: What about if you exercise do you experience increased fatigue or a worsening of your symptoms after engaging in exercise?: No; Yes. a (8d) If you replied No to 8b: Is that because you are not exercising or does exertion just not affect your symptoms, or does it even make you feel better?: Not exercising; No effect; Feel better. a (8e) If you replied Not Exercising to 8d: Why aren t you exercising?: Not interested; No time; Would like to but cannot because of fatigue; Cannot because exercise makes symptoms worse. a (9) For the past day (past 24 hrs), please rate the amount of perceived energy you have had using a scale from 0 to 100 where 0 = No energy and 100 = Abundant energy. b (10) For the past day (past 24 hrs), please rate the amount of energy you have expended (used) using a scale from 0 to 100 where 0 = No energy and 100 = All of your available energy. b (11) For the past day (past 24 hrs), please rate the amount of fatigue you have had using a scale from 0 to 100 where 0 = No fatigue and 100 = Severe fatigue. b (12) For the past week, please rate the amount of perceived energy you have had using a scale from 0 to 100 where 0 = None of your available energy and 100 = Abundant energy. b 54% 86% 93% 78% 86% 100% 86% 91% 80% 86% 80% 82% 74% 86% 93% 84% r =.59* r =.68** r =.05 r =.73** r =.40 r = 92** r =.15 r =.50** r =.22 r =.48 r =.72** r =.84** r =.77** r =.73** r =.43 r =.81**

13 Original Research 53 (13) For the past week, please rate the amount of energy you have expended (used) using a scale from 0 to 100 where 0 = None of your available energy and 100 = All of your available energy. b (14) For the past week, please rate the amount of fatigue you have had using a scale from 0 to 100 where 0 = No fatigue and 100 = Severe fatigue. b (15) How would you describe the course of your illness/health problems?: Constantly getting worse; Constantly improving; Persisting; Relapsing & remitting; Fluctuating. a CFS (n = 15) Depression (n = 15) Controls (n = 15) Total Sample (N = 45) r =.59* r = 67** r =.82** r =.64** r =.81** r =.83** r = 53* r =.92** 93% 86% 73% 84% (16a) Do you have any known diagnosed medical 93% 100% 93% 96% conditions? a (16b) Please list medical conditions. 93% 100% 93% 96% (17a) Are you currently taking any medications? a 93% 100% 93% 96% (17b) If you replied Yes to 17a: What medications 86% 100% 93% 93% are you taking? a (18) How often do you drink alcohol?: Never; 86% 100% 93% 93% Rarely; Weekly; Daily. a (19) When you drink, how much do you typically 93% 93% 100% 96% drink? a (20a) Are you currently using 100% 93% 100% 98% recreational drugs? a (20b ) If you replied Yes to 20a: Which drugs 100% 93% 100% 98% and how often and much do you use? (21a) Have you ever used recreational drugs in 93% 100% 100% 98% the past? a (21b) If you replied Yes to 21a: Which drugs and 100% 100% 100% 100% how often and much do you use? (22a) Have you ever been diagnosed or treated 100% 100% 100% 100% for an eating disorder? a (22b) If you replied Yes to 22a: When did that problem begin? 100% 100% 100% 100% Do you still have an eating disorder? a 100% 100% 100% 100% When did the problem stop? 100% 100% 100% 100% a Categorical data b Continuous data C Could not be calculated because no control participants endorsed these items. * p <.05 ** p <.01

14 54 JOURNAL OF CHRONIC FATIGUE SYNDROME coefficients at.70 or higher (Table 2). For the numeric rating scales, the average test-retest reliability for the CFS group was.77 (range ), the average for the MDD group was.77 (range ), and the average for the controls was.88 (range ). Lowest reliabilities were found for tender/sore lymph nodes (.58 and.08 for the CFS and MDD groups, respectively), and pain in multiple joints (.49 for the CFS group). DISCUSSION The goal of this study was to evaluate the reliability and diagnostic utility of an experimental diagnostic instrument. This diagnostic instrument, the CFS Questionnaire, was designed for clinicians and researchers to use when assessing patients presenting with unexplained chronic fatigue. It is a relatively short, easy to complete self-report instrument that is based on the Fukuda et al.(4) CFS case definition. Item by item, respondents were asked to indicate whether they had the symptoms that comprise CFS, as well as any of the conditions that would exclude them from a diagnosis of CFS. In effect, by reviewing a respondent s data, a clinician and researcher can quicklygetagoodideaifthepatientmeetsthediagnosticcriteriaforcfs. Results of the present investigation demonstrated that the CFS Questionnaire is a reliable instrument. The instrument has good sensitivity in that two clinicians were able to independently identify 93% of the CFS Table 2. Test-Retest Intra-Class Correlation Coefficients for Numeric Ratings of Symptom Severity. CFS (n = 15) Depression (n = 15) Controls (n = 15) Total Sample (n = 45) Fatigue.94***.88***.72***.96*** Sore throat.83***.90***.95***.90*** Tender/Sore lymph nodes.52*.08.60*** Muscle pain.96***.94***.85***.96*** Pain in multiple joints without.49*.90***.85***.77*** swelling or redness Impaired memory & concentration.85***.81***.87***.95*** Unrefreshing sleep.93***.64**.92***.94*** Post-exertional malaise.75***.88***.92***.95*** Headaches.70***.94***.94***.87*** *Significant at the.05 level. **Significant at the.01 level. ***Significant at the.001 level.

15 Original Research 55 sample as having CFS. The positive likelihood ratio was 9.3, indicating that the odds of a person with CFS obtaining a positive result was 9.3 times higher than the odds that a person in the Depression or Control groups would obtain a positive result. It is important to note that the one CFS participant who was not identified by either rater as having CFS in fact failed to endorse at least 4 of the 8 symptoms on the CFS Questionnaire. This error in classification was the result of unreliability in self-report data. The CFS Questionnaire also demonstrated adequate specificity (90%), and the negativelikelihoodratio was.08, indicatingthat the odds of an individual with CFS obtaining a negative result was lower than the odds for the Depression and Control groups. Two depressed participants and one control participant were misdiagnosed by one of the raters as having CFS. The two raters were asked to discuss these cases in an effort to understand their diagnostic decisions and why there was disagreement on these three cases. In reviewing the data for the control participant, one rater believed the participant had CFS given that the major criteria for the case definition were met and the participant endorsed at least seven of the eight symptoms. On the other hand, the other rater, who diagnosed the participant as not having CFS, noticed that the participant was spending a total of 73 hours a week on household, social, and occupational activities. This rater, therefore, concluded that the participant s fatigue was the result of ongoing exertion, an exclusionary condition. The first rater agreed with this assessment and changed the diagnosis to not CFS. This discrepancy provides an example of observation variance or differences in the salient information that each rater used to make their diagnostic classification. Because many conditions exclude a person from having CFS (4), having a thorough knowledge of the different conditions and specific circumstances under which a person would not meet criteria for having CFS is key for accurate diagnosis. This example demonstrates the intricacies involved in diagnosing CFS and further suggests the need to use standardized measures in order to minimize observation variance. In reviewing the depressed participants who were misclassified by one rater as having CFS, another important assessment issue was identified. We believe that an additional item should be added to the CFS Questionnaire that assesses the participant s attribution for fatigue. Had such an item been on the questionnaire, it is possible that the respondents may have reported depression was the cause of their fatigue and health problems. This item is very important because individuals with depression can come very close to meeting the diagnostic criteria for CFS. An item asking respondents what they believe the cause of their fatigue is, either a

16 56 JOURNAL OF CHRONIC FATIGUE SYNDROME psychological or physical agent, might help to discriminate some depressed individuals from individuals with CFS. Indeed, one study found that persons with primary psychiatric disorders readily disclosed that they had a psychiatric condition, such as major depression, when asked (19). This lends support to the idea that, in addition to assessing the presence of symptoms, simply asking people what they feel is primarily causing their fatigue may aid in this differential diagnosis between CFS and a primary psychiatric disorder. When assessing interrater reliability, the CFS Questionnaire demonstrated very good classification accuracy, with a Kappa coefficient of.85. Test-retest reliability data also suggest that the overall instrument is a reliable measure over a short period of time. A few exceptions were noted for some of the individual items: in the CFS group, items measuring fatigue onset; work-related activities for the past month, time spent on social activities prior to reducing activity levels, whether fatigue improves with rest, how long needed to rest for the fatigue to go away; item 6 which measures restriction of activity levels, how physical activity affects fatigue, duration of post-exertional malaise, amount of daily perceived and expended energy, daily fatigue, and amount of expended energy in the past week. A common element among many of these items with lower interrater reliability is that they require an estimate of time. It is possible that people with CFS have greater difficulty producing time estimates given that many people with CFS experience a variety of cognitive symptoms that impair their memory, concentration, and cognitive functioning. It is therefore very important for clinicians to spend extra time assessing information that requires a time estimate, especially if the information directly pertains to the diagnostic criteria (e.g., onset of fatigue and duration of post-exertional malaise). The remaining four items on the CFS Questionnaire that had lower test-retest reliability in the CFS group were also found to have lower test-retest reliability in the MDD and control groups. Three items asked participants to rate on a scale of the amount of perceived energy, amount of expended energy, and amount of fatigue they experienced in the past 24 hours; and the fourth item asked participants to rate on a scale of the amount of expended energy in the last week. Interestingly, when asked to make the same ratings for the past week, test-retest reliability for amount of perceived energy and amount of fatigue improved considerably across the three groups. These findings suggest that on a day to day basis, perceived energy, expended energy, and fatigue tend to fluctuate. However, over the course of a week, greater consistency on these items can be obtained.

17 Original Research 57 The present study suggests that the CFS Questionnaire is a reliable diagnostic tool. Further, the CFS Questionnaire appears to be of clinical utility when accurately classifying individuals with either having or not having CFS. The positive likelihood ratio (9.3) and negative likelihood ratio (.08) seem to indicate that the questionnaire does demonstrate clinical utility in effectively ruling in or ruling out the diagnosis of CFS. This questionnaire, then, appears to be an important tool that would aid clinicians and researchers in assessing the presence of CFS in a structured manner with their patients and research participants. To date, few measurement tools have been developed specifically for this purpose. After testing this instrument in the present study, a few improvements were made to further refine the instrument. First, an item assessing attribution for fatigue and health problems was added in an attempt to improve the instrument s ability to distinguish between cases of CFS and primary depression. Second, the time frame for assessing changes in levels of occupational, social, and household activities was modified to ensure this item captured such changes. Finally, wording regarding the ratings of an individual s daily and weekly perceived energy, expended energy, and fatigue was also clarified. It is hoped that these changes will lead to even higher levels of reliability and diagnostic accuracy, and future studies should investigate the psychometric properties of this revised instrument with larger samples of people with chronic fatigue. In summary, the provision of standardized measures for assessment and scoring guidelines should reduce clinicians difficulty with the criteria and their need to modify the criteria in clinical practice. In effect, use of standardized measures should reduce criterion variance and improve diagnostic reliability. NOTE 1. Two individuals in the CFS group did not rate for severity several symptoms at the second assessment, so the number of participants was 13 rather than 15. REFERENCES 1. Spitzer R, Endicott J, Robins E. Research diagnostic criteria. Arch of Gen Psychiatry 1978; 35: Leckliter IN, Matarazzo JD. Diagnosis and classification. In: Van Hasselt VB, Hersen M, Eds. Advanced Abnormal Psychology. New York: Plenum Press; 1994: Jason LA, King CP, Richman J, Taylor R, Song S, Torres S. U.S. Case definition of Chronic Fatigue Syndrome: Diagnostic and theoretical issues. J Chronic Fatigue Syndr 1999; 5(3): 3-33.

18 58 JOURNAL OF CHRONIC FATIGUE SYNDROME 4. Fukuda K, Straus SE, Hickie I, Sharpe MC, Dobbins JG, Komaroff A. The chronic fatigue syndrome: A comprehensive approach to its definition and study. Ann Intern Med 1994; 121: Spitzer R, Endicott J, Robins E. Clinical criteria for psychiatric diagnosis and DSM-III. American J Psych 1975; 132: Cantwell DP. Classification of child and adolescent psychopathology. J Child Psychol Psych 1996; 37: Altman DG, Bland JM. Statistics notes: Diagnostic tests 1: sensitivity and specificity. BMJ 1994; 308: Deeks JJ, Altman DG. Statistics notes: Diagnostic tests 4: likelihood ratios. BMJ 2004; 329: Friedberg F, Jason LA. Understanding chronic fatigue syndrome: An empirical guide to assessment and treatment. Washington, DC: American Psychological Association; Dittner AJ, Wessely SC, Brown, RG. The assessment of fatigue. A practical guide for clinicians and researchers. J Psychosom Res 2004; 56: Wagner D, Nisenbaum R, Heim C, Jones JF, Unger ER, Reeves WC. Psychometric properties of the CDC symptom inventory for the assessment of chronic fatigue syndrome. Population Health Metrics 2005, 3:8. (accessed from pophealthmetrics.com/content/pdf/ pdf on August 4, 2005). 12. Jason LA, Ropacki MT, Santoro NB, Richman JA, Heatherly W, Taylor R. et al. A screening scale for Chronic Fatigue Syndrome: Reliability and validity. J Chronic Fatigue Syndr 1997; 3: Chalder T, Berelowitz G, Pawlikowska T, Watts L, Wessley S, Wright D, Wallace E. Development of a fatigue scale. J Psychosom Med 1993; 37: King C, Jason LA. Improving the diagnostic criteria and procedures for chronic fatigue syndrome. Biolog Psychol 2005; 68: First MB, Spitzer RL, Gibbon M, Williams JBW. Structured Clinical Interview for DSM-IV Axis I Disorders, Clinician Version (SCID-CV). American Psychiatric Press, Inc.: Washington, DC; Hartz A, Kuhn EM, Levine PH. Characteristics of fatigued persons associated with features of chronic fatigue syndrome. J Chronic Fatigue Syndr 1998; 4: Komaroff AL, Fagioli LR, Geiger AM, Doolittle TH, Lee J, Kornish RJ et al. An examination of the working case definition of chronic fatigue syndrome. Am J Med 1996; 100: Jensen M, Karoly P. Self-report scales and procedures for assessing pain in adults. In: Turk D, Melzack R, Eds. Handbook of Pain Assessment. The Guildford Press: New York, NY; Torres-Harding SR, Jason LA, Cane V, Carrico A, Taylor RR. Physicians diagnoses of psychiatric disorders for persons with chronic fatigue syndrome. Int J Psych Med 2002; 32: doi: /j092v13n04_05 RECEIVED: 23/05/05 ACCEPTED: 29/11/05

19 Original Research 59 APPENDIX A CFS QUESTIONNAIRE (1a) Are you currently experiencing any problems with fatigue or tiredness? No Yes (1b) Ifyou replied Yes to 1a: When did the fatigue begin? (1c) If you replied Yes to 1a: What do you think the cause of your fatigue is? (2) When your problem with fatigue began, did it develop (check one): Rapidly- within 24 hours Over 1 week Over 1 month Over 2-6 months Over 7-12 months Over 1-2 years Longer than 2 years Had problems with fatigue since childhood or adolescence N/A Not having problem with fatigue (3) In the past month, how many hours a week have you spent doing: Household related activities? Social-related activities? Work-related activities? (4a) In the past 6 months, have you had to reduce the number of hours you previously spent on occupational, social or family activities because of your health or problems with fatigue? No Yes (4b) If you replied Yes to 4 a: Which activities and by how many hours per week have you cut back? Occupational: decreased by hrs/week Social: decreased by hrs/week Family: decreased by hrs/week (4c) If you replied Yes to 4b: How many hours did you spend on: Occupational activities? Social activities? Family activities?

20 60 JOURNAL OF CHRONIC FATIGUE SYNDROME (5a) If you rest, does your fatigue go away entirely, partially, or does rest have no effect on your fatigue? (check one): Entirely Partially No effect (5b) If you replied Entirely or Partially to 5a: Howlong do you havetorestforyourfatigueentirelyorpartiallygoesaway? Will your fatigue return if you stop resting and start doing something? No Yes (6) Do you restrict your activity levels to avoid experiencing severe fatigue? No Yes (7) Does physical activity make you feel: Worse Better Has no effect (8a) In the past 6 months, how often have you experienced a persistent or recurrent problem with post-exertional malaise? By post-exertional malaise I mean do you begin to feel worse after engaging in activities that require either physical or mental exertion? Never Seldom Often or Usually Always (8b) If you replied Often or Usually or Always to 8a: How long does the post-exertional malaise for? (check one): Less than 1 hour 1-3 hrs 4-10 hrs hrs More than 13 hrs (specify how long) More than 24 hrs (8c) If you replied Never or Seldom to 8a: What about if you exercise do you experience increased fatigue or a worsening of your symptoms after engaging in exercise? No Yes (8d) If you replied No to 8c: Is that because you are not exercising or does exertion just not affect your symptoms, ordoes it even make you feel better? Not exercising No effect Feel better (8e) If you replied Not Exercising to 8d: Why aren t you exercising? Not interested No time Would like to but cannot because of fatigue Cannot because exercise makes symptoms worse

21 Original Research 61 (9) For the past day (past 24 hrs), please rate the amount of perceived energy you have had using a scale from 0 to 100 where 0 = No energy and 100 = Abundant energy (10) For the past day (past 24 hrs), please rate the amount of energy you have expended (used) using a scale from 0 to 100 where 0 = No energy and 100 = All of your available energy (11) For the past day (past 24 hrs), please rate the amount of fatigue you have had using a scale from 0 to 100 where 0 = No fatigue and 100 = Severe fatigue (12) For the past week, please rate the amount of perceived energy you have had using a scale from 0 to 100 where 0 = None of your available energy and 100 = Abundant energy (13) For the past week, please rate the amount of energy you have expended (used) using a scale from 0 to 100 where 0 = None of your available energy and 100 = all of your available energy (14) For the past week, please rate the amount of fatigue you have had using a scalefrom0to100where0=nofatigueand100=severefatigue (15) How would you describe the course of your illness/health problems (check one): Constantly getting worse Constantly improving Persisting (no change) Relapsing and remitting (having good periods with no symptoms and bad periods) Fluctuating (symptoms periodically wax and wane, but never disappear completely) (16a) Doyou have any known diagnosed medical conditions? (16b) For which these conditions are you currently receiving treatment or taking medication?

22 62 JOURNAL OF CHRONIC FATIGUE SYNDROME ( 17a) Are you currently taking any medications? No Yes (17b) If you replied Yes to 17a: What medications are you taking? (18) Howoften do you drink alcohol: Never Rarely Weekly Daily (19) When you drink, how much do you typically drink? (20a) Are you currently using recreational drugs? No Yes (20b) If you replied Yes to 20a: Which drugs and how often and much do you use? (21a) Have you ever used recreational drugs in the past? No Yes (21b) If you replied Yes to 21a: Which drugs and how often and how much do you use? (22a) Have you ever been diagnosed or treated for an eating disorder? No Yes (22b) If you replied Yes to 22a: When did that problem begin? Do you still have an eating disorders? Yes No: When did the problem stop? For the symptoms below, please indicate in the first column by placing a check ( ) those symptoms that have persisted or reoccurred during six or more consecutive months of the fatigue illness or during your health problems. In the next column, please check ( ) those symptoms that began before you started having a persistent or recurring problem with fatigue. In the third column, please indicate how often you have experienced any of the following symptoms in the past six months using these response categories: Never, seldom (about once a month or less), often or usually (occurs monthly), or always. In the last column please rate the severity of each symptom you have experienced over the past six months using a scale of 0 to 100 where 0 = no problem and 100 = the most severe problem possible.