Clustering Rasch Results: A Novel Method for Developing Rheumatoid Arthritis States for Use in Valuation Studiesvhe_

Transcription

1 Volume 13 Number VALUE IN HEALTH Clustering Rasch Results: A Novel Method for Developing Rheumatoid Arthritis States for Use in Valuation Studiesvhe_ Helen M. McTaggart-Cowan, PhD, John E. Brazier, PhD, Aki Tsuchiya, PhD University of Sheffield, Sheffield, UK ABSTRACT Purpose: Health states that describe an investigated condition are a crucial component of valuation studies. The health states need to be distinct and comprehensible to those who appraise them. The objective of this study was to describe a novel application of Rasch and cluster analyses in the development of three rheumatoid arthritis health states. Methods: The Stanford Health Assessment Questionnaire (HAQ) was subjected to Rasch analysis to select the items that best represent disability. K-means cluster analysis produced health states with the levels of the selected items. The pain and discomfort dimension from the EuroQol-5D was also incorporated. Results: The results demonstrate a methodology for reducing a dataset containing individual disease-specific scores to generate health states. The four selected HAQ items were bending down, climbing steps, lifting a cup to your mouth, and standing up from a chair. Conclusions: The combined use of Rasch and k-means cluster analysis has proved to be an effective technique for identifying the most important items and levels for the construction of health states. Keywords: cluster analysis, health state, quality of life, Rasch analysis, rheumatoid arthritis. Introduction Economic evaluation plays a significant role in guiding healthcare resource allocation decisions. For this reason, many health technology assessment organizations, including the National Institute for Health and Clinical Excellence (NICE), have proposed guidelines for the proper conduct of economic evaluations. Specifically, NICE guidelines state that the preferred methodology is cost-effectiveness analysis, in which the benefits of health interventions are quantified using quality-adjusted life-years (QALYs) rather than units specific to the condition under investigation [1]. The QALY describes an individual s preference for a health state by capturing quantity and quality of life (QOL) into a single summary measure. In making decisions regarding the allocation of health-care resources, there is a normative debate as to who should be providing values for health states. The current recommendation is that these values be obtained from the general population rather than patients [2]. This follows the concept that in a publicly funded health-care system, the main objective is to meet societal preferences for maximizing health. The drawback, however, is that the general population, when appraising the impaired health state, do not consider that they can adapt to the condition over time [3]; this can result in significant ramifications when these values are incorporated into a cost-effectiveness analysis [4]. Therefore, to test the effect of providing disease adaptation information to general population respondents, health state descriptions need to be developed. The states need to both accurately portray symptoms a patient experiences and be comprehensible for respondents who may not be knowledgeable about the condition under investigation. In addition, the health states need to demonstrate detriment in some of the item levels to Address correspondence to: Helen M. McTaggart-Cowan, School of Health and Related Research, University of Sheffield, Regent Court, 30 Regent Street, Sheffield S1 4DA, UK. h.m.cowan@sheffield.ac.uk; hmctc@hotmail.com /j x enable evaluation of the influence of disease adaptation information on general population values. If the health states do not demonstrate any detriment in at least one item level, then the descriptions are effectively synonymous to full health. Respondents will not need to consider the concept of adaptation in their valuation of this full health state. In this case, the impact of disease adaptation information on the value of this particular health state is non-existent. Health states can be developed using various methods. Expert judgments, such as those from physicians, permit a range of patient experiences to be elicited; however, these opinions are subjected to biases. Physicians viewpoints of health states may be distorted if their patients exaggerate their QOL in an attempt to please their doctor or family members. Patient responses, on the other hand, provide direct information about how the investigated health state impacts their lives. The use of interviews and focus groups allows researchers to gain in-depth knowledge about life in different health states, but the results may be subject to volunteer bias, as the small number of participants might not be representative of most patients. As such, using responses on condition-specific instruments from a large sample of patients may be a better alternative in the development of health states. Condition-specific instruments are comprised of numerous items addressing different aspects of the disease under investigation. Nevertheless, for patients responses on condition-specific instruments to be of use in the construction of health states, the number of items in the instrument needs to be minimized; this reduces the burden placed on the respondents when completing the valuation exercise. In addition, responses on conditionspecific instruments need to be combined in such a way that patients with similar QOL are grouped together so that distinct health states are formed. One method that identifies the most representative items of condition-specific instruments is Rasch analysis. It is a technique that converts categorical responses into a continuous latent scale [5,6]. It has been employed in the development of QOL instruments [7], and, more recently, in the construction of a health state classification systems [8,9]. Although Rasch analysis is not the 2010, International Society for Pharmacoeconomics and Outcomes Research (ISPOR) /10/

2 788 McTaggart-Cowan et al. only method for selecting maximally discriminatory items, this technique was chosen for this study because it offers a systematic approach to identify items while making the best use of the richness and sensitivity of the original instrument. As Rasch analysis identifies the instrument s most meaningful items, the cognitive and time constraints placed on respondents are minimized when they are appraising the health states. Furthermore, items can be ranked from easiest to the most difficult; this ensures that the health states capture the widest range of severity. Rasch analysis aims to create a system that is ordered, additive, and of specific objectivity. Although Rasch analysis selects the representative items of the condition-specific instrument, there is still a need to identify different combinations of levels of the selected items to form distinct health states. A technique that meets this objective is cluster analysis. The main purpose of this statistical approach is to group similar objects together; for example, individuals within the same cluster are more similar to each other than individuals from different clusters. One of the first studies that employed cluster analysis to develop health states did so by identifying patterns in the physical and mental health domains of the Medical Outcomes SF-12 questionnaire [10]; other studies using cluster analysis have followed (e.g., [11,12]). By applying cluster analysis to the Rasch-reduced instrument, natural groupings in the dataset can be identified, thereby forming plausible health states. This article aims to describe the combination of Rasch and cluster analyses alongside classical psychometric methods to develop health states. In this work, the Stanford Health Assessment Questionnaire (HAQ) [13], a condition-specific measure, and the EuroQol (EQ-5D) [14] were used to construct rheumatoid arthritis (RA) health states. This process is the first stage of a two-part procedure in the assessment of informed general population values. We used Rasch analysis to select items from responses on the condition-specific HAQ instrument, and then used cluster analysis to group the identified item levels to form health states. In this article, we do not claim that the joint use of Rasch and cluster analyses is the definitive method for developing health states; rather, we attempt to shed light on its potential as a novel approach to doing so. The second stage of this project is a quasi-experiment to determine whether individuals alter their values for the constructed health states after being given information about the ability of patients to adapt to their disease. The results from the second stage are beyond the scope of the current work and will be presented in a future article. Methods The HAQ RA afflicts 0.8% of the United Kingdom population [15]. It is a chronic autoimmune inflammatory disorder, which results in upper- and lower-limb disability and discomfort. Although treatments are improving, RA can pose a burden on its patients by reducing their QOL. As such, physicians often use conditionspecific instruments such as the HAQ to assess their patients [14]. Although there are other condition-specific instruments that also assess the disability that a RA patient faces, the HAQ was chosen because it has been administered in various rheumatic populations for nearly three decades. The instrument is widely used in clinical trials to evaluate RA outcomes [16] and is shown to be valid [17] and responsive [18]. Despite its popularity, the complete HAQ which also includes questions on symptoms, medication use, and medical history is lengthy. The HAQ is also known to have a floor effect, such that severely disabled individuals could be represented by scores implying minimal disability [19]. The component of the HAQ of particular relevance to this study contains 20 items that assesses an individual s ability to complete daily tasks in the following domains: dressing and grooming, arising, eating, walking, personal hygiene, reach, grip, and other activities. Two or three items comprise each domain (Table 1). Each item has four levels: no, some, or much difficulty performing the task, or an inability to perform the task. Respondents can select a score between zero and three, with higher scores implying a greater disability. The score on an individual item is increased by another point when the respondent requires assistive devices or additional help. The greatest item score yields the domain score. The averaged scores from the eight domains form an overall HAQ score. Dataset Rasch analysis is sensitive to large sample sizes; using too large a sample generates a greater frequency of statistically significant items [6], making item reduction difficult. Nevertheless, the literature reports that sample sizes in the range of 400 to 700 have been used successfully [20,21]. For this study, an anonymized dataset from the National Data Bank for Rheumatic Diseases (NDB) was used. The NDB is a non-profit organization that contains longitudinal outcomes research data from rheumatology patients [22]. The dataset contained information from a stratified sample of 600 RA patients. An equal number of individuals in three severity ranges (i.e., overall HAQ scores of <1, 1 2, >2) was used to ensure that each disability level was well represented. Initial Criteria for Reducing the HAQ Although Rasch analysis was the main approach in the selection of most representative HAQ items, psychometric methods were conducted simultaneously to ensure greater strength in the results [8]. The frequency and the internal consistency the correlation between item and domain scores were evaluated for the HAQ responses [10]. If some items elicited poor responses (i.e., low frequency) or poor internal consistency (i.e., weak correlation), they were considered to be less representative of disability than other HAQ items. As has been done in previous work, questions pertaining to the use of assistive devices were excluded from the analysis [19,23]. The research team decided a priori that three states composed of five items needed to be constructed. The developed states had to crudely define a form of mild, moderate, and extreme RA that patients may experience. The decision to have five items to describe the states was based on the fact that previous studies have shown that five-itemed instruments do not overburden respondents [14,24]. Although it may seem like a limitation to not have the chosen statistical methods identify the ideal number of health states and the number of items to be included in the descriptions, restricting the number of items and levels is normal practice in valuation studies. For this study, we wanted to use the health states in our subsequent valuation study; therefore, specific requirements as to what a successful product needed to look like had to be met. Nevertheless, if rigid criteria were not needed in the formation of health states, then the chosen statistical methods could be applied to ensure that the developed health states were entirely data driven. Selection of HAQ Items That Best Describe Disability Rasch analysis. Rasch analysis verifies that the scale of the instrument is unidimensional, a fundamental requirement of

3 RA States Developed by Rasch and Cluster Analyses 789 Table 1 Summary of the 20 functioning items of the HAQ and summary of eliminated items Failed to meet Rasch goodness-of-fit criteria DIF characteristic and split Question Domain Variable name Item level collapsed Are you able to dress yourself, including tying shoelaces and doing buttons? Dressing and grooming Dress With much difficulty and unable to do Are you able to shampoo your hair? Dressing and grooming Shampoo With much difficulty and unable to do Are you able to stand up from a straight and armless chair? Arising Standup Are you able to get in and out of bed? Arising Bed With much difficulty and unable to do Are you able to cut your meat? Eating Cutmeat Failed Are you able to lift a full cup or glass to your mouth? Eating Liftcup Are you able to open a new milk carton? Eating Openmilk With much difficulty and unable to do Are you able to walk outdoors on flat ground? Walking Walk Failed Are you able to climb up five steps? Walking Climb Are you able to wash and dry your body? Hygiene Wash With much difficulty and unable to do Are you able to take a tub bath? Hygiene Tubbath With much difficulty and unable to do Are you able to get on and off the toilet? Hygiene Toilet With much difficulty and unable to do Reach Overhead With much difficulty and unable to do Are you able to reach and get down a 5-pound object (such as a bag of sugar) from just above your head? Are you able to bend down to pick up clothing from the floor? Reach Benddown Are you able to open car doors? Grip Cardoor Failed Are you able to open jars which have been previously opened? Grip Openjars Male vs. female Failed Are you able to turn faucets on and off? Grip Faucets With much difficulty and unable to do Are you able to run errands and shop? Activities Errands Are you able to get in and out of a car? Activities Car Are you able to do chores such as vacuuming or yardwork? Activities Chores With much difficulty and unable to do HAQ, Health Assessment Questionnaire. construct validity. Unidimensionality ensures that the overall score of the instrument is describing what is actually happening and is not diluted by items that are insensitive to the underlying construct of the instrument [25]. The Rasch approach converts an instrument that uses a categorical item response system into a continuous scale by using a logit model; for example, when an instrument that measures QOL using a response scale is subjected to Rasch analysis, the resulting scale is interpreted to be a continuous measure of QOL. When the Rasch model is fitted to responses to a condition-specific instrument, the following inferences can be made [23]: 1) the easier the item is, the more likely it will be affirmed by the respondent; and 2) the more able the respondent, the more likely he/she will affirm an item compared with a less able respondent. Rasch analysis deconstructs each item of the instrument into its component steps: in the HAQ, from zero to one, from one to two, and from two to three. It then examines the likelihood of individuals successfully attaining each item level. This gives an estimate of item difficulty, which is then used to assess person ability. The Rasch model assumes that the probability of a given patient affirming an item or task is a logistic function of the relative distance between the item location parameter (task difficulty, b i) and the respondent location parameter (person ability, q): θ bi e pi( θ)= + θ 1 e ( ) ( bi ), (1) where p i(q) is the probability that patients with ability q will be able to perform item (task) i. The Rasch analysis then seeks to combine person ability and item difficulty by taking the difference between q and b i. This difference governs the likelihood of what is supposed to happen when a person of given ability uses that ability against a given task [23]. The Rasch transformation, which is reported in logits, converts the discrete items onto a continuous scale based on the natural logarithm. The relationship between person ability and item difficulty can be best understood by the fact that, for example, a person with a logit score of 2.0 (q) will have an equal probability p i(q) of affirming or not affirming, or a step on an item, with a difficulty level of 2.0 logits (b). The overall goodness-of-fit test statistic measured in terms of item-trait interaction, person separation index (PSI), and fit residuals describes how well the Rasch model fits the original data [9,10]. Item-trait interaction measures whether the data fits the Rasch model for the given respondent group. PSI calculates the level of agreement between respondents, whereas fit residuals estimate the degree of divergence between the expected and observed responses for each respondent or item response. Fit residuals are summed over all items for a given person (person fit residuals) or over all persons for a given item (item fit residuals). RUMM2010 [26] was used for all Rasch analysis. Conducting Rasch Analysis Step I: Establish whether the HAQ domains fit the Rasch model. The first step was to assess which of the 20 HAQ items best represented disability by checking if each domain fitted the Rasch model. Each of the eight domains was individually fitted to a Rasch model, and the resulting goodness-of-fit test statistics for each model were examined. This preliminary step is considered to be more crucial for instruments with domains consisting of numerous items. For this reason, less emphasis was placed on these results as there were only, at most, three items per HAQ domain. Because of the small number of items per domain for this instrument, reducing HAQ items based on this criterion

4 790 McTaggart-Cowan et al. would prove to be premature; nevertheless, this step was included to provide a complete picture of the analysis process. Step II: Short-list the items. The second step was to examine the threshold probability curves of all items; this provided a way to identify items that respondents were unable to distinguish between item-response levels. These curves show the distribution of the item levels across latent space; examples are shown in Figure 1. Although the HAQ has four response levels, three curves are used to demonstrate the thresholds between item levels (e.g., from item level zero to item level one, etc.). The horizontal and vertical axes represent the underlying latent scale and the probability of being in a particular item level, respectively. Ideally, the item levels in the threshold probability curves should be appropriately ordered and spaced, and have an opportunity of occurring (i.e., not lying on the horizontal axis) as shown in Figure 1a. For an ordered item, the thresholds between item levels are the points at which each item level is equally likely to occur; disordered items indicate that the respondent was unable to distinguish between item levels. As done previously [10], formal guidelines were not used to collapse item level; instead, adjacent item levels were merged using an item-by-item approach to achieve order. When identifying condition-specific items for use in health state descriptions, an objective was to select items that respond to the full range of severity across the condition under investigation. Therefore, in cases where it was necessary to collapse levels, these items were excluded from further consideration in the health state description because they failed to respond to the full range of the severity of the condition; these items, however, were not removed from the Rasch models. The main reason for not including these items is the importance for health state classification systems to have the same interpretations across all item levels; otherwise, it can potentially be confusing for respondents at the valuation stage. Furthermore, if respondents were unable to distinguish between existing item levels then that item is not considered to be representative of the underlying domain [10]. If, after merging of the disordered levels, any of the levels for the remaining items were poorly spread (i.e., the thresholds between levels were not of approximately equal spacing when inspected visually) or had a low probability of occurring, these levels were merged with the adjacent level; this step was conducted independently for each item. Both the overall item-trait fit and the individual item fit test statistics were examined to determine the best possible model that resulted. If any of the individual items did not fit the model (i.e., had a significance level of P < 0.01 because there is a deviation between the observed and expected responses), it was excluded from any subsequent modelling, as this item did not contribute to the underlying latent scale [9]. The model with the smallest overall item-fit test statistic (i.e., largest P-value) was chosen to be the best resultant model. If any of the individual items did not contribute to the underlying latent scale (P < 0.01), they were excluded from any further modelling [9]. The process was repeated until only well-fitting items remained and the overall item-trait goodness-of-fit of the model was P 0.01 [27]. Step III: Differential item functioning. Differential item functioning (DIF) analyses examined whether item responses differed among respondent characteristics. For example, patients who were male, younger, or with fewer years of RA diagnosed were hypothesized to select less difficulty for items relating to strength than patients who were female, older, or with greater years of RA diagnosed. Those items for which it was necessary to adjust for systematic DIF across groups of respondents are limited in value a b c d Figure 1 (a) An example of appropriately ordered item levels. (b) An example of disordered item levels. (c) An example of poorly spread item levels. (d) An example of a level lying close to 0% probability. for making cross-population comparison and were therefore excluded from further considerations [10]. Item characteristic curves and item-by-characteristic analysis of variance (ANOVA) statistics assessed whether sex, age in years (i.e., <50, 50 65, and

5 RA States Developed by Rasch and Cluster Analyses 791 >65), and duration of RA in years (i.e., <10, 10 20, and >20) influenced item responses. Step IV: Final item selection. Each of the remaining items was removed one at a time based on its location on the latent disability scale. The item s position on the scale indicates its degree of disability such that a negative value indicates an item of lesser disability, and a positive value indicates an item of greater disability. The greater the distance between the maximum and minimum values, the more sensitive the disability scale is; this, of course, will depend on the other items included in the model. The item with the greatest impact on the distance between the maximum and minimum values of the remaining items was not considered for inclusion in the health state description; that is, the item that resulted in the shortest scale was removed. This process was repeated until the desired number of items remained while ensuring that the model afforded the best overall Rasch model item-trait goodness-of-fit. If these two criteria were in disagreement, another item was selected to be excluded to ensure that the resultant model produced the best overall goodness-of-fit. Forming Health States k-means cluster analysis. Once the reduced set of items from the HAQ was selected, the next step was to formulate plausible RA states. Although there are numerous types of cluster analyses, the k-means algorithm was selected as the method of choice. In cases where multiple dimensions of health are to be considered, the use of k-means cluster analysis does not require that the states be sequentially ordered. Although k-means clustering is best suited for continuous variables, we opted to use this approach with the discrete data because, after fitting the Rasch model to the HAQ, ordering along a latent continuous scale is assumed. The k-means algorithm groups n observations into k partitions or clusters by finding the centres of natural clusters in the given dataset. The algorithm starts by randomly partitioning the data points into k initial sets. Then the mean point, or centre, is calculated for each set. The algorithm then constructs a new partition by associating each point with the closest centre. The centers are recalculated for the new clusters, and the algorithm is repeated until convergence is achieved, such that the data points no longer switch clusters. This approach seeks to identify a set of groups that both minimizes within-cluster variation and maximizes between-cluster variation in a similar fashion to that of ANOVA. Although we prespecified the number of health states (clusters) for our future valuation study at three, ideally this should be determined using the given dataset. (For a number of methods to evaluate the favorable number of clusters based on the data, refer to [28].) The stability of the formed clusters was examined by running the k-means algorithm using three, four, and five clusters. By running cluster analyses on a range of cluster numbers, the optimal spread of the data can be determined by assessing whether the combination of item levels change with the increasing number of clusters. K-means cluster analysis was conducted using SPSS version 14.0 for Windows [29]. Pain and discomfort. After identifying the clusters comprised of the four HAQ items, there was a need for the health states to also include an item that pertained to pain and discomfort, a symptom commonly experienced by patients with RA. Although patients can report their level of pain experience on the HAQ, it is measured continuously on a visual analogue scale, ranging from no pain to severe pain; within the Rasch framework, discrete rather than continuous variables are modeled. As a result, we opted to include the pain and discomfort dimension from the EQ-5D in the description of the RA states. After k-means clustering, patients were stratified into each of the identified clusters. The modal response to this dimension was then incorporated into each of the clusters. Additional analysis. With the final clusters, results from one-way ANOVAs evaluated whether differences between the respondents age, RA duration, and QOL scores existed. Results Initial Psychometric Methods for Reducing the HAQ Patient responses indicated that the correlations between the item and domain scores were mostly internally consistent (rho 0.7). Three of the items (full list shown in Table 1), however, were not internally consistent: faucets (rho = 0.59), wash (rho = 0.59), and toilet (rho = 0.68). Although these internally inconsistent items suggested that they may not be representative of the domain, they were retained for Rasch modeling. Rasch Analysis Short-list the items. The first step of Rasch analysis should be to determine whether the HAQ domains fit the Rasch model. Nevertheless, this step was not carried out because Rasch analysis requires at least two items per domain, and each of the HAQ domains had at most three items. As such, the threshold probability curves were used to short-list the items. These curves indicated that all the item levels were ordered appropriately except for the most severe levels for the shampoo and tubbath items; the misfitting nature of these items has been reported elsewhere [19]. As a result, the second and third levels of these two items were merged together in subsequent Rasch models. Once again, a Rasch model was conducted on the current base model, and the threshold probability curves for all items containing four levels were reevaluated. A detailed account of the results has been reported elsewhere [30]. A summary of the items short-listed at this stage is presented in Table 2. At the end of short-listing, the following items remained: standup, cutmeat, liftcup, walk, climb, benddown, cardoor, openjars, and car. For these items, their levels were not disordered or poorly spread and did not have a low probability of occurring. Differential item functioning. Only one item, openjars, was split for DIF because of different responses to the HAQ by sex (P = 0.02) (Table 1). Final item selection. The remaining items were removed one at a time, based on the position of the item on the disability scale, until the four-item model with the longest disability scale was identified. The scale indicated that most individuals with a mild form of RA would have problems climbing up steps, as it was represented by a negative value; however, only the most severe cases would be unable to lift a cup represented by a positive value (Table 2). Using the sequential removal-reassessment process where one item was removed, and individual item test statistics was examined to determine the best model that arose cutmeat, walk, cardoor, openjars, car were removed. Hence, the final Rasch-reduced model is composed of standing up from a straight and armless chair, lifting a full cup or glass to one s mouth, climbing up five steps, and bending down to pick

6 792 McTaggart-Cowan et al. Table 2 Results for items not excluded from Rasch model validation Item Domain Location Rasch criteria Item level Residual c 2 (P-value) Spread at logit zero Responses at floor (%) Responses at ceiling (%) Other criteria Missing data (%) Correlation with domain score Standup Arising (0.56) Cutmeat Eating (0.79) Liftcup Eating (0.17) Walk Walking (0.26) Climb Walking (0.04) Benddown Reach (0.82) Cardoor Grip (0.60) Openjars Grip (0.03) Car Activities (0.31) up clothes from the floor; the threshold probability curves for these items are presented in Figure 2. K-Means Cluster Analysis Using the raw scores from the HAQ, the k-means algorithm using three, four, and five clusters demonstrated the stability of the underlying structure (Table 3): when an extra cluster is added, one of the previous clusters is split into two levels and the rest remain the same. As identified above, we were looking for three clusters to represent three different RA states for members of the general population to value in our future study. The identified three-cluster model did not suit our purpose. The first cluster of the three-cluster model demonstrated no disability in any of the item levels: as a result, one of the states would be synonymous to full health, resulting in only two impaired health states for respondents to value. For the purpose of our subsequent valuation study, three RA states were needed to evaluate the effect of disease adaptation information on general population values. If one state was deemed full health, we anticipated that there would be no influence on the health state values because the respondents would not need to be informed about Figure 2 Threshold probability curves.

7 RA States Developed by Rasch and Cluster Analyses 793 Table 3 HAQ item number disease adaptation. Thus, cluster centers two, three, and four of the four-cluster model was used to describe three states that all included a level of disability. Incorporation of Pain and Discomfort Table 4 shows the frequency of individuals in the three clusters of the four-cluster model responding to different levels of the pain and discomfort dimension of the EQ-5D. Among all severity groups, the most frequent response was moderate pain and discomfort. Nevertheless, the distribution across the three levels clearly indicates that there is a pain gradient across the three RA states. We considered it to be misleading to give all three states the same pain levels, and therefore decided to label this item as mild, moderate, and extreme pain and discomfort (Fig. 3). Respondent Characteristics of Final Health States Table 5 displays the demographic and QOL information of the NDB patients. There were no differences in age and RA duration across cluster groups (P = 0.54 and 0.07, respectively), as DIF analysis illustrated those items that were sensitive to these variables. Nevertheless, in terms of HAQ, EQ-5D, and EQ-VAS scores, the QOL measures distinguished well across the severity groups (P 0.001): a monotonic gradient was observed, such that a lower QOL was associated with more severe forms of RA. These results provided evidence that the constructed health states had the ability to discriminate between different levels of RA severity. Discussion The modal results from the k-means cluster analysis Cluster Three clusters Standup Liftcup Climb Benddown Four clusters Standup Liftcup Climb Benddown Five clusters Standup Liftcup Climb Benddown Pain and discomfort HAQ, Health Assessment Questionnaire. The results demonstrate that the combined use of Rasch and cluster analyses, alongside psychometric techniques, can create Table 4 Frequency of level responses for the pain and discomfort domain of the EQ-5D Cluster centre No pain and discomfort (%) Moderate pain and discomfort (%) Extreme pain and discomfort (%) One 58 (23.4) 183 (73.8) 7 (2.8) Two 2 (1.2) 134 (82.2) 27 (16.6) Three 2 (1.9) 60 (55.6) 46 (42.6) Four 0 26 (52.0) 24 (48.0) EQ-5D, EuroQol 5D. distinct and plausible health state descriptions for use in valuation studies. This approach allows researchers to meet prespecified needs of their study design. For example, to reduce respondent burden in the completion of complex valuation tasks, x number of states composed of n items can be constructed. Nevertheless, it must be noted that the combined use of Rasch and cluster analyses could also be used in such a way to yield a set of completely data-driven set of health states, where no such constraints exist. Rasch and cluster analyses are regarded as tools that aid in the development of health states, but they should not supersede experienced judgments. Although members of the general population (i.e., our target population for valuing the developed health states) may focus on the classic RA symptoms, such as mobility limitations (e.g., climbing steps), the purpose of the constructed states was to create a descriptive and well-rounded picture of individuals living with RA using a limited number of items. To date, there have been two other studies that have used Rasch analysis on the HAQ, although their overall objectives were different from this present study. Tennant et al. investigated the scaling of the HAQ and the fit of the data to a Rasch model [23]. Similar to the findings of the present study, the item describing lifting a full glass to one s month adequately represented the upper level of disability, such that those who have difficulty with this task, or find it impossible, have the severest form of RA. Wolfe et al. also applied Rasch analysis to the HAQ to reduce the instrument down to 10 items [19]; however, they opted to use a revised version, the HAQ-II, rather than the original instrument so their results are not directly comparable with those presented here. The HAQ-II contained new items: waiting in a line for 15 minutes, doing outside work, lifting heavy objects, and moving heavy objects. The remaining six items of the HAQ-II were getting on and off the toilet, opening car doors, walking on flat surfaces, reaching over head, standing up from a straight and armless chair and climbing five steps; of these, only standing up from a straight and armless chair and climbing five steps remained in the Rasch-reduced HAQ in the work reported here. Four of the five items in the present health states were composed of the HAQ. Although the HAQ measures pain and discomfort, these responses were not used in the development of the health states. Although Rasch analysis requires categorical variables for its modeling, the continuous representation of pain on the HAQ (i.e., rating scale) is more meaningful to physicians monitoring their patients over multiple time points. As members of the general population were to appraise the final RA states, pain values, which were measured on visual analogue scale values, may be difficult for them to evaluate. As pain is a common symptom experienced by most patients living with RA, including this domain was critical in providing an accurate description of its impact on QOL. Therefore, to ensure that the health states are both comprehensible, and, more importantly, plausible, the decision was made to include the pain and discomfort domain, but rather than using this information from the HAQ, to substitute it with the categorical pain and discomfort dimension from the EQ-5D data collected from the same patients at the same time as the HAQ data. k-means cluster analysis was conducted using the four Raschreduced HAQ items; the pain and discomfort dimension was incorporated at the final stage. The reason for treating pain and discomfort differently to those of the HAQ items was that, while both the measurement scales were categorical, the definition of the item levels differed between the two instruments. We felt that clustering items from different instruments would be problem-

8 794 McTaggart-Cowan et al. Health State 1 You have no difficulty bending down to pick up clothes from the floor. You have no difficulty climbing up 5 steps. You have no difficulty lifting a full cup or glass to your mouth. You have no difficulty standing up from a straight and armless chair. You have moderate pain and discomfort. Health State 2 You have some difficulty bending down to pick up clothes from the floor. You have some difficulty climbing up 5 steps. You have some difficulty lifting a full cup or glass to your mouth. You have some difficulty standing up from a straight and armless chair. You have moderate pain and discomfort. Health State 3 You have much difficulty bending down to pick up clothes from the floor. You have much difficulty climbing up 5 steps. You have much difficulty lifting a full cup or glass to your mouth. You have much difficulty standing up from a straight and armless chair. You have extreme pain and discomfort. Figure 3 Final health state descriptions. atic, as the k-means technique is best suited for continuous variables. Because the HAQ items were subjected to the Rasch model, ordering along a latent continuous scale was assumed; this, however, was not the case for the pain and discomfort dimension of the EQ-5D. As with any study, there are limitations; although none of them should significantly affect the findings. There is a potential for misclassification as the HAQ score was used as a proxy to determine the RA severity of the patients. The instrument is known to have a floor effect, such that the most severe patient could potentially be represented by a decent score (i.e., a good QOL). Nevertheless, across the different measures of QOL, the relationship between RA severity and QOL at the aggregate levels is in the anticipated direction, suggesting that any misclassification of severity is likely minimal. Rasch and cluster analyses tend to afford results that fit the given sample, but may be limited in terms of their external validity; combining the two approaches may exacerbate this problem. The issue of external validity of using Rasch and cluster analyses need to be explored by repeating it in another population with RA in the future. Table 5 Characteristics of the final health states Mean (SD) Minimum Maximum Mild rheumatoid arthritis (n = 163) Age* 61.9 (12.6) Duration of RA (years) 16.1 (12.3) HAQ 1.59 (0.58) EQ-5D 0.66 (0.16) EQ-5D VAS (18.10) Moderate rheumatoid arthritis (n = 108) Age* 61.2 (14.1) Duration of RA (years) 17.3 (10.5) HAQ 2.17 (0.29) EQ-5D 0.52 (0.18) EQ-5D VAS (18.32) Severe rheumatoid arthritis (n = 50) Age* 63.1 (12.4) Duration of RA (years) 20.4 (15.0) HAQ 2.51 (0.25) EQ-5D 0.39 (0.20) EQ-5D VAS (21.55) *One-way ANOVA results: F = 0.72, P = One-way ANOVA results: F = 2.35, P = One-way ANOVA results: F = , P One-way ANOVA results: F = , P One-way ANOVA results: F = 64.76, P EQ-5D, EuroQol 5D; EQ-5D VAS, EuroQol 5D visual analogue scale; HAQ, Health Assessment Questionnaire; RA, rheumatoid arthritis. Although k-means clustering is better suited for continuous variables, this approach was utilized for this study. This was because of the assumption that after fitting the Rasch model to the HAQ, the items would be ordered along a latent continuous scale. Nevertheless, it may be better to do the clustering on the latent variable scale in the future because the k-means technique assumes that the numerical difference between the item levels accurately represents the distance between them. Further work is needed to investigate this. How the pain and discomfort item was included in the health state descriptions may also be a cause for concern. We labeled this item as mild, moderate, and extreme pain and discomfort to account for the pain gradient across the identified mild, moderate, and severe RA states. Nevertheless, Table 4 demonstrates that the majority of patients in the mild state reported moderate pain and discomfort, whereas those in the moderate and severe states reported nearly equal frequencies for both moderate and extreme pain and discomfort. All three health states indicate that moderate pain and discomfort was the most commonly responded level; this lack of sensitivity may be a result of the three response levels available to assess this dimension on the EQ-5D. Conversely, in creating a pain gradient across the descriptions, the ability to represent the patients living in the identified states may be lost. It may be more accurate, for example, to describe the moderate and severe RA states as moderate-severe pain and discomfort. For this study, we opted to differentiate this item by labeling it as mild, moderate, and extreme pain and discomfort. This was in an attempt to ensure that our target population in our subsequent valuation study could distinguish between the states. Despite the potential limitations of using Rasch and cluster analyses, the objective of defining RA states for use in future valuation studies was achieved. Rasch analysis provided a means to identify items representative of the condition, while cluster analysis generated groups of item levels to form health states. Although experienced judgments were used to refine the final states, the combined use of Rasch and cluster analyses proved able to reduce responses from a condition-specific instrument to generate distinct health states. Furthermore, this approach can be applied to other studies where there are no restrictions on the number of states to be chosen. Acknowledgments The authors would like to thank Fred Wolfe for generously allowing access to the dataset from the National Data Bank for Rheumatic Diseases and for input during the health state devel-

9 RA States Developed by Rasch and Cluster Analyses 795 opment, and Tracey Young for her invaluable input during the development of the health states and comments during the manuscript preparation stage. The authors also thank the reviewers for all their helpful comments. The lead author was supported by a doctoral research award from the Canadian Institutes of Health Research. Source of financial support: No funding was received for this manuscript. References 1 National Institute for Health and Clinical Excellence. Guide to the Methods of Technology Appraisal. London: National Institute for Health and Clinical Excellence, Gold MR, Siegal JE, Russell LB, et al. Cost-Effectiveness in Health and Medicine. Oxford: Oxford University Press, Ubel PA, Loewenstein G, Jepson C. Whose quality of life? A commentary exploring discrepancies between health state evaluations of patients and the general public. Qual Life Res 2003;12: Brazier J, Akehurst R, Brennan A, et al. Should patients have a greater role in valuing health states? Appl Health Econ Health Policy 2005;4: Rasch G. Probabilistic Models for Some Intelligence and Attainment Tests. Chicago: University of Chicago Press, Tesio L. Measuring behaviours and perceptions: Rasch analysis as a tool for rehabilitation research. J Rehabil Med 2003;35: Tennent A, McKenna SP, Hagell P. Application of Rasch analysis in the development and application of quality of life instruments. Value Health 2004;7(Suppl. 1):S Young T, Yang Y, Brazier J, et al. The use of Rasch analysis as a tool in the construction of preference-based measures: the case of AQLQ. Discussion paper 07/01. Sheffield: Health Economics and Decision Science, Available from: content/1/c6/01/87/47/heds%20dp% pdf [Accessed May 24, 2010]. 9 Young T, Yang Y, Brazier JE, et al. The first stage of developing preference-based measures: constructing a health-state classification using Rasch analysis. Qual Life Res 2009;18: Sugar CA, Sturm R, Lee TL, et al. Empirically defined health states for depression from the SF-12. Health Serv Res 1998;33: Sugar CA, James GM, Lenert LA, et al. Discrete state analysis for interpretation of data from clinical trials. Med Care 2004;42: James GM, Sugar CA, Desai R, et al. A comparison of outcomes among patients with schizophrenia in two mental health systems: a health state approach. Schizophr Res 2006;86: Fries JF, Spitz P, Kraines G, et al. Measurement of patient outcome in arthritis. Arthritis Rheum 1980;23: Brooks R. EuroQol: the current state of play. Health Policy 1996;31: Symmons DPM. Looking back: rheumatoid arthritis aetiology, occurrence and mortality. Rheumatology 2005;44(Suppl. 4):iv Bruce B, Fries JF. The Stanford Health Assessment Questionnaire: a review of its history, issues, progress, and documentation. J Rheumatol 2003;30: Marra CA, Woolcott JC, Kopec JA, et al. A comparison of generic, indirect utility measures (the HUI2, HUI3, SF-6D, and the EQ-5D) and disease-specific instruments (the RAQoL and the HAQ) in rheumatoid arthritis. Soc Sci Med 2005;60: Marra CA, Rashidi AA, Guh D, et al. Are indirect utility measures reliable and responsive in rheumatoid arthritis patients? Qual Life Res 2005;14: Wolfe F, Michaud K, Pincus T. Development and validation of the Health Assessment Questionnaire II. A revised version of the Health Assessment Questionnaire. Arthritis Rheum 2004;50: Uprichard S, Kupshik G, Pine K, et al. Dynamic assessment of learning ability improves outcome prediction following acquired brain injury. Brain Inj 2009;23: Lundström M, Pesudovs K. Catequest-9SF patient outcomes questionnaire: nine-item short-form Rasch-scaled revision of the Catquest questionnaire. J Cataract Refract Surg 2009;35: National Data Bank for Rheumatic Diseases. NDB frequently asked questions [Online]. n.d. Available from: arthritis-research.org/ndb_faq.htm [Accessed November 15, 2007]. 23 Tennant A, Hillman M, Fear J, et al. Are we making the most of the Stanford Health Assessment Questionnaire? Br J Rheumatol 1996;35: Yang Y, Tsuchiya A, Brazier J, et al. Estimating a preferencebased single index from the Asthma Quality of Life Questionnaire (AQLQ). Discussion paper 07/02. Sheffield: Health Economics and Decision Science, Streiner D, Norman G. Measurement Scales. Oxford: Oxford University Press, Rasch Unidimensional Measurement Models (RUMM) RUMM Laboratory Pty Ltd Kubinger KD. Psychological test calibration using Rasch model some critical suggestions on traditional approach. Int J Test 2005;5: Sugar C, James G. Finding the number of clusters in a dataset. J Am Stat Assoc 2003;98: SPSS for Windows. Release Chicago: SPSS Inc., McTaggart-Cowan H, Brazier J, Tsuchiya A. Combining Rasch and cluster analyses: a novel method for developing rheumatoid arthritis states for use in valuation studies. Discussion paper 08/15. Sheffield: Health Economics and Decision Science, Available from: HEDS%20DP% pdf [Accessed May 24, 2010].