OUTCOME ASSESSMENT IN COMPLEX REGIONAL PAIN SYNDROME

Transcription

1 OUTCOME ASSESSMENT IN COMPLEX REGIONAL PAIN SYNDROME

2 OUTCOME ASSESSMENT IN COMPLEX REGIONAL PAIN SYNDROME By TARA PACKHAM, BHSc(OT) A Thesis Submitted to the School of Graduate Studies In Partial Fulfilment of the Requirements For the Degree Master of Sciences McMaster University Copyright by, April 2011 II

3 MASTER OF SCIENCES (2011) (Rehabilitation Sciences) McMaster University Hamilton, Ontario TITLE: Outcome measurement in Complex Regional Pain Syndrome AUTHOR: SUPERVISOR: Dr. Joy MacDermid NUMBER OF PAGES: 136 III

4 Table of contents Abstract 1 Introduction. 2 Background on Complex Regional Pain Syndrome (CRPS) CRPS: the measurement challenges Clinical measurement concepts and principles Use of cognitive interviewing to inform outcome measures Development of the Hamilton Inventory for CRPS Conceptual development of an outcome measure for CRPS Initial steps of development Preface 14 Chapter One: A systematic review of outcome assessments for CRPS: describing the elephant. 15 Introduction Methods Results Discussion References Chapter Two: Do you see what I see? A cognitive debriefing study of the CB-HI-CRPS 59 Introduction Materials and Methods Results Discussion References Conclusions Outcome measurement in CRPS Progress of HI-CRPS tool Future directions for HI-CRPS References 120 Appendices: Appendix A: Evaluation tool for article quality and scoring guidelines Appendix B: CB-HI-CRPS Appendix C: User / Scoring Manual for HI-CRPS Appendix D: SR-HI-CRPS Appendix E: Cognitive Interview questions IV

5 OUTCOME ASSESSMENT IN COMPLEX REGIONAL PAIN SYNDROME Abstract: Complex regional pain syndrome is a neurological condition characterized by a constellation of variable and seemingly disparate signs and symptoms for which there is presently no definitive diagnostic test. The opportunity exists for development of a condition-specific outcome measure for complex regional pain syndrome affecting any limb(s) that could be used by therapists, physicians and researchers to evaluate their patients, make treatment decisions, and monitor the changes in both impairments and quality of life experienced by those affected individuals. This thesis addresses outcome assessment in complex regional pain syndrome, incorporating 2 papers. The first is a systematic review of the current scope and psychometric rigor of outcome assessments available to health professionals to guide their management of this condition. The second paper describes the preliminary development steps of a new measurement tool for complex regional pain syndrome, with a focus on a cognitive debriefing study of current assessment practices and preferences of a cross-section of health professionals used to inform the definitions and user manual for a multidisciplinary assessment. Finally, the thesis addresses areas for future refinement and testing of the proposed outcome measure. 1

6 Introduction and Background Complex regional pain syndrome (CRPS) is a form of neuropathic pain that may develop after trauma or surgery, although is sometimes described as occurring spontaneously (Bruehl, Harden, and Galer et al,1999; Schwartzman, Erwin and Alexander, 2009). It can be associated with a nerve injury, and the International Association for the Study of Pain (IASP) diagnostic classification emphasizes this distinction, with CRPSI being defined as occurring without injury to a major nerve, while CRPSII has an accompanying nerve trauma (Albazaz, Wong, and Homer-Vanniasinkam, 2008). While there are few population based epidemiological studies, De Mos et al (2007) reported the incidence in a Dutch cohort at 26.2 per 100,000 person years (95% CI: ), and additionally found a >3:1 ratio of women to men affected. The sample also found a higher rate of involvement in the upper extremity (59.2% versus 39.1%, p < 0.001), and reported that fractures were the most common precipitating event, accounting for 44% of cases. Pain is considered to be the key feature of the syndrome, often described as disproportionate to the injury, and spreading regionally beyond the original insult. Other symptoms may include swelling and changes in blood flow; trophic features such as changes in hair, skin and nails; motor symptoms such as stiffness, dystonia and guarding; and sensory alterations such as hypersensitivity and cold intolerance (Marinus and Van Hilten, 2006). In a follow-up study of patients seen in a pain centre with the diagnosis of complex regional pain syndrome (CRPS) within a one year period, all reported 2

7 severe pain as the first symptom, and all but one respondent (n=56) also reported weakness and swelling as early signs of the condition (Galer, Henderson, Perander and Jensen, 2000). All respondents also reported that their condition caused substantial interference in their general activity, sleep, work, mobility, recreation, social activities, mood and relationships (Galer et al, 2000). The majority of patients seen in a 2009 study (n=102; De Mos et al) reported impairments persisting for more than two years since onset, and 31% (95%CI 19-43) had been unable to return to work. CRPS- the measurement challenges Historically, there has been a strong emphasis on diagnostic criteria in the field of CRPS; the debate has continued even after IASP published a benchmark taxonomy in 1994 in an attempt to create a unifying nomenclature for research and clinical practice. This new taxonomy was intended to replace former terms such as reflex sympathetic dystrophy (RSD), algodystrophy and causalgia (Reinders, Geertzen, and Dijkstra, 2002). However, the ongoing lack of agreement, coupled with the variable clinical presentation, has resulted in many small studies that cannot be compared to each other and whose claims to validity are not evidence based (Harden et al, 2007). Further contributing to small studies are the IASP diagnostic criteria themselves, which divides CRPS into two sub-types (CRPS I and II), on the basis of whether or not there is an accompanying nerve injury. For homogeneity, many studies focus only on CRPS 3

8 I, often including just upper or just lower limbs, reducing not only the sample sizes but also the generalizability of the research to the entire population. The challenge and opportunity still exists for an outcome measure that could be applied for both CRPS I and II, and which is capable of being utilized for any patient, regardless of which limb(s) have been affected. Furthermore, even as the search continues for a pathognomonic test to diagnose CRPS, there remains a role for discriminative assessment tools which can assist in identifying those patients who will not only have the condition, but also those who will require early intervention (Harden et al, 2010). The work presented in this thesis will describe the present state of outcome assessment in CRPS, and the early development of a new outcome measure intended to address these concerns. 4

9 Clinical measurement concepts and principles Clinical assessment tools are used in rehabilitation to serve one of three purposes: 1) evaluation measuring change over time, or 2) discrimination screening or classifying individuals according to a particular trait or risk factor, or 3) prognosis classifying individuals as to whether they will respond to treatment or not (Finch, Brooks, Stratford, and Mayo, 2002). As the focus of this thesis is outcome measurement, this section will concentrate only on the measurement concepts and principles related to evaluation. Reliability can be described as a measure of the reproducibility, agreement, or degree of error inherent in any measurement or measurement tool. Finch et al (2002) represent two essential components: relative reliability, or the ability of a tool to differentiate between subjects, presented numerically as the intraclass correlation coefficient (ICC), and absolute reliability, or the measurement error intrinsic to the scale, expressed as the standard error of measurement (SEM). It encompasses several forms, including the agreement between raters (intra-rater reliability) and consistency of results with any given subject (test-retest reliability). In theory, the reliability of an assessment tool can be improved by reducing the amount of errors through standardization of administration methods, training of those who administer the assessment, and by using rating scales with sufficient precision for discrimination (Streiner and Norman, 2003; Preston and Colman, 2000). 5

10 Validity encompasses multi-faceted analyses of the veracity attributed to an assessment tool: commonly examined forms include content, criterion, and construct validity, which in turn look at how the assessment tool can be used to describe, predict, and evaluate the concepts of interest (Tickle-Degnan, 2002). The evaluation of validity can be described as an ongoing process of accumulating evidence on multiple aspects of the test (Sechrest, 2005), including content, relationships to other variables, the internal structure of the assessment (Goodwin, 2002), the response process, and consequences (Cook and Beckman, 2006). Content validity is studied by testing the comprehensiveness and/or accuracy of an assessment tool in measuring or embodying the constructs it purports to represent. Face validity is an extension of content validity: it is tested by surveying whether the intended users and/or target populations subjectively judge the tool to be reasonably representative (Hulley, Martin and Cummings, 2007). Internal consistency is sometimes described as construct validity, other times as reliability. Construct validity is the degree to which we can reasonably estimate that an assessment reflects the hypothetical attributes or behaviours of both the tool, and the constructs of interest; it should be considered an estimate, however, as these relationships are usually complex and non-linear (Sechrest, 2005). Discriminative validity is an extension of construct validity; it could be defined as the ability of a measure to discriminate between two known groups (i.e. those 6

11 who have acute symptoms and those who have chronic symptoms) this is also referred to as known group validity (Finch et al, 2002). This should be distinguished from discriminant validity, or the relationship of the tool findings about the constructs measured in contrast to the findings of another tool intended to measure different constructs within the same sample (Streiner and Norman, 2003). Criterion validity is how the findings of a tool compare to the findings of a gold standard test or widely accepted assessment in concurrent measures of the population of interest (Finch et al, 2002). Responsiveness is described as the ability to detect clinically meaningful changes in status; it can be represented by both the effect size (ES) and standardized response mean (SRM) (MacDermid, Richards, Donner, Bellamy and Roth, 2000). Use of Cognitive Interviewing to inform outcome measures With the increased focus on incorporating the perspective of the patient in health care measurement, there has been a proliferation of self-report measures, and a corresponding drive to make them more scientifically rigorous (Kayes and McPherson, 2010). Cognitive interviewing is one such qualitative method: it is used to examine how potential users interpret, retrieve relevant information to formulate a decision, and respond to survey questions (Housen et al, 2008). Traditionally, qualitative studies seeking to refine clinician based (CB) assessment tools have used a Delphi or other consensus method for validation 7

12 of the content coverage (Fink, Kosecoff, Chassin, and Brook, 1984; Brunner et al, 2008). However, consensus methods do not generally allow for an understanding of the sources of difference. The process of constructing judgements from observations and assessments within clinical decision-making has been studied both quantitatively and qualitatively within the context of diagnosis (Norman, Coblentz, Brooks and Babcock, 1992; Regher and Norman, 1996), however, a recent (albeit not exhaustive) search of the health sciences literature could not produce a descriptive or qualitative study which focused on the process of observer variation across a variety of disciplines when scoring clinical attributes or status using clinician based (CB) assessment tools. Cognitive interviewing provides an opportunity to address the process by which health professionals arrive at a judgement when making scalar decisions about clinical signs or patient characteristics, and to investigate differences between the professions. This insight can inform the assessment tool and scoring manual to ensure that multidisciplinary values and practices are best reflected to enhance consistent application of the scoring guidelines. This thesis will present the results of a cognitive debriefing study of the a portion of an outcome measure for CRPS currently in development, the Hamilton Inventory for complex regional pain syndrome. 8

13 Development of the Hamilton Inventory for Complex Regional Pain Syndrome Conceptual development of an outcome measure for CRPS In 2007, several students in the MScOT program at McMaster University undertook a literature review under the supervision of the author (TP) to identify the characteristics attributed to CRPS, in an attempt to establish the parameters for content coverage for a new multi-disciplinary assessment tool, as recommended by Hicks (2004). The goals of developing a new tool were: 1) to provide a common taxonomic and assessment framework for therapists and physicians to evaluate their patients, make treatment decisions, and monitor progress; 2) to foster research by developing an inclusive tool that would allow patients with CRPSI or CRPSII, involving any extremity to be compared to each other; 3) to employ a broad framework which included impairment, activity and participation perspectives but was condition-specific; 4) to develop a reliable and valid tool that would yield useful and durable information for both research and clinical practice; and 5) potentially to identify patterns of symptoms, and thus discriminate between patient groups not on the basis of chronicity or nerve injury, but on the basis of functional symptom clusters. Interviews were also conducted with local champions in CRPS care, and with patient stakeholders to informally gather their input. An initial draft was developed which contained 99 items divided into two sections: a 44 item assessment to be completed by a health professional (covering the content areas of sensory symptoms, autonomic 9

14 function, trophic changes and motor function) and a self-report section containing 55 items addressing quality of life in three domains: physical symptoms, participation in daily activities, and socio-emotional impact. The tool was named the Hamilton Inventory for Complex Regional Pain Syndrome (HI-CRPS). Initial steps of development The first major refinement undertaken by the author was the revision of the scoring system. Many items were initially scored categorically, limiting the discriminative ability of the tool (Streiner and Norman, 2003; p. 31). Thus all a seven-point scale (0-6) was developed for all items, with a higher score indicating greater symptom severity. The seven-point scale was chosen for its anticipated contribution to reliability, validity and ease of completion (Preston and Colman, 2000). A combination of Likert and end-anchored adjectival forms were used, as incorporating both formats allowed easy ranking of both performance and attitudinal attributes. Ambiguous and double-barrelled items were also rewritten, and some questions were redistributed to minimize their potential influence on adjacent items. It also became apparent that the 17 pain descriptors included in the symptoms subscale added unintentional weighting to that scale relative to the other subscales (Steiner and Norman, 2003; p. 104). Consequently, a group of 19 patients with CRPS of either the upper or lower extremity were recruited to assist in reducing the number of pain descriptors included in the symptom subscale of Section B. From the initial list of 17 descriptors, patients were asked 10

15 to select the 7 words that they felt best described the pain they experienced with CRPS; the seven words selected most frequently were then retained for the questionnaire, and the remainder were deleted (refer to table 1 for summary of endorsement). The combined effects of these changes reduced the number of items in Section B to 45, with 14 items on the symptoms subscale, 19 items on the function scale, and 12 items on the socio-emotional scale. Table 1. Summary of patient selections of pain descriptors Pain descriptor # of Status endorsements Sharp 12 Keep Hot 8 Dull 2 Cold 3 Sensitive 11 Keep Unpleasant 8 Itchy 0 Numb 4 Throbbing 13 Keep Shooting 8 Stabbing 15 Keep Aching 11 Keep Pricking 4 Burning 11 Keep Spontaneous 1 Deep intense 16 Keep Radiating 6 At this point, a formal pilot-testing study was submitted to the Research Ethics Board of Hamilton Health Sciences and McMaster University, consisting of 2 phases. Phase one requested review of the entire HI-CRPS (Draft 14) from a selection of international experts who had published on CRPS in the past 5 years, as well as to a group of clinicians treating CRPS in Southern Ontario (n=25), inviting them to review the tool using a structured questionnaire, including 11

16 rating content coverage, and making suggestions for adding or deleting items. Only 3 responses were received: 2 from local clinicians and the third from an internationally recognized researcher. One of the respondents did not complete the numerical rankings of components, but did comment on content and taxonomy. Given the poor response rate, these results are not presented here as they cannot be considered generalizeable. Phase two involved pilot-tested of the entire tool for inter-rater and test-retest reliability with student evaluators; again, the recruitment rate was very low (n=5) due to the short time frame of the study. Phase three was a cognitive debriefing study of the clinician-based portion of the tool, and marks the transition of this endeavour from a clinical research project to a scholarly program of study. The cognitive debriefing study is described in detail in chapter two of this thesis. The overall purpose of this thesis is to critically examine outcome measurement in complex regional pain syndrome and to consider the multidisciplinary assessment values and taxonomy related to the evaluation of the signs and symptoms of CRPS. Firstly, we undertook a systematic review of the quality and extent of psychometric evaluations published for current outcome measures specific to CRPS: this is the basis of chapter one of this thesis. Secondly, we conducted a cognitive debriefing study of the concepts covered by the clinician-based portion of the HI-CRPS tool, gathering definitions, scale 12

17 anchors, and assessment practices and preferences from a sampling of the intended user groups; the methods and results comprise chapter two. 13

18 Preface This dissertation follows a sandwich thesis format. I,, am the first author for the two journal papers presented within. The ideas and work, including the design, data collection, data analysis, and manuscript preparation, associated with each paper are primarily my own. Drs. Joy MacDermid, James Henry and James Bain were coauthors. In their role as my supervisors, they provided guidance, critical feedback, and suggestions in regards to study design, data analysis, and writing up the findings. Dr. MacDermid also assisted in completion of the systematic review, acting as the second reviewer of the studies therein. 14

19 CHAPTER ONE A systematic review of outcome assessments for CRPS: describing the elephant Abstract Purpose: To conduct a systematic review of the quality and extent of psychometric examinations of disease-specific outcome measures for complex regional pain syndrome (CRPS). Method: Health database searches yielded 23 papers covering 19 assessment instruments. Each article was scored for quality using a 12-item structured tool; data was also extracted for comparison of tool content. Results: Article quality ratings ranged from 25% to 88%. Six of the tools were specific to the upper extremity; 5 for the lower extremities while the remaining 8 were general. Many general tools focused on a single construct, such as pain, skin temperature or allodynia. Most psychometric data was based on small studies (mean n=33); only one study addressed all relevant issues of reliability, validity and responsiveness. Conclusions: Despite the variety of outcome measurement tools reported for CRPS rehabilitation, large gaps in both comprehensiveness and supporting psychometric evidence remain. The existing state of evaluation for CRPS might be illustrated by the analogy of a blindfolded person examining parts of an 15

20 elephant and forming an inaccurate conclusion about the essence of an elephant. Comprehensive, relevant and psychometrically sound tools for monitoring treatment outcomes are needed to address the pain and functional limitations experienced by this population. Introduction Complex regional pain syndrome (CRPS) is a neurological disorder characterized by a variable collection of signs and symptoms affecting either the upper or lower extremities. These signs may include pain, sensory disturbances, trophic changes, altered vascular/thermal regulation, edema, joint stiffness, and other motor impairments [1, 2]. The condition typically develops as a seemingly disproportionate response to some form of trauma [3]. Often, the patient reports increasing burning pain, swelling and stiffness that appears days to weeks after the injury, characteristically extending beyond the site of injury. Initially, these features may be difficult to distinguish from post-traumatic findings, but ultimately, they exceed the expectations for known damage [1]. Symptoms may vary with activity, environment and stress [4, 5], and often leads to a decrease in the spontaneous movement and function of the affected limb, as well as a decrease in participation in daily activities [6, 7]. There is no single diagnostic test that can accurately diagnose CRPS; hence, the condition is identified primarily on the basis of clinical assessment [8,9]. The ongoing emergence of new etiological theories, along with an everevolving understanding of the syndrome has contributed to a proliferation of tools 16

21 to assist in diagnosis and monitoring of the signs and symptoms [10]. Tools that can accurately diagnose require discriminative properties; and the multidimensionality of CRPS and lack of a uniform specific biological finding limit the ability to develop accurate diagnostic tests. Despite difficulties in accurately making the diagnosis, there remains a need to assess patient status over time. This requires a different type of tool that is able to assess changes in the sign/symptoms of CRPS. Since the condition is a composite of multiple signs and symptoms, then the analogy might be one of describing an elephant by looking at its characteristic components. A number of assessment tools have been proposed for measuring the outcomes of treatments for CRPS. However, there have been no syntheses of the available information; nor consensus on which tools should be used in clinical research studies or practice. Conversely, recent systematic reviews addressing both medical and rehabilitative treatment recommendations for CRPS [11,12,13] have highlighted the inconsistent use of outcome measures to evaluate this population. The purpose of this study was to conduct a systematic review of the quality and extent of psychometric evidence for disease-specific outcome measures for complex regional pain syndrome, and to consider if there is sufficient weight of evidence to recommend utilization of any of these tools for clinical practice and /or research involving persons with CRPS. 17

22 METHODS: In September 2010, a systematic search was performed using Pub Med, Embase, Ovid Healthstar and Medline; see table 1 for search terms. The search was limited to articles available in English. The search revealed Table 1: Search terms for systematic review Complex regional pain syndrome Reliability Reflex sympathetic dystrophy Validity Causalgia Psychometrics Algodystrophy Questionnaire Neuropathic pain Responsiveness Shoulder-hand syndrome Rasch analysis Outcome measurement Self assessment Terms within each column were combined with or and then the results of each of these searches were combined with and two distinct categories of evaluations: those intended for diagnosis, and those developed to measure outcomes. Since the focus of this review was tools to measure change in status over time, diagnostic tools were excluded. All abstracts were reviewed and diagnostic articles were removed from the set, leaving 36 articles for full review. We were unable to obtain two of the papers in print or electronic form. A further 11 articles were eliminated after examination of the full paper because a) they did not meet the inclusion criteria or b) less than 20% of the study population (or less than n=50) had a diagnosis of CRPSI or II. This yielded 23 papers to be critically appraised and scored for quality. Refer to figure 1 for a flow diagram of the systematic review process. These 23 studies addressed 19 different instruments; 9 were self-report and 8 were performancebased, and two combine components of both. Twelve of the instruments were 18

23 comprised of multiple subscales; 7 were unidimensional. Figure 1: Flow diagram for systematic review Database search keywords: (complex regional pain syndrome or reflex sympathetic dystrophy or algodystrophy or causalgia or neuropathic pain or shoulderhand syndrome) and (reliability or validity or questionnaire or self-assessment or outcome measurement or psychometrics or responsiveness or Rasch analysis) not (cancer or low back pain or neuralgia) Search results: PubMed Embase CINAHL Medline Title / abstract review (n=115 after removal of duplicates) Selected for review (n=36): Outcome measure Diagnosis of CRPS Excluded: Review articles Diagnostic test Unable to obtain full text of article (n=2) Full review N=23 Excluded on review of full article (n=11): Diagnostic tests Less than 20% of study population is CRPS 19

24 Critical appraisal and psychometrics summaries All studies were appraised that a) addressed some aspect of psychometrics of an outcome assessment and b) at least 20% of the test population was comprised of persons with CRPS. Data on internal consistency, reliability, validity, and responsiveness was extracted from each paper using a structured instrument. This standardized 12-item evaluation instrument and accompanying guide was developed for appraisal of psychometric studies and has been used in previous systematic reviews of shoulder function, neck disability, and obesity-specific quality of life measures [14,15,16]. The 12 evaluation questions address the research question, study design, measurements, analysis and recommendations: each of the items is rated using a 3-point scale (0-2) to yield a total quality score out of 24, and the tool has a guide that provides descriptors to assist in ranking the items [17]; see appendix A for a copy of the tool. The reviewers (TP and JM) initially scored two articles jointly for a practice consensus round to ensure common interpretation of scoring guidelines, then appraised each of the 23 studies independently, and met to compare and discuss ratings until consensus was reached on the 12 quality score items for each article (see Table 2). All independent ratings were recorded and entered into SPSS 16.0 for analysis of agreement. Individual items had an average agreement rating of kappa = 0.87; the agreement for the total scores was ICC=0.98 (95%CI ). 20

25 Table 2. Consensus quality ratings of paper (arranged highest to lowest) Study Reference Geertzen et al [19] Kemler and devet [27] Burnham et al [35] Foufanzar et al [34] Perez et al [29] Roorda et al [31] VanEijs et al [33] Tool Grip strength Foot function Skin temp (IR thermom.) Sample size Quality Rating Items (see key below) / 20 17/ N A N A (%)Total VAS pain Rising and Sitting Q; Walking Stairs Q Walking Stairs Q Brush Allodynia N A Roorda et al [28] Walking activity Q N A Geertzen et al [18] Heitz et al [30] Roorda et al [32] Krause and Backonja [38] Galer and Jensen [39] Collins et al [40] Oerlemans et al [21] Oerlemans et al [24] Dworkin et al [37] ROM N A N A Rising and Sitting Q; Walking Stairs Q; Walking activity Q Rising and Sitting Q Neuropath. Pain Q NPS 288/ 78 42/ 26 TReND symptom inventory Impairment sum score (U/E) Radboud skills Q (RSQ) N A 528/ N 149 A SF McGill

26 Schasfoort et al [22] Brunner et al [25] Schasfoort et al [23] Bianchi et al [20] Perez et al [26] Davidoff et al [36] Upper Limb Activity Monitor (ULAM) RSQ RSQ Upper Limb Activity Monitor CRPS evaluation Impairment sum score (L/E) RSD assessment process N A (25) 43/ 58 Key to Quality Rating Items 1 Research Question (clarity and rationale) Measurement procedures 2 Description of setting and participants 3 Hypotheses about reliability and validity 4 Scope of psychometrics under 10 Selection of appropriate examination statistical tests 5 Sample size 11 Use of benchmarks and confidence intervals 6 Recruitment and retention 12 Conclusions and recommendations 8 Standardization and reduction of bias 9 Analysis related to hypotheses RESULTS Summary of the outcome measures Table 3 contains a brief descriptive summary of the 19 different assessment tools covered by the 23 papers, including the target populations, number and types of items included in the evaluation, administration time, and equipment required. None of the papers indicated that a manual was available for any of the assessment tools. Six upper extremity tools were revealed by this review: ROM of the upper limb [18]; grip strength dynamometry [19]; a composite CRPS 22

27 evaluation [20] combining pain visual analogue scale (VAS), swelling, AROM measures, and grip strength; the Impairment Sum Score (ISS) [21], including edema measured by volumeter, skin temperature measured by infrared thermometry, an AROM score, pain VAS, and pain descriptors from the shortform McGill; the Upper Limb Activity Monitor [22, 23], a portable sensor array that measures frequency and intensity of upper limb motion over a 24hr period in the clients home; and the Radboud Skills Questionnaire (RSQ)[24, 25], a 45 item self-reported evaluation of personal care, domestic activities, and other activities (including work, social and leisure activities). Five lower extremity instruments also emerged: a lower extremity version of the ISS [26]; a four-part foot function evaluation [27] of 1) forward/backward shifting and 2) lateral shifting of a foot panel, 3) alternately touching two bells, and 4) depressing a pedal; and a trio of self-report questionnaires. The Walking Activity questionnaire [28, 29] covers walking indoors and outdoors, walking speed and the use of aids with 35 questions. The Walking Stairs questionnaire [29, 30, 31] includes 15 items, such as time, effort, avoidance, use of aids, and the need for assistance. The Rising and Sitting questionnaire [29, 30, 32] is a unidimensional scale consisting of 39 items, such as sitting/rising from high and low seats, getting on/off toilet, and getting in/out of the car. 23

28 Table 3. Description of Tools Tool Population Items Admin time Impairment Not Sum Score reported [21,26] CRPS 1 (U/E and L/E); also compared with normals for L/E group Pain (VAS and partial McGill) Skin temperature Volume ROM Cost/equipment Volumeter for edema measurement, goniometer for ROM, IR thermometer for temperature measures Grip strength [19] Foot function [27] SF McGill [37] Rising and Sitting [29, 30, 32] Walking Stairs [29, 30, 31] Walking Activity [28, 30] RSD assessment process [36] CRPS1 Normals and CRPS Individuals with chronic pain, including neuropathic pain CRPS1 CRPS1, other disorders affecting l/e function i.e. CVA, OA CRPS1 RSD (CRPS1) Dynamometry: Full fist grip 3 point pinch grip Lateral pinch grip Forward/backward shifting and lateral shifting of a foot panel Alternately touching 2 bells Depressing a pedal 22 items: pain descriptions rated on a 10 pt scale 39 items, such as: Sitting/rising from high and low seats Getting in/out of car Getting in/out of bed Getting on/off toilet 15 items, such as: Time Effort Use of aids Need for assistance Avoidance 35 items, such as: Walking indoors Walking outdoors Walking speed Use of aids Pain from joint palpation Edema Skin temperature AROM McGill VAS for pain Not reported Not reported Not reported About 10 minutes About 10 minutes Not reported Calibrated dynamometers Wooden footboard with pedal and foot panel (generic instructions given for construction) Pen and paper; licensing fee when used in research by forprofit organizations Pencil and paper Pencil and paper Pencil and paper 45 min Volumeter for edema measurement, goniometer for ROM, IR thermometer for temperature measures 24

29 NPS [39] Radboud skills questionnaire [24, 25] TreND symptom inventory [40] Neuropathic pain, including CRPSI and CRPSII CRPS 1 (u/e) CRPS1 and fibromyalgia 10 questions: pain sharpness, heat/cold, dullness, intensity, surface / deep pain, and overall unpleasantness 45 items: Personal care Domestic activities Other activities (social, work, recreation) 164 items in 10 subscales, i.e.: sensory, trophic, autonomic, motor, visceral symptoms ROM [18] CRPS1 u/e AROM of shoulder, elbow and wrist VAS pain CRPS1 Average pain [34] intensity Neuropathic Neuropathic Pain pain, including Questionnaire CRPS1 [38] ULAM [22, 23] Skin temperature (IR thermometry) [35] CRPS evaluation [20] Brush Allodynia [33] CRPS1 Normals and CRPS (not specified if type I or II) CRPS (Not specified if type I or II) CRPS1 Not reported Not reported 12 items: Not reported Activity limitations based on recorded patterns of limb use, including speed and frequency of movements Skin temperature side differences: can be used on upper or lower extremities Pain, swelling, ROM, grip strength Brush-evoked allodynia 15 min Pen and paper min Pen and paper min Pen and paper Recordings based on a 24 hour monitoring period Not reported Not reported Not reported Inclinometer and goniometer 100mm VAS scales and rulers Pencil and paper Acceleration sensors, portable recorder, and software for analysis IR thermometers range from under $100 to $1500 Dynamometer and goniometer Soft brush The remaining 8 tools addressed more global concepts including brush evoked allodynia [33]; reported average pain intensity with a VAS [34]; and skin temperature [35] with 3 different measurement instruments (thermistor, tympanic 25

30 and skin thermometers). The RSD assessment process [36] was a composite score including a numeric rating scale (NRS) for pain from joint palpation, edema measures, skin temperature, AROM, McGill, and VAS for pain, again reflecting a mix of observed and self-reported components. The most recent version of the short-form McGill pain questionnaire (SF-MPQ-2) [37] has been expanded to include 22 items rated on 10-point metric in 4 subscales (continuous pain, intermittent pain, predominantly neuropathic pain, and affective descriptors). The Neuropathic Pain Questionnaire (NPQ) [38] asks respondents to rank 12 items from a single scale with a NRS 0-100, including burning pain, increased pain due to weather changes, and questions such as How overwhelming is your usual pain, while the Neuropathic Pain Scale (NPS) [39] has 10 questions rated 0-10 such as pain sharpness, heat/cold, dullness, pain intensity, surface / deep pain, and overall unpleasantness. Finally, the Trauma Related Neuronal Dysfunction (TReND) questionnaire [40] is a self-report including 164 items in 10 subscales incorporating sensory, trophic, autonomic, motor, and visceral domains. Quality of research The consensus quality ratings of individual items for each paper can be found in table 2. Of the 23 studies reviewed covering 19 different outcome assessments, only 8 papers achieved a quality score of greater than 70% (or 18/24). These papers included examinations of 8 different tools, with the Dutch Walking Stairs questionnaire being the only tool represented by more than one higher quality 26

31 study. The highest quality score was 21/24 (88%); this was assigned to Geertzen et al [19] for their study of grip strength measurements in persons with CPRS; the next highest (82%) was Kemler and devet [27] for their novel assessment of foot function. The lowest score was assigned to the earliest paper of the series by Davidoff et al [36] (RSD assessment process); however, this work was expanded by Oerlemans et al [21] to form the foundation for the Impairment Sum Score assessment tool. Overall, the mean quality score was 60%(+ 16). Study populations and sample size One major limitation of all of the papers reviewed was the failure to address sample size and statistical power. When the study population was comprised exclusively of persons with CRPS, sample sizes ranged from 4 to 58, with an average of 33 participants (Table 2). Developers of new tools may be anxious to put them out for publication before large sample sizes have been acquired; however, most authors failed to acknowledge this is a limitation. Precise estimates of population scores and standard error depend on adequate sampling [41]. In the interest of homogeneity, most studies also limited their focus to persons with CRPSI and excluded those with CRPSII. Assessment tools were also frequently designed to focus on one area of the body, with 6 tools either designed or tested for the upper extremity; 5 for the lower extremities, and 8 generic tools (see table 4). 27

32 Depth of exploration of tools None of the outcome tools have been studied for the full spectrum of psychometric properties; refer to table 4 for a summary of the examinations. The paper by Burnham et al [35] looking at skin temperature measurements reflects the greatest breadth of study: the paper covered test-retest reliability, construct validity, criterion validity, and responsiveness in a small sample of CRPS patients (n=17); internal consistency was not addressed as it did not apply (skin temperature was the singular measurement/construct). While many authors identified the need for reliable and valid outcome measures for use with the CRPS population, their concluding remarks often failed to recognize the need for further research to address gaps in the available psychometric data before considering the evidence produced by any given tool as sufficiently robust for use in research and clinical practice. Summary of findings by psychometric properties Internal consistency Internal consistency reflects the degree to which the items on the tools, or a subscale, demonstrate commonality or inter-correlation. It can be measured by the correlation of individual items scores to the scores of the scale (or relevant subscale) as a whole [item total correlations, or ITC] and is often expressed using Cronbach s alpha [41]. It was frequently labeled as reliability in the papers we reviewed. However, since internal consistency reflects a consistent 28

33 Table 4. Outcome Measure Summaries Tool Constructs examined UPPER EXTREMITY TOOLS Impairment Composite sum score score of ROM, [21] edema, temp diff, and pain Grip strength [19] ULAM [22,23] Radboud skills questionnaire [24, 25] CRPS evaluation [20] ROM [18] Quality score (%) Languages 54.2 Dutch version of McGill Self report One component Properties Studied # * Grip strength 87.5 N/a N _ Frequency and 50 N velocity of u/e N/a * movements Activity and participation limitations CRPS symptom inventory LOWER EXTREMITY TOOLS Impairment Composite sum score score of ROM, [26] edema, temp Walking Stairs [29, 30, 31] Dutch Dutch German Y # 11/24 Italian N % Joint ROM 13/22 N/a N _ diff, and pain Stair climbing in community dwelling adults 33.3 Dutch McGill Dutch * * One component * Y # $ $ % % Rising and Sitting [29,30,32] Walking activity [28, 30] Foot function [27] 63.6 Sit to stand 75 Walking in community dwellers German Dutch German Dutch German Y # $ * $ * Y # Foot function 87.5 N/A N * * * 29

34 Tool Constructs examined Quality score Languages Self report Properties studied GENERALIZED ASSESSMENT TOOLS TReND Symptoms of 58.3 Dutch Y $ symptom CRPS, FM or * inventory [29] RSI VAS pain [34] VAS pain 75 N/A Y _ $ * RSD assessment process [36] Composite assessment incl. McGill, skin temperature differences, edema NPS [39] Neuropathic pain SF McGill [37] Neuropathic Pain Questionnaire [38] Skin temperature (IR thermometry) [35] Brush Allodynia [33] Pain descriptions and severity Neuropathic pain Skin temperature side differences Brush-evoked allodynia 25 N/A N 58.3 English Y % 50 English Y # * 59.1 English Y # 77.3 N/a N _ $ * 75 N/a N _ # % KEY: # internal consistency content validity factorial validity $ test-retest reliability discriminant inter-rater reliability % responsiveness * construct validity criterion validity 30

35 response to different items of the scale at a single point in time, it provides little direct evidence on the important aspects of reliability for clinical measurement [42]. Analysis of internal consistency can be performed on both cross-sectional and longitudinal data; however, in our review, this information was only reported in 8 of 19 studies where it would have been possible and appropriate to collect and analyze this data (see table 4). Using longitudinal data, Oerlemans et al [24] reported high Spearman s coefficients of >0.80 for tester A and mostly above 0.60 (p. 238) for tester B using the RSQ with persons experiencing CRPSI in a single upper extremity. Krause and Backonja [38] found high internal consistency (Cronbach s alpha= 0.95) amongst the initial 32 items of the NPQ. Since overly high internal consistency indicates redundancy of items it was appropriate that they proceeded to develop a shorter version of the scale; however internal consistency was not reported for the final 12-item scale. Reliability Reliability is a reflection of the consistency or true variability reflected in the scores of an assessment tool; higher levels of reliability can give the user confidence that any differences in measurements between individuals are due to true variation rather than random or systematic error [41]. Most of the studies that assessed reliability (46% of the total papers) used reliability coefficients expressed as the intra-class correlation coefficient (ICC). These values indicate relative reliability as they compare the variability between subjects to the total variability as a ratio. Inter-rater reliability does not apply to self-report measures; 31

36 and was reported in 4/9 of the clinician-based measures. Kemler and DeVet [27] reported inter-rater reliability ranging from ICC= while simultaneously scoring patients using their foot function test board, however, these ratings were based only on the normative sample of healthy adults. Oerlemans et al [24] used a different method of comparison to evaluate the oral administration of the RSQ. Using 2 different raters, they compared the coefficient of variation (CV) between them, reporting a median CV of 2.3% (range 0%± 5.1%) for tester A, while the median for tester B was 4.5% (range 0%± 18.9%). Test-retest reliability, which assesses stability of measures over periods of time where the patient should is not changing, can apply to both self-report and clinician-based measures and was assessed in 9/19 of the measures (refer to table 4). The Dutch Walking Stairs Questionnaire (WSQ) and Rising and Sitting Down Questionnaire (RSDQ) were examined by Perez et al [26] in persons with CRPSI of a single lower extremity: they reported test retest reliability as measured by ICCs >0.78 (WSQ) and >0.84 (RSDQ). Excellent test-retest reliability was reported for the TReND questionnaire with ICCs > 0.90 in both the CRPS sample and overall group of persons with CRPS and fibromyalgia [40]. Burnham et al [23] compared three different devices for measuring skin temperature, and found high ICCs (>0.96) for all three methods: thermistor, tympanic thermometer, and IR skin thermometer. Reliability coefficients can also be determined by using generalizability theory to simultaneously consider different sources of error variance; however, 32

37 this approach is uncommon, being only reported by a single paper [18] in this review. That study examined the reliability of ROM measures considering multiple sources of measurement error and found that all variables (ROM measures for individual joints) were influenced by the interaction of the patient, session and observer: this resulted in ICCs ranging from 0.55 to 0.97 for different joints. The smallest detectable difference, an extension of the standard error of measurement, ranged from 7 degrees for elbow flexion to over 25 degrees for external rotation of the shoulder. Validity When considering the measurement properties of a tool, it is possible to examine multiple forms of validity all intended to test different facets of the truthfulness of the data gathered by the assessment tool. See table 4 for a compiled account of the types of validity investigated by the papers included in this review. While multiple papers attributed content validity to their tool (see table 4), only two [21, 24] described in detail a process for formal testing. Content validity methods included a Delphi consensus round with experts to verify that their outcome measures reflected the spectrum of symptoms experienced by persons with CRPS, and were not missing any key signs or symptoms. Other authors invoked content validation based on literature reviews or when test items appeared to cover areas identified as key domains by others [29,30]. While Dworkin et al [37] did not overtly refer to content validity, they did in fact conduct 33

38 focus groups to assist in the generation of meaningful descriptors of pain qualities, and then had over 800 persons living with pain review the items developed for the newest iteration of the SF-MPQ-2 as part of an on-line survey. Factor analysis was employed by Dworkin et al [37] to scrutinize their revised scale. They used data from their web survey to cluster the new items into subscales, and then performed a confirmatory analysis of the old and new sub-scales (goodness of fit index >0.90 for all 4 sub-scales) once the overall tool revisions had been finalized this could be considered factorial validity. Krause and Backonja [38] also used factor analysis to assist in reducing items for the NPQ from the initial group of 32 down to 12 items, ensuring they retained items that loaded strongly (>0.60) onto each of the factors that related to their core theoretical concepts. Five papers examined discriminant validity, which includes both the ability of the tool to identify known groups and the aspect of contrasting against other tools measuring different constructs. Galer and Jensen [39] demonstrated the ability of four items from the NPS [sharp, cold, sensitive and itchy) to discriminate between persons with post-herpetic neuralgia and those with other types of neuropathic pain including those with CRPS- using repeated analysis of variance (ANOVA) and p<0.01. Kemler and devet [27] validated their assessment of foot function by comparing the bilateral scores of healthy participants with bilateral scores from a group of persons with CRPS of one lower extremity; they ratified their hypothesis that persons with CRPS of a single lower 34

39 extremity would perform poorer than healthy controls (matched for age, sex and dominance) on both their affected and unaffected side (p<0.05). Van Eijs et al [33] validated the ability of brush-evoked allodynia testing to predict which persons with CRPSI would respond positively to spinal cord stimulation; they found a mean allodynia score of below 2.5/10 predicted those who would benefit from the procedure (sensitivity of 0.75 and specificity of 0.81). Construct validity, or how closely the tool behaves to how the developers predict it will (based on their theoretical foundation) was widely considered by fourteen of twenty-one papers; often this took the form of comparing the results of the measure against the results of other tools. Burnham et al [35] found strong correlations between several methods for measuring skin temperature (thermistor, tympanic or skin) and intramuscular temperature readings at the same body sites (r >0.90). Kemler and devet [27] verified their postulation that the assessment of foot function would have low correlations with existing tools such as the 3 minute walk (r=0.30, p>0.05) and timed up and go (r= -0.01, p>0.05) because the individual tasks (pressing a pedal, ringing a bell) measured different components of foot impairment as compared to the mobility-based assessments. Dworkin et al [37] extensively evaluated the construct validity of the SF-MPQ-2 by making multiple comparisons to other recognized measures of pain intensity, activity and participation limitations related to pain, finding (among others) positive correlations with the Brief Pain Inventory (BPI) average pain score (r=0.60, p<0.001) and Multidimensional Pain Inventory interference 35