Measuring Psychosocial Recovery After Brain Injury: Change Versus Competency

Transcription

1 538 ARTICLES Measuring Psychosocial Recovery After Brain Injury: Change Versus Competency Robyn L. Tate, MPsychol, PhD, Anne Pfaff, MA, Ahamed Veerabangsa, MBBS, FAFRM, Adeline E. Hodgkinson, MBBS, FAFRM ABSTRACT. Tate RL, Pfaff A, Veerabangsa A, Hodgkinson AE. Measuring psychosocial recovery after brain injury: change versus competency. Arch Phys Med Rehabil 2004;85: Objectives: To determine the psychometric properties of an alternative form of the Sydney Psychosocial Reintegration Scale (SPRS) that focuses on competency of functioning (Form B) as opposed to the original form that examines change from the premorbid level (Form A). Design: Descriptive correlational study. Ratings were made by 2 treating clinicians on patients at discharge and 1 week later by using Forms A and B of the SPRS. A subset of 25 close relatives of the patients also completed the 2 SPRS forms and the London Handicap Scale at patient discharge. Setting: Inpatient brain injury rehabilitation unit. Participants: Sixty-six people with brain injury. Interventions: Not applicable. Main Outcome Measure: The SPRS is a 12-item questionnaire, rated on a 7-point scale, that measures function in occupational, interpersonal, and living skills domains. Results: The internal consistency of Form B was high (.90), as was stability (intraclass correlation coefficient [ICC].90) and interrater agreement (ICC.84). Similarly, good psychometric properties were found for Form A for internal consistency (.90), stability (ICC.90) and interrater agreement (ICC.82), which replicated previous findings with this form. Comparability between forms was excellent (ICC.97), and correspondence between ratings of the clinician and close relatives on Forms A and B was fair to good (ICC.57, ICC.67, respectively). Conclusions: Form B of the SPRS has psychometric properties equally sound to those of Form A. It is anticipated that there will be many advantages in having alternative forms of the same instrument, thus enabling measurement of psychosocial integration from different perspectives. Key Words: Brain injuries; Outcome assessment (health care); Rehabilitation; Social adjustment by the American Congress of Rehabilitation Medicine and the American Academy of Physical Medicine and Rehabilitation THE PAST DECADE HAS SEEN the development of scales to measure psychosocial functioning that are specifically designed for people with traumatic brain injury (TBI). From the Rehabilitation Studies Unit, Department of Medicine, University of Sydney and Royal Rehabilitation Centre Sydney, Ryde (Tate); and Brain Injury Rehabilitation Unit, Liverpool Hospital, Sydney (Pfaff, Veerabangsa, Hodgkinson), Australia. No commercial party having a direct financial interest in the results of the research supporting this article has or will confer a benefit upon the author(s) or upon any organization with which the author(s) is/are associated. Reprint requests to Robyn Tate, PhD, Rehabilitation Studies Unit, Royal Rehabilitation Centre Sydney, PO Box 6, Ryde, NSW 1680, Australia, rtate@med. usyd.edu.au /04/ $30.00/0 doi: /s (03) Such scales were previously described as measures of handicap, 1 and within the revised World Health Organization (WHO) model are now referred to as participation. 2 WHO constructs of handicap and participation have been reviewed recently, 3,4 as have instruments that measure levels of functioning in this domain. 5-7 Advantages of these types of instruments over the traditional semistructured interview are that they are usually less time consuming, use a standardized format, and provide scores that can then be subjected to statistical analysis. These features are important for research purposes. Additionally, they are particularly pertinent in clinical situations in which it is often necessary to make repeated evaluations and provide objective evidence of treatment progress, yet the time available in which to gather the data is usually at a premium. Representative TBI-specific instruments that target participation include the Brain Injury Community Rehabilitation Outcome Scales 8 (BICRO-39); Community Integration Measure 9 (CIM); Community Integration Questionnaire 10,11 (CIQ); Community Outcome Scale 12,13 (COS); Disability Rating Scale 14 (DRS); Glasgow Outcome Scale 15 (GOS), its extended version 16 (GOS-E) and its revision 17,18 ; and the instrument developed by our group, the Sydney Psychosocial Reintegration Scale 19 (SPRS). Additionally, although the Craig Handicap and Reporting Technique 20 (CHART) was originally designed for the spinal cord injury (SCI) population, its subsequent inclusion of a cognitive independence scale 21 has made it appropriate for the TBI group. 7 Table 1 provides a description of the main features of these scales. Each of the scales has strengths and weaknesses in relation to the measurement of participation and participation restriction (ie, handicap). The CIM is unique among scales of participation in that it is client centered and measures the psychological construct of a sense of belonging in the community. It contains a minimum set of items to potentially ensure reliability; has an adequate score range; established validity; and, pending results of further investigations into its reliability, is probably the scale of choice for this type of information. After the DRS and GOS/GOS-E, the CIQ is the best-known and most widely used instrument with which to measure participation after TBI objectively. It is a more specific instrument than either the DRS or GOS-E in that it targets community integration rather than overall level of functioning. It also has the advantage of containing a range of standard items, as opposed to the DRS in which only 2 items level of independent functioning and employability are directly pertinent to community integration; or general descriptions against which a person is broadly classified, as in the GOS-E. Recent work with the GOS-E has produced an instrument that is well operationalized, containing a flowchart of questions and response possibilities, 18 but it remains a classificatory scale. Dijker s review of the CIQ 22 was critical of aspects of its psychometric properties, including the initial study that used a subject:variable ratio of just over 1:1 in a factor analysis to reduce the 47 items to 15. Other concerns were raised about small and overlapping

2 PSYCHOSOCIAL RECOVERY AFTER BRAIN INJURY, Tate 539 Table 1: Comparison of Features in Some Scales Measuring Psychosocial Integration After TBI Scale and Source Item Description Response Format Psychometric Properties BICRO-39 8 CIM 9 CIQ 10,11 COS 12,13 CHART 20,21 DRS 14 GOS-E interview format 17,18 SPRS items; 8 domains (personal care, mobility, self-organization, partner/child contact, parent/sibling contact, socializing, productive employment, psychological well-being) 10 items; 1 domain (sense of belonging in community) 15 items; 3 domains (home integration, social integration, productive activities) 4 items (mobility, occupation, social integration, engagement) Original scale 27 items; 5 domains (physical, mobility, occupational, social, economic); revised 32-item scale 21 adds a cognitive domain 8 items; 4 categories (arousability, awareness and responsivity; cognitive ability for self-care activities; dependence on others; employability) Semistructured interview accompanied by flowchart of classifications for responses items; 3 domains (occupational activity, interpersonal relationships, independent living skills) 6-point rating scale, some domains use frequency ratings, other domains use degree of assistance. Score range for total: Lower scores equal better outcome. 5-point rating scale ( always agree to always disagree ). Score range Higher scores equal better integration. Open-ended responses converted to 3-point rating scale, and 6-point rating scale for 3 items. Score range for total Higher scores equal better integration. 7-point rating scale ( no problem/fully compensated to no longer living in community ). Score range Higher scores equal better outcome. Open-ended responses given weighted scores. Score range for each domain. Higher scores equal greater independence (percentage independence). Variable score ranges for categories. Total score range Higher scores equal greater disability. Classificatory scale using 8 categories (death, vegetative state, severe disability lower level, severe disability upper level, moderate disability lower level, moderate disability upper level, good recovery lower level, good recovery upper level). 7-point rating scale ( no change to extreme change ). Score range for total Higher scores equal better reintegration. Internal consistency: range , with 4/8 domains 0.8 Stability: carer data (mean, 6.1d).75 for 7/8 domains Patient-proxy agreement: all statistically significant; 5/8 scales.70 Responsiveness: demonstrated on personal care, mobility, and psychological well-being scales for total sample. Validity: concurrent validity with similar domains on FIM FAM (eg, mobility,.76; personal/self-care,.60), and psychological well-being with HADS (.68;.81) Internal consistency: -.87 Stability: no information Interrater/patient-proxy agreement: no information Responsiveness: may be subject to ceiling effects; at least 2/10 items had 50% obtaining maximum score Validity: factor analysis resulted in a 1-factor solution; significant group differences between TBI and 2 non brain-injured groups; concurrent validity with standard handicap measures is low (CIQ,.34), but higher and statistically significant for more socially focused measures (ISEL,.48) Internal consistency: range.35.84, with 1/3 domains 0.8 Stability: (7 10d) for domains Patient-proxy agreement: for all domains, but ICCs reported by Tepper et al 34 are lower: Responsiveness: no information Validity: significant group differences between TBI and non brain-injured groups, and for TBI subgroups with different levels of residential independence; concurrent validity with DRS (Sander et al 33 ) (eg, employability/productive,.58; level of functioning/home competency,.46) Internal consistency: no information Stability: no information Interrater/patient-proxy agreement: no information Responsiveness: no information Validity: concurrent validity with FIM FAM (cognitive scale,.47 to.67); engagement with HADS (.45.55); variable associations with injury severity (.20 to.51). As reported by Whiteneck et al 20 for SCI population; see Mellick et al 21 and Hall et al 7 for some TBI data Internal consistency: Rasch analysis indicates a well calibrated linear scale Stability: (1wk).80 for all domains Patient-proxy agreement:.75 for 3/5 domains; cognitive domain in TBI sample,.82 Responsiveness: no information Validity: significant group differences on 4/5 domains for those rated by clinicians having high vs low handicap As reviewed by Hall et al 37 : Internal consistency: Rasch analysis indicates it measures a wide range of disability but is less sensitive at the upper level Stability: (1d).95 Interrater agreement: Responsiveness: 71% showed improvement between admission and discharge Validity: concurrent validity with GOS at discharge,.67; predicted return to work with 87% accuracy Internal consistency: no information Stability: no information Interrater agreement: 78%,.85 Responsiveness: no information Validity: concurrent validity with Barthel Index,.61; with DRS,.89 Internal consistency: range.70.89; 0.8 for 1/3 domains Stability: (1mo).90 Interrater agreement:.95 Responsiveness: significant group differences between admission and discharge scores Validity: concurrent validity with GOS,.77; LHS,.85; SIP,.76; significant group differences among GOS subgroups; convergent validity with similar domains on SIP (eg, psychosocial/relationships,.76), and discriminant validity with dissimilar domains (eg, physical/relationships,.23) NOTE. Since acceptance of this article, Pettigrew et al 38 have provided stability and additional interrater reliability data for the GOS-E, both of which are high. Abbreviations: FIM FAM, Functional Independence Measure and Functional Assessment Measure; HADS, Hospital Anxiety and Depression Scale; ISEL, Interpersonal Social Evaluation List; LHS, London Handicap Scale; SIP, Sickness Impact Profile.

3 540 PSYCHOSOCIAL RECOVERY AFTER BRAIN INJURY, Tate samples on which psychometric properties were established and ceiling effects, thereby impacting on the responsiveness of the scale to detect real changes over time. Recent work has used a larger sample size to provide a better foundation for the factor structure of the CIQ 23 and included suggestions for a revised scoring format. Scales measuring similar constructs to the CIQ include the BICRO-39, COS, and SPRS. The BICRO-39 has a broad sampling of relevant items and a wide score range. Unlike many recently developed scales, its psychometric properties have been thoroughly investigated, but the results have been somewhat variable (see table 1). Limited information is available for the COS, and, in particular, no reliability data were reported in the original publications. Because of the small number of items (n 4), its reliability could be low. The 12- item SPRS was derived from our earlier classificatory scale 24 to measure handicap. The psychometric properties of the SPRS are very good, with high internal consistency, evidence of responsiveness, and high intraclass correlation coefficients (ICCs) for 1-month stability and interrater agreement. As shown in table 1, it has concurrent validity with other pertinent scales, such as the GOS-E, London Handicap Scale 25 (LHS), and Sickness Impact Profile. 26 There is preliminary evidence for construct validity (both convergent and discriminant), and the scale can distinguish among different GOS subgroups. 19 These findings encouraged us to continue work with the scale. An advantage of the SPRS, as compared with the aforementioned scales, is that it explicitly measures change from the preinjury level. This rating format is the method of choice in many situations, both for clinical practice and research, and directly addresses handicap or participation restriction. Respondents, such as family members, can readily relate to the response format and scores are easily interpreted. The person with TBI is thus his/her own control, and this bypasses the need for normative data required to validly interpret scores on the CIM and CIQ. Moreover, scales that are reliant on normative data can only provide information about the patient relative to the general population and cannot indicate whether there have been changes in the patient s level of functioning from an earlier (ie, premorbid) time. Change from the premorbid level is implied in the response format of other scales (eg, DRS, COS, GOS-E, some BICRO-39 items), but the COS and DRS have a restricted number of items about psychosocial functioning (n 4, n 2, respectively) and the GOS-E has a restricted score range (1 8). Each of these factors has an impact on reliability and responsiveness, respectively. A special strength of the CHART is that its developers used weighted scores, with the maximum score of 100 representing roles fulfilled at a level equivalent to that of most able-bodied persons. 20(p520) The converse, however, is that ceiling effects may apply in the sense that a person cannot do better than no handicap or complete independence. In our 1999 article, 19 we drew attention to the distinction between change in functioning from the premorbid level and capacity or competency of the current level of functioning. There are several circumstances in which the focus on change per se may be neither pertinent nor appropriate. For instance, the clinician generally has no knowledge of a person s preinjury level of functioning; thus, he/she cannot make valid direct ratings of the degree of change the injured person experiences without input from an informant such as a relative or the patient. Moreover, when the injury occurred many years previously, it can be difficult for relatives or the injured person to recall accurately the level of function and to precisely rate the degree of change. Similarly, when the injury occurred in childhood, a measure of psychosocial integration as an adult is difficult to make when expected levels of competence in areas such as work, relationships, and independent living skills are different. 27 In other situations in which repeated measures are taken, such as for program evaluation, it may be more appropriate simply to document psychosocial function at a given point in time, without reference to some earlier state. Furthermore, in a scale measuring change, no reference is made to the quality of the premorbid level of function against which such change is compared. As an extreme example, a person who before injury did not have a record of steady employment and, after the injury, does not have any significant impairments that affect his/her capacity to work, yet still does not have steady employment, would be rated in the best category no change. Clearly then, the category no change is not synonymous with excellent competency or similar descriptor, nor should this necessarily be the case. Yet, in many situations, it is important that the quality of levels of functioning be addressed. Three of the aforementioned scales (CHART, CIM, CIQ) document level of function without reference to the premorbid state. Of these, both the CHART and CIQ measure integration in objective terms (eg, social contacts, work). The SPRS also takes this approach. In this study, we examined the clinical utility of an alternative version of the SPRS that examines the patient s level of psychosocial functioning without reference to the premorbid state. We considered it advantageous to have alternative forms of the same instrument so that the more appropriate form could be used in particular settings. The 2 formats of the SPRS provide complementary information; its original version, referred to as Form A, measures change from the premorbid level, and the new version, referred to as Form B, measures current level of competency in the same psychosocial domains. Form B of the SPRS uses the same 12 items as Form A, as described by Tate et al, 19 and the same 7-point rating scale, but the items are phrased without reference to the premorbid level of functioning. Responses range from very good to extremely poor, and the lowest 5 response categories (from a little difficulty to extremely poor ) are operationalized with behavioral descriptors. Form B is reproduced in appendix 1. As with Form A, the total score for Form B ranges from 0 to 72, and each domain (occupational activities, interpersonal relationships, independent living skills) ranges from 0 to 24. Higher scores indicate better degrees of function. The major aims of this study were to assess the psychometric properties of Form B of the SPRS, to examine the comparability between Forms A and B, and to compare the ratings of clinicians on the 2 forms with those of close relatives of people undergoing inpatient rehabilitation for brain injury. METHODS Approval to conduct the study was granted by the South Western Sydney Area Health Service. The target group comprised a consecutive series of all admissions over a 22-month period (November 1998 September 2000) to the Brain Injury Rehabilitation Unit at Liverpool Hospital between November 1998 and September 2000 who were undergoing inpatient rehabilitation. The 66 patients (47 men, 19 women) had an average age standard deviation of years (range, 14 67y) at the time of their injuries, which were most frequently caused by road traffic crashes (n 41, 62.1%), falls (n 14, 21.2%), assaults (n 7, 10.6%), and other neurologic causes (n 4, 6.1%; ie, hypoxia, n 2; embolism, n 2). Patients were severely brain injured; posttraumatic amnesia (PTA) data were available for 54 patients, who had a mean PTA duration of days (range, 2d to 6mo), with

4 PSYCHOSOCIAL RECOVERY AFTER BRAIN INJURY, Tate 541 Table 2: Forms B and A of the SPRS for All Administration Conditions Mean Scores and Standard Deviations Total Score Occupational Activities Domains Interpersonal Relationships Living Skills Form B (competency) Rater 1 (N 66) Rater 2 (N 66) Time 1 (n 46) Time 2 (n 46) Relative (n 25) Form A (change) Rater 1 (N 66) Rater 2 (N 66) Time 1 (n 46) Time 2 (n 46) Relative (n 25) Percentage Obtaining Extreme Scores Form B: 6: very good : extremely poor Form A: 6: no change : extreme change the duration for 98% (n 53) being 1 week or more, and for 69% (n 37) being more than 1 month. At the time of discharge, 2 clinicians (psychologist, rehabilitation physician) who had worked with the patients clinically and knew them well, independently rated them with both Forms A and B of the SPRS. The order of completing the forms was counterbalanced both between and within raters in a predetermined manner. To measure stability, the ratings of a subset of patients (n 46) were repeated 1 week later without reference to the earlier ratings. Close relatives of the first 50 of the inpatients were considered for inclusion in the study. Twenty-five relatives consented to participate and completed the 2 forms (in counterbalanced order) when the patient was discharged. The reasons the other 25 relatives did not participate were as follows: not fluent in English (n 5), patient did not have relatives visit or living in Sydney (n 11), relative was not approached because he/she was deemed to be too emotionally distressed (n 5), short admission (n 1), patient remained minimally conscious at discharge (n 1), and relative declined to participate (n 2). The relatives comprised 7 parents (28%), 13 spouses (52%), 3 (adult) children (12%), and 2 siblings (8%). In addition to the SPRS, the relatives completed the LHS that had previously shown high correlations with the Form A of the SPRS (.85). 19 RESULTS Data Analysis Initial screening of the data revealed that the total score was normally distributed for both forms of the SPRS and on each occasion that they were administered. Scores for the 3 domains comprising the scale (occupational activities, interpersonal relationships, living skills), however, showed significant skewness for several variables, although less than 20% obtained the extreme scores (0 or 6) for the mean of any of the domains for either form. Descriptive statistics of all variables for all conditions, along with percentages obtaining the extreme scores, are in table 2. Results of a series of Wilcoxon signed-ranks tests showed no significant differences between raters or between occasions on the total scores for either Forms A or B. Although significant differences were found between raters on Form A occupational activities (z 2.2, P.03) and on Form B occupational activities and living skills (z 2.42, z 2.50, respectively; both P.02), the results did not meet critical levels when Bonferroni adjustments were made to control for multiple comparisons. Moreover, the mean scores suggest that such differences were not clinically significant. There were no significant differences between ratings by relatives and clinicians on either form. One-way random effect intraclass correlations were calculated to examine stability, interrater agreement, and comparability between Forms A and B by using the SPSS, version 10.0, a for Windows. Analyses for stability used the data from rater 1 for half of the patients and rater 2 for the other half, alternating between raters, starting with rater 1. Time 2 stability data used the scores from the same rater 1 week later. Comparisons between clinician and relative used the data from rater 1 at time 1. Psychometric Properties of Form B (Competency) All items correlated significantly with the total SPRS score. The item-total correlation coefficients Spearman were in excess of 0.7 for 9 of the 12 items, with the exceptions being item 1 (work,.39), item 2 (work skills,.67), and item 6 (family,.51). Internal consistency of Form B was high (Cronbach.93). All ICCs (table 3) were statistically significant, both for the total scores as well as those for each of the domains. For the total score, the stability ICC was equal to.90, and the interrater agreement ICC was equal to.84. According to the criteria described by Cicchetti, 28 the clinical significance of ICCs greater than.75 is excellent. ICCs for the specific

5 542 PSYCHOSOCIAL RECOVERY AFTER BRAIN INJURY, Tate Table 3: ICCs for Forms B and A of the SPRS for All Administration Conditions Stability Time 1 vs Time 2 (n 46) Interrater Agreement Rater 1 vs Rater 2 (N 66) Clinician vs Relative (n 25) Form B (competency) Total score Occupational activities Interpersonal relationships Living skills Form A (change) Total score Occupational activities Interpersonal relationships Living skills FormAvsB: Clinician FormAvsB: Relative Total score Occupational activities Interpersonal relationships Living skills domains were generally lower but were still within the good to excellent ranges of clinical significance, both for stability (ICC range,.76 for interpersonal relationships to.93 for living skills) and interrater agreement (ICC range,.63 for occupational activities to.82 for living skills). Attention is drawn to the result for the interrater agreement for Forms A (ICC.82) and B (ICC.84). This result is a composite of 3 methods, introduced sequentially. Initially, individual patient ratings were made by each rater independently, without discussion of either patient data or scoring issues. After the data for the first 21 participants were collected, a preliminary analysis was conducted, and it was deemed that interrater agreement was unacceptably low for clinical purposes, although the coefficients would be described as fair and were statistically significant, both for Forms A and B total scores (ICC.54, ICC.65, respectively; P.004, P.001, respectively). An attempt was therefore made to improve interrater agreement by using the method of Levin et al. 29 After the case conference immediately preceding the patient s discharge, both raters discussed the information that had been given at the conference about the patient s level of functioning, although no specific reference was made to either the items or rating scale of the SPRS. The subsequent 25 patients had interrater agreement determined by this method and, as expected, interrater reliability improved substantially for both Forms A (ICC.88) and B (ICC.90). The acid test was to determine whether this training would generalize to a new set of patients for whom no discussion was conducted, thus reinstating the first method of determining interrater agreement. For the final 20 patients, both raters made their ratings in isolation without discussion from the case conference. The coefficients were somewhat lower than the previous method (Form A, ICC.82; Form B, ICC.81) but higher than the initial subset of patients and well in excess of the.75 classification of excellent. Comparison of Forms A and B Table 3 also shows that the correspondence between the total scores on Forms A and B was very high, using either clinician ratings (ICC.97) or those of the relatives (ICC.95). The ICCs for each domain were also very high: the clinician ratings ranged from ICC equal to.93 for occupational activities to ICC equal to.96 for interpersonal relationships. The descriptive statistics (table 2) show that the mean scores obtained for the total score and for each of the domains were similar. These data suggest that, although Forms A and B measure different constructs (change vs competency, respectively), they have good comparability. Comparison of Relative and Clinician Ratings The correspondence between relative and clinician was lower than that between the 2 clinicians, for both forms (table 3). The ICCs (Form A, ICC.57; Form B, ICC.67) were statistically significant (both P.001) but only of fair and good clinical significance, respectively. Even so, for both forms there was little difference between the mean scores of the clinician and the relative for the group as a whole (table 2). The LHS was completed by 21 relatives. In this small subset, it correlated highly with the clinician and relative ratings of the SPRS, both for Form A (.73,.88, respectively; both P.000) and Form B (.71,.92, respectively; both P.000). DISCUSSION The psychometric properties of Form B of the SPRS (competency of current functioning) are very good. The form is internally consistent, stable, and has excellent interrater reliability, particularly when raters are trained in the use of the scale s response format. It also shows good comparability with Form A (change from the premorbid level), with the very good psychometric properties of that version of the scale 19 being replicated in this independent sample. The present results, together with those previously published about Form A, 19 suggest that this scale can be a valuable addition to the evaluation of psychosocial reintegration after TBI. A major finding of this study was that Form A (change from preinjury level) and Form B (current competency) are roughly parallel and show good correspondence. This was the case for the ratings from both clinicians and relatives. The statistically and clinically significant coefficients obtained between the 2 forms suggest that the scores on 1 version of the SPRS have direct interpretation in terms of the other version. Thus, the person with a lot of change from the preinjury level will also have a poor level of competency in psychosocial functioning. As previously noted, currently available scales generally adopt 1 of 2 response formats, either focusing on change from the premorbid level (eg, COS, DRS, GOS-E) or current competency (eg, CHART, CIM, CIQ). There will be circumstances in which one or another type of information will be more pertinent for patient management, policy development, and research purposes, but such scales do not lend themselves to the variety of clinical and management questions that arise. The original reason for developing Form B was to account for premorbid individual differences that occur, particularly with respect to areas such as occupational activities and interpersonal relationships. It was therefore expected that there would be differences in the mean scores obtained for both forms. The mean scores for Forms A and B, particularly for the 3 domains that were very close, may suggest that Form B does not measure constructs that are much different from those examined by Form A. There may, however, be other explanations. It is likely that any differences in mean scores between Forms A and B were attenuated because a rehabilitation sample was used. The SPRS was originally designed for use in a community setting, and in an inpatient rehabilitation setting there is little opportunity for the person to engage in activities

6 PSYCHOSOCIAL RECOVERY AFTER BRAIN INJURY, Tate 543 rated by items, such as work skills, relationships with friends, use of transport, and so forth. It is therefore expected that greater discrepancies between the scores on Forms A and B will be apparent in a community sample. Another possible explanation for the close correspondence between scores on Forms A and B is that the mean total scores, as well as those of the individual domains, are masked by individual variability. Individual raw data records confirm that the 2 forms measure different facets of particular domains, as was intended. An example is that of case 33, a 37-year-old man who was injured in a fall and had PTA for 116 days. He was hospitalized for 9 months (6 of which were spent in the rehabilitation unit). At discharge, his total scores were 26 and 11 on Forms A and B, respectively (clinician ratings). Marked discrepancies between forms occurred in the interpersonal relationships and living skills domains. For example, overall, the quality of his interpersonal relationships (Form B) was rated as experiencing major difficulties (average domain score, 2.3), but in comparative terms there had been a lesser degree of change in his relationships compared with the premorbid level (Form A). The overall rating on that form was a little change (average domain score, 4.3). Pettigrew et al 17 have pointed to the weaknesses in scales such as the DRS and GOS in that they do not consider any premorbid psychological or physical problems. They argue for a greater recognition of preexisting impairments as factors that complicate the recovery process. The comparative use of Forms A and B, as shown in case 33, permits evaluation of both the quality of current level of functioning as well as the contribution of premorbid factors, in addition to changes that have occurred as a result of the injury. In our earlier report on Form A, 19 the method used to determine interrater agreement differed from the current method. Previously, we used 2 clinicians to interview relatives separately and sequentially. That procedure resulted in very high interrater agreement (ICC.95). The present method used clinicians ratings that were based on their knowledge of the patient s level of functioning, rather than direct interviews. Not surprisingly, interrater agreement was lower (ICC.82 for Form A), although it was still classified as excellent and was highly statistically significant (P.001). More important, however, were the widely divergent results for interrater agreement according to the method used with this study (Form A, ICC.54 before training, ICC.88 during training, ICC.82 after training). Although not explicitly stated, it is possible that Levin s group 29 used the method of discussion after the case conference to maximize interrater agreement (ICC range,.88.90) on their 7-point rating Neurobehavioural Rating Scale (NBRS). This method of conferring between raters is not ideal, and it is important that good interrater agreement can be achieved by using independent ratings. In this respect, it is of interest that a subsequent report on the NBRS by Vanier et al, 30(p797) found excessive interobserver differences in scoring and so opted to reduce the 7-point scale to a 4-point scale. By using this method, interrater reliability was fair to excellent for the new set of factors derived from their investigation, with the average ICC of.69 (range,.56.85). The cost of reducing the 7-point rating scale to a 4-point one, however, is likely to affect the scale s responsiveness. It is clear that interrater reliability of the SPRS is excellent when raters have been trained to use the scale and the response format, but it is only fair when such training has not been given (cf the NBRS). Although it is to be expected that training will result in improved interrater agreement, it should also be remembered that the present raters were experienced and expert clinicians who had worked in the same brain injury rehabilitation unit for many years. The assumption is frequently made that experienced clinicians will have similar frames of reference, and hence their assessments of a situation will be comparable. Our data showed that this was not the case. It was only after training in the use of the scale and the response format that differences between judges were reduced. Because we found that such training was generalizable and resulted in excellent interrater agreement, we recommend using the training procedure rather than reducing the response categories to maintain the scale s responsiveness. Correspondence between the ratings of close relatives and clinicians on Forms A and B, although statistically significant, showed only modest agreement. The ICCs were somewhat higher for Form B (ICC.67) than Form A (ICC.57). The mean scores show that relatives rated the injured person as having marginally better function than did the clinicians, but, for the group as a whole, the difference was of no consequence. It appears, however, that on a case-by-case basis, there is considerable disparity between clinicians and relatives perceptions of patients levels of function in the postacute stages of recovery. This was particularly apparent on both forms for the domain of interpersonal relationships, where the ICCs were poor (Form A, ICC.35; Form B, ICC.37). Although many studies have reported on agreement between self-ratings and those of relatives or other informants for scales reviewed in this report, 8,31-34 few studies have reported on clinician-informant ratings. A relevant report is that of Heilbronner et al, 35 who compared patient and staff ratings and found that agreement on ratings of success in therapy was low; however, their study was limited by a small sample size and the fact that different rating instruments were used by staff and patients. Clearly, however, this is an area needing further research on clinical grounds alone, given the importance of communication among staff, patients, and relatives regarding a patient s progress and outcome. CONCLUSIONS The SPRS measures up very well against the psychometric (or clinimetric, to use the term of Dijkers et al 6 ) criteria proposed by Andresen. 36 Andresen identified 11 domains, each rated at 1 of 3 levels (A to C). Data for the SPRS are published on most aspects of 8 of the domains (conceptual framework, measurement model, respondent burden, administrative burden, reliability, validity, responsiveness, alternate forms), virtually all at the A level. Further work on the remaining domains described in Andresen s article is required (notably, normative data, instrument bias, and cultural adaptations). Additionally, it is recognized that the current study of Form B was conducted with an inpatient sample and requires replication in a community sample. Nonetheless, the available data indicate that, in comparison with several instruments reviewed in this report, Form A of the SPRS rates at a comparable or better level (table 1). Our alternative version of the SPRS (Form B) adds to the strength of the scale. Having 2 versions of the scale with comparable item content and scoring format, but with a different focus (change from the premorbid level and current competency), gives the SPRS a distinct advantage over other scales that do not have this feature. APPENDIX 1: FORM B OF THE SPRS All items answered on a 7-point scale, as follows (behavioral descriptors are attached to responses in the lower 5 categories details available from author): 6 Very good 5 Slight difficulty 4 A little difficulty 3 Definite difficulty

7 544 PSYCHOSOCIAL RECOVERY AFTER BRAIN INJURY, Tate 2 A lot of difficulty 1 Very poor 0 Extremely poor Part A: Work and Leisure 1. Current work: How do you rate your relative s work (study) for number of hours and type of work? 2. Work skills: How do you rate your relative s work (study) skills? 3. Leisure: How do you rate the type and number of leisure activities your relative does? 4. Organising activities: How do you rate the way your relative organises work and leisure? Part B: Interpersonal Relationships 5. Spouse or partner: How do you rate your relative s relationship with his/her spouse or partner? 6. Family: How do you rate your relative s relationships with other family members? 7. Friends and other people: How do you rate your relative s relationships with other people outside family (such as close friends, work mates, neighbours)? 8. Communication: How do you rate your relative s communication skills with other people (ie. talk with other people and understand what they say)? Part C: Living Skills 9. Social skills: How do you rate your relative s social skills and behaviour in public? 10. Personal habits: How do you rate your relative s personal habits (eg, care in cleanliness, dressing, and tidiness)? 11. Community travel: How do you rate your relative s use of transport and community travel? 12. Accommodation: How do you rate your relative s living situation? References 1. World Health Organization. International classification of impairments, disabilities and handicaps (ICIDH). A manual of classification relating to the consequences of disease. Geneva: WHO; World Health Organization. ICF. International classification of functioning, disability and health. Geneva: WHO; Dahl TH. International classification of functioning, disability and health: an introduction and discussion of its potential impact on rehabilitation services and research. Scand J Rehabil Med 2002; 34: Gray DB, Hendershot GE. The ICIDH-2: developments for a new era of outcomes research. Arch Phys Med Rehabil 2000;81(12 Suppl 2):S Cardol M, de Haan RJ, van den Bos GA, de Jong BA, de Groot IJ. Handicap questionnaires: what do they assess? Disabil Rehabil 1999;21: Dijkers MP, Whiteneck G, El-Jaroudi R. Measures of social outcomes in disability research. Arch Phys Med Rehabil 2000; 81(12 Suppl 2):S Hall KM, Bushnik T, Lakisic-Kazazic B, Wright J, Cantagalio A. Assessing traumatic brain injury outcome measures for long-term follow-up of community-based individuals. Arch Phys Med Rehabil 2001;82: Powell JH, Beckers K, Greenwood RJ. Measuring progress and outcome in community rehabilitation after brain injury with a new instrument: the BICRO-39 scales. Arch Phys Med Rehabil 1998; 79: McColl MA, Davies D, Carlson P, Johnston J, Minnes P. The Community Integration Measure: development and preliminary validation. Arch Phys Med Rehabil 2001;82: Willer B, Rosenthal M, Kreutzer JS, Gordon WA, Rempel R. Assessment of community integration following rehabilitation for traumatic brain injury. J Head Trauma Rehabil 1993;8(2): Willer B, Ottenbacher KJ, Coad ML. The community integration questionnaire. A comparative examination. Am J Phys Med Rehabil 1994;73: Stillwell P, Stillwell J, Hawley C, Davies C. Measuring outcome in community-based rehabilitation services for people who have suffered traumatic brain injury: the Community Outcome Scale. Clin Rehabil 1998;12: Stillwell P, Stillwell J, Hawley C, Davies C. The National Brain Injury Study: assessing outcomes across settings. Neuropsychol Rehabil 1999;9: Rappaport M, Hall KM, Hopkins K, Belleza T, Cope DN. Disability rating scale for severe head trauma: coma to community. Arch Phys Med Rehabil 1982;63: Jennett B, Bond MR. Assessment of outcome after severe brain damage. A practical scale. Lancet 1975;i: Jennett B, Snoek J, Bond MR, Brooks N. Disability after severe head injury: observations on the use of the Glasgow Outcome Scale. J Neurol Neurosurg Psychiatry 1981;44: Pettigrew LE, Wilson JT, Teasdale GM. Assessing disability after head injury: improved procedures of the Glasgow Outcome Scale. J Neurosurg 1998;89: Wilson JT, Pettigrew LE, Teasdale GM. Structured interviews for the Glasgow Outcome Scale and the extended Glasgow Outcome Scale: guidelines for their use. J Neurotrauma 1998;15: Tate RL, Hodgkinson AE, Veerabangsa A, Maggiotto S. Measuring psychosocial recovery after traumatic brain injury: psychometric properties of a new scale. J Head Trauma Rehabil 1999; 14: Whiteneck GG, Charlifue SW, Gerhart KA, Overholser JD, Richardson GN. Quantifying handicap: a new measure of long term rehabilitation outcomes. Arch Phys Med Rehabil 1992;73: Mellick D, Walker N, Brooks CA, Whiteneck G. Incorporating the cognitive independence domain into CHART. J Rehabil Outcomes Meas 1999;3(3): Dijkers M. Measuring the long-term outcomes of traumatic brain injury: a review of the Community Integration Questionnaire. J Head Trauma Rehabil 1997;12(6): Sander AM, Fuchs KL, High WM, Hall KM, Kreutzer JS, Rosenthal M. The Community Integration Questionnaire revisited: an assessment of factor structure and validity [published erratum appears in: Arch Phys Med Rehabil 1999;80:1608]. Arch Phys Med Rehabil 1999;80: Tate RL, Lulham JM, Broe GA, Strettles B, Pfaff A. Psychosocial outcome for the survivors of severe blunt head injury: the results from a consecutive series of 100 patients. J Neurol Neurosurg Psychiatry 1989;52: Harwood RH, Rogers A, Dickinson E, Ebrahim S. Measuring handicap: the London Handicap Scale, a new outcome measure for chronic disease. Qual Health Care 1994;3: Bergner M, Bobbitt RA, Carter WB, Gilson BS. The Sickness Impact Profile: development and final revision of a health status measure. Med Care 1981;19: Newitt H. Paediatric head injury: implications for psychosocial adjustment into adulthood [PhD dissertation]. Melbourne: Univ Melbourne; Cicchetti DV. Guidelines, criteria, and rules of thumb for evaluating normal and standardized assessment instruments in psychology. Psychol Assess 1984;6: Levin HS, High WM, Goethe KE, et al. Neurobehavioural rating scale: assessment of the behavioural sequelae of head injury by the clinician. J Neurol Neurosurg Psychiatry 1987;50: Vanier M, Mazaux JM, Lambert J, Dassa C, Levin HS. Assessment of neuropsychologic impairments after head injury: interrater reliability and factorial and criterion validity of the Neurobehavioral Rating Scale Revised. Arch Phys Med Rehabil 2000;81: Cusick CP, Gerhart KA, Mellick DC. Participant-proxy reliability in traumatic brain injury outcome research. J Head Trauma Rehabil 2000;15:

8 PSYCHOSOCIAL RECOVERY AFTER BRAIN INJURY, Tate Cusick CP, Brooks CA, Whiteneck GG. The use of proxies in community integration research. Arch Phys Med Rehabil 2001; 82: Sander AM, Seel RT, Kreutzer JS, Hall KM, High WM, Rosenthal M. Agreement between persons with traumatic brain injury and their relatives regarding psychosocial outcome using the Community Integration Questionnaire. Arch Phys Med Rehabil 1997;78: Tepper S, Beatty P, DeJong G. Outcomes in traumatic brain injury: self-report versus report of significant others. Brain Inj 1996;10: Heilbronner RT, Roueche JR, Everson SA, Epler L. Comparing patient perspectives of disability and treatment effects with quality of participation in a post-acute brain injury rehabilitation programme. Brain Inj 1989;3: Andresen EM. Criteria for assessing the tools of disability outcomes research. Arch Phys Med Rehabil 2000;81(12 Suppl 2): S Hall KM, Hamilton BB, Gordon WA, Zasler ND. Characteristics and comparisons of functional assessment indices: Disability Rating Scale, Functional Independence Measure, and Functional Assessment Measure. J Head Trauma Rehabil 1993;8(2): Pettigrew LE, Wilson JT, Teasdale GM. Reliability ratings on the Glasgow Outcome Scales from in-person and telephone structured interviews. J Head Trauma Rehabil 2003;18: Supplier a. SPSS Inc, 233 S Wacker Dr, 11th Fl, Chicago, IL