This article appeared in a journal published by Elsevier. The attached copy is furnished to the author for internal non-commercial research and

Transcription

1 This article appeared in a journal published by Elsevier. The attached copy is furnished to the author for internal non-commercial research and education use, including for instruction at the authors institution and sharing with colleagues. Other uses, including reproduction and distribution, or selling or licensing copies, or posting to personal, institutional or third party websites are prohibited. In most cases authors are permitted to post their version of the article (e.g. in Word or Tex form) to their personal website or institutional repository. Authors requiring further information regarding Elsevier s archiving and manuscript policies are encouraged to visit:

2 Psychiatry Research 187 (2011) Contents lists available at ScienceDirect Psychiatry Research journal homepage: Cognition UPSA score relationships: A further analysis of Silverstein et al. (2010) data and some caveats Steven M. Silverstein a,b,, Sherrie D. All a,b, Judith Jaeger c a University of Medicine and Dentistry of New Jersey, University Behavioral HealthCare, United States b Robert Wood Johnson Medical School, United States c AstraZeneca, United States article info abstract Article history: Received 12 August 2010 Received in revised form 22 November 2010 Accepted 1 December 2010 Keywords: Cognition Functioning UPSA MATRICS IntegNeuro Silverstein et al. (2010) reported correlations between scores on the UCSD Performance-Based Skills Assessment (UPSA) and scores on two cognitive test batteries (MATRICS Consensus Cognitive Battery and IntegNeuro) that were lower than those reported in past studies. The large sample size of that 4-site study (155 patients with schizophrenia) allowed for further analyses of the data to explore the reasons for the discrepancy. We examined the data from Silverstein et al. (2010) to determine if the correlation values obtained were affected by UPSA scoring method, site differences, patient level of functioning, range restriction, missing data, and/or whether data from the first or second administration of each cognitive test battery were used. Results indicate that the overall lower cognition UPSA correlations were a function of a single site with unusually low correlations. However, the low correlations at this site were not a function of any of the potential causes we examined. Correlations at the other sites were close to or within the range reported in past studies. Interestingly, the correlation between IntegNeuro and UPSA composite scores was higher at Time 2, suggesting that cognition UPSA correlations are affected by familiarity with the computerized test format Elsevier Ireland Ltd. All rights reserved. 1. Introduction During the 1990s, it became increasingly clear that cognitive impairment was of functional significance in schizophrenia. Specifically, studies consistently demonstrated that cognitive deficits were related to a host of outcomes such as reduced benefit from psychosocial interventions, poorer social functioning, and impaired work performance (Green, 1996; Silverstein et al., 1998; Green et al., 2000). Such studies spurred research into pharmacological and psychological methods to improve cognitive functioning. More recently, it has been recognized that cognitive measures, by themselves, are not adequate outcome measures for studies of schizophrenia and that measures of psychosocial functioning need to be included as co-primary measures in studies of cognitive change (Juckel and Morosini, 2008) for the following reasons: a) it is important to actually assess how people function and not just the correlates of functioning; and b) it is recognized that cognition mediates, but does not completely determine, how a person functions in the real world (e.g., Bowie et al., 2010). As a result, a number of functional outcome measures have been developed and compared (Leifker et al., 2009), and their relationships to cognition have been determined in multiple Corresponding author. UMDNJ, 151 Centennial Avenue, Piscataway, NJ 08840, United States. Tel.: ; fax: address: silvers1@umdnj.edu (S.M. Silverstein). studies. Of these measures, scores on the UCSD Performance Based Skills Assessment (UPSA) (Patterson et al., 2001) have consistently been demonstrated to correlate significantly and highly with scores on cognitive tests. Table 1 summarizes past studies of this issue, and demonstrates that the reported correlations have consistently been within the range, including with a revised and a brief version of the UPSA. Because of the magnitude of these correlations, which are generally higher than those reported in earlier studies of cognition outcome relationships that used a variety of older outcome measures (e.g., see Green, 1996), the UPSA has become a widely used measure of functional outcome. It also has several advantages over older measures, such as that it is performance-based and not subject to patient self-report bias or poor quality of informant information. In contrast to past findings on cognition UPSA relationships, we (Silverstein et al., 2010) recently found lower correlations between the UPSA and two widely used measures of cognition: the MATRICS Consensus Cognitive Battery (MCCB) (Nuechterlein et al., 2008) and IntegNeuro (Paul et al., 2005). That multi-site study included 155 schizophrenia patients tested twice (1 month apart) on both cognitive batteries, as well as at baseline on the UPSA. The correlation between the MCCB total battery composite score and UPSA total score was r=0.42 (pb0.001, N=121). The correlation between the IntegNeuro total battery composite score and UPSA total score was r=0.33 (pb0.001, N=115). Similar (but even lower) correlations were found with two other functional outcome measures, the Social Functioning /$ see front matter 2010 Elsevier Ireland Ltd. All rights reserved. doi: /j.psychres

3 S.M. Silverstein et al. / Psychiatry Research 187 (2011) Table 1 Previous studies reporting correlations between UPSA and cognitive battery scores. Study Measures N Correlation Diagnosis Relevant sample characteristics Patterson et al. (2001) UPSA vs. MMSE 50 Hierarchal multiple regression significant Beta = R 2 change over demographic, illness, and symptom variables =0.127 Total R 2 =0.623 Schizophrenia Middle aged and elderly (mean age= 56 [8.5]) Psychiatrically, physically and pharmacologically stable Twamley et al. (2002) UPSA vs. Mattis DRS Schizophrenia Age years (mean=54.6 [9.6]) Psychiatrically, medically, and pharmacologically stable. Mood disorder with psychotic features Excluded: Psychosis NOS current hospitalization physical or mental illness severe enough to preclude completion of the assessments Keefe et al. (2006a,b) UPSA vs. BACS Schizophrenia Inpatients in rehabilitation center and psychiatry unit Mausbach et al. (2007) UPSA-B vs. Mattis DRS Schizophrenia Middle aged and older adults (N40 years) Residents in board and care facilities McClure et al. (2007) UPSA vs animal naming, 181 Canonical correlation Schizophrenia Older adults WCST-64, TMT (A&B), R=0.64 Community dwelling RAVLT, WAIS-III, digit Excluded MMSE below 18 symbol, digit span and Required to meet at least one of three LNS. criteria: (1) an inpatient admission for psychosis in the past 2 years (2) an emergency room visit for psychosis in the past 2 years (3) at least one PANSS positive symptoms items relating to delusions, hallucinations, or conceptual disorganization 4 (moderate) at baseline. Green (2009) UPSA vs. MCCB Schizophrenia Subjects in MATRICS-CT study Keefe et al. (2009b) UPSA-2 vs. MCCB Schizophrenia Maintained on a stable dose of a second generation antipsychotic therapy Keefe et al. (2009a) UPSA-B vs. MCCB Schizophrenia Aged Buchanan et al. criteria for stability CGI-S total score 4 PANSS positive scale items 4 (moderate) No hospitalization for psychiatric illness for at least 8 weeks prior to screening. Pietrzak et al. (2009) UPSA vs. MCCB and the CogState Schizophrenia Battery MCCB=98 MCCB=0.79 Schizophrenia Free from an acute severe exacerbation of psychosis, clinically significant depressive symptoms, and change in antipsychotic medication for at least three months CogState=0.76 CGI-S total score 4 CogState=0.93 PANSS total scoren55 No hospitalization or acute intervention for a psychiatric condition within four months No ECT within two years Depp et al. (2009) UPSA vs. Mattis DRS Bipolar disorder Middle-aged and older adults Harvey et al. (2009) UPSA-B vs. TMT (A&B), USA=244 USA=0.58 Schizophrenia Older adults WAIS-III LNS & Vocabulary, Sweden=146 Sweden=0.54 Outpatients RAVLT, WCST Bowie et al. (2010) UPSA-B/EFB (UPSA-B plus Everyday Functioning Battery)vs.RAVLT,TMT (A&B), WAIS: LNS & Coding, WCST-64, semantic fluency test, 4-digit CPT-IP Schizophrenia Full or mixed Ashkenazi Jewish background Age 21 and older Bipolar I disorder Outpatient or residential treatment settings UPSA = UCSD Performance-based Skills Assessment, UPSA-B = UPSA Brief, UPSA-2 = UPSA 2nd Edition, MCCB = MATRICS Consensus Cognitive Battery, MMSE = Mini Mental State Examination, Mattis DRS = Mattis Dementia Rating Scale, BACS = Brief Assessment of Cognition in Schizophrenia, WCST-64 = Wisconsin Card Sorting Test, 64 Items, TMT (A&B) = Trail Making Test Parts A and B, RAVLT = Rey Auditory Verbal Learning Test, WAIS-III = Wechsler Adult Intelligence Scale, 3rd Edition, LNS = WAIS-III Letter Number Sequencing, CPT-IP = Continuous Performance Test Identical Pairs.

4 426 S.M. Silverstein et al. / Psychiatry Research 187 (2011) Scale (SFS) (Birchwood et al., 1990) and the Multidimensional Scale of Independent Functioning (MSIF) (Jaeger et al., 2003; Miles et al., in press). The purpose of this report is to clarify the reasons for the discrepancy between the recent Silverstein et al. study and past studies. To do this, we conducted a re-analysis of the UPSA data, focusing on site differences in the strength of the correlations, score distributions (i.e., range restriction) on the MCCB, IntegNeuro and UPSA, and missing data rates. We also report on the magnitude of the correlations using both the Time 1 and Time 2 testing data, to determine if increased familiarity with the cognitive tests affected the size of the correlations. 2. Method 2.1. Subjects As reported in Silverstein et al. (2010), subjects were 155 people (102 or 65% male) with schizophrenia, diagnosed using the Structured Clinical Interview for DSM-IV Diagnosis (SCID) (First et al., 1995) and 75 healthy (i.e., without a diagnosable major Axis-I psychiatric condition, confirmed by the non-patient version of the SCID) controls (28 or 37% male). The control group was matched to the patient group based on group means for age and maternal education. All patients were outpatients or partial hospital patients. The schizophrenia group was 37% African-American, 0.6% Asian, 3.2 Hispanic, 2.5% Pacific Islander or Other, and 48.4% White (with 8.3% undetermined). The control group was 34.7% African-American, 2.7% Asian, 7% Hispanic, 1.3% Pacific Islander or Other, and 53.3% White. Inclusion criteria were identical to those in the MCCB validation study (Nuechterlein et al., 2008) (with 1 minor deviation involving medication side effect assessment; see point g below), to ensure that relatively asymptomatic patients on stable treatment regimens were studied. Specifically, inclusion criteria were as follows: a) between the ages of b) clinically stable, not hospitalized, and in the residual (non-acute) phase of their illness for at least 8 weeks; c) maintained on current antipsychotic and other concomitant psychotropic medications for a specified period of time sufficient to minimize potential complications of assessment of cognitive status (i.e., at least 6 weeks) and on current dose for at least 2 weeks. Any dose changes during study participation could not exceed ±25% of dose at the beginning of the period; d) no more than a moderate severity rating on the hallucinations and delusions items of the Positive and Negative Syndrome Scale (PANSS) (Kay et al., 1987) [note that the PANSS rather than the Brief Psychiatric Rating Scale (BPRS) (Ventura et al., 2000), which was used in the MCCB validation study (Nuechterlein et al., 2008), was used for this study]; e) no more than a moderate severity rating on positive formal thought disorder (e.g., PANSS Conceptual Disorganization item score); f) no more than a moderate severity rating on any negative symptom item (e.g., PANSS items N1 N7); g) a minimal level of extrapyramidal symptoms [e.g., Simpson Angus Scale (Simpson and Angus, 1970) total score (excluding items for leg drop and head drop) less than 5]. Note that the MCCB validation study cutoff was 6, but that used the full 10 item scale whereas we used 8 items; h) a minimal level of depressive symptoms, as reflected in a Calgary Depression Scale (Addington et al., 1990) total score less than 10. Exclusion criteria were as follows: a) inability to give informed consent; b) diagnosis of schizoaffective disorder; c) substance dependence in the past 6 months; d) positive result on a laboratory toxicology screen performed during the study; e) blood alcohol content N0.05% during any study visit; f) diagnosis of dementia or other neurodegenerative disorder; g) diagnosis of other neurologic disorder (e.g., epilepsy, Tourette's syndrome); h) significant head injury or other brain injury leading to cognitive impairment; i) mental retardation (premorbid IQb70); j) first degree relative with schizophrenia (for controls); k) presence of a psychotic, mood, or substance abuse disorder (for controls); l) ECT treatments within the past 8 weeks. Data reported below are for the patient sample only Data collection sites Data were collected at 4 institutions: 1) Partial hospital and outpatient clinics of the University Behavioral HealthCare system (UBHC) of the University of Medicine and Dentistry of New Jersey (UMDNJ) (Piscataway, New Brunswick, and South Brunswick, NJ); 2) partial hospital and outpatient programs of Thresholds, Inc. (Chicago, IL); 3) partial hospital and outpatient programs at the Weill Medical College of Cornell University, New York Presbyterian Hospital's Payne Whitney Westchester Division (White Plains, NY); and 4) intensive psychiatric rehabilitation treatment (IPRT), day treatment, and pre-vocational and vocational rehabilitation programs of The Zucker Hillside Hospital (Glen Oaks, NY) Cognitive test administration As reported in Silverstein et al. (2010), the MCCB and IntegNeuro were completed within a week of each other, with the order of the batteries counterbalanced across subjects. Each battery was then completed a second time, one month after its initial administration UPSA scoring In Silverstein et al. (2010), the UPSA composite score was calculated as the sum of the raw scores on each subscale. Other studies, however, used a scoring method wherein: a) scores on each scale are converted to a 10-point scale; b) scores are then doubled to reflect a 0 20 range; and c) the resulting 5 scaled scores are summed so that the composite score is expressed within a range of This scoring method gives equal weight to each scale (and the domain tapped by it, such as bill paying skills, navigating via public transportation, following a recipe, etc.). The scoring method used in Silverstein et al. (2010) essentially weights each item equally but provides an estimate of the composite score that is biased by the number of items on each subscale. Therefore, we re-scored the UPSA data from Silverstein et al. (2010) using the method used by other studies so that our data would be comparable to theirs, and to determine whether doing so would alter the magnitude of the correlations with the MCCB and IntegNeuro. 3. Results Table 2 reports Pearson r correlation coefficients, by study site and for all sites combined, between the UPSA and both the MCCB and IntegNeuro, for the initial testing on each battery. Examination of the results for all sites combined indicates that re-scoring of the UPSA had little effect on cognition UPSA correlations, which remained essentially the same as those reported in Silverstein et al. (2010). For example, the Pearson r between the UPSA and MCCB composite scores was 0.47 (N=121), and that between the UPSA and IntegNeuro composite scores was 0.35 (N=115). However, it is also apparent that there were large differences between sites in the magnitude of these correlations. For the MCCB, Pearson r correlations at 3 of the sites (UMDNJ, Thresholds, Cornell) were between 0.57 and 0.60, which is within the range reported in past studies. In contrast, data from Zucker Hillside indicate a correlation of Comparisons of the strength of these correlations, calculated using r-to-z transformations, indicated that Zucker Hillside differed from Cornell, Thresholds and UMDNJ at p values less than 0.05, 0.01, and 0.07 respectively (all tests 2-tailed). This site is hence an outlier with respect to the broader literature, and as a subsample within the Silverstein et al. study, its data are responsible for lowering the combined sample correlation. This can be seen in the table, where the correlation for the other three groups combined is For IntegNeuro, a similar, although less pronounced, situation was observed. Pearson r values at UMDNJ, Thresholds, and Cornell ranged from 0.29 to 0.54, whereas the value at Zucker Hillside was The latter differed from the r values at Cornell, Thresholds and UMDNJ at p values less than 0.06, 0.09, and 0.31 respectively. The correlation for the three other sites combined was 0.46, which approaches the range reported in past studies. Table 3 reports Pearson r values using the data from the second administration of each cognitive test battery, which took place 1 month after the initial testing. These data reveal the same pattern of site differences as were observed at the Time 1 testing. For example, for the MCCB, Pearson r correlations at UMDNJ, Thresholds, and Cornell ranged from 0.55 to 0.56, whereas at Zucker Hillside, r= For the other three groups combined, r=0.54. These values are quite similar to those observed at Time 1. The r value at Zucker Hillside differed from those at Cornell, Thresholds and UMDNJ at p values less than 0.05 in all cases. Interestingly, correlations between UPSA and IntegNeuro scores are higher in some cases at Time 2. For example, at UMDNJ and Thresholds, r values were 0.75 and 0.61,

5 S.M. Silverstein et al. / Psychiatry Research 187 (2011) Table 2 Cognition UPSA correlations at Time 1. Measure Correlation type Site N Correlations coefficient p value IntegNeuro Pearson r UMDNJ Thresholds Cornell Zucker Hillside (ZHH) All sites All sites w/o ZHH MCCB Pearson r UMDNJ Thresholds Cornell Zucker Hillside All sites All sites w/o ZHH Denotes pb0.05. Denotes pb0.01. respectively, whereas at Cornell and Zucker Hillside they were 0.37 and (similar to values at Time 1), respectively. For all sites combined except Zucker Hillside, the r value was The r value at Zucker Hillside differed from those at Cornell, Thresholds, and UMDNJ at p values less than 0.18, 0.01, and 0.001, respectively. These data indicate that, at least for computerized batteries such as IntegNeuro, the magnitude of cognition UPSA relationships may be affected by the degree of familiarity with the computerized test format. Specifically, more valid, and perhaps more reliable, scores may be obtained after an initial practice or training session with the battery, especially for patients who are not computer literate. To better understand site differences in the magnitude of the correlations, we examined score distributions and missing data rates for each measure at each site, at both time points. Table 4 summarizes the number of subjects at each site who had at least 1 missing data point on each measure/battery at each time point. The data clearly indicate, at Time 1, a greater proportion of subjects with missing data at Zucker Hillside for the MCCB and UPSA and a high rate of missing data on IntegNeuro; however, UMDNJ had the highest percentage for this variable. At Time 2, Zucker Hillside also had the highest rate of missing data on the MCCB, and Zucker Hillside and UMDNJ had the most missing data on IntegNeuro. Annotated testing reports indicate that on IntegNeuro, most missing data resulted from an inability to complete the practice trials. When this data was collected, failure to complete a sufficient number of practice trials led IntegNeuro to automatically skip the corresponding test (leading to missing data) and to proceed to the next test in the battery (Silverstein et al., 2010); however, it should be noted that in subsequent versions of IntegNeuro, the test administrator has the option of running a test even if practice trials are failed. On the MCCB, missing data typically resulted from an inability of the subject to understand the task Table 3 Cognition UPSA correlations at Time 2. Measure Correlation type Site N Correlations Coefficient p value IntegNeuro Pearson r UMDNJ Thresholds Cornell Zucker Hillside (ZHH) All sites All sites w/o ZHH MCCB Pearson r UMDNJ Thresholds Cornell Zucker Hillside All sites All sites w/o ZHH Denotes pb0.01. Table 4 Percentage of subjects who had at least 1 missing data point, by measure, site, and timepoint. Time 1 Missing data by site at Time 1 Study site Cases Valid Missing N Percent N Percent MCCB UMDNJ % 1 3.1% Thresholds % 0 0.0% Cornell % 0 0.0% Zucker Hillside % % INTEGNEURO UMDNJ % % Thresholds % 3 7.5% Cornell % 2 6.7% Zucker Hillside % % UPSA UMDNJ % 0 0.0% Thresholds % 0 0.0% Cornell % 0 0.0% Zucker Hillside % 5 9.6% Time 2 Missing data by site at Time 2 Study site Cases Valid Missing N Percent N Percent MCCB UMDNJ % 2 7.1% Thresholds % 1 2.6% Cornell % 0 0.0% Zucker Hillside % % INTEGNEURO UMDNJ % % Thresholds % 3 7.7% Cornell % 2 6.7% Zucker Hillside % % Table 5 Descriptive statistics of UPSA, MCCB, and IntegNeuro data. Measure Site N Mean Standard Deviation Median Min Max Time 1 MSIF UMDNJ Thresholds Cornell Zucker Hillside UPSA UMDNJ Thresholds Cornell Zucker Hillside IntegNeuro UMDNJ Thresholds Cornell Zucker Hillside MCCB UMDNJ Thresholds Cornell Zucker Hillside Time 2 IntegNeuro UMDNJ Thresholds Cornell Zucker Hillside MCCB UMDNJ Thresholds Cornell Zucker Hillside IntegNeuro means are expressed as z scores, relative to a large (N=1000) age and gender matched normative sample.

6 428 S.M. Silverstein et al. / Psychiatry Research 187 (2011) instructions. On the UPSA, where Zucker Hillside was the only site with missing data, this typically resulted from subject refusal to move to where the mock pantry was to complete that scale item, or to unavailability of the pantry. Because Zucker Hillside did not have the highest missing data rate on IntegNeuro, this would suggest that the low correlations at this site are not due to it having the lowest functioning patients. Similarly, the annotated data reports indicate that missing UPSA data were not due to an inability to complete the scale. On the other hand, Zucker Hillside did have the highest missing data rate on the MCCB, and so we explored the possibility that Zucker Hillside patients were lower functioning than those at other sites. To examine this issue, we inspected the means and standard deviations of scores on each measure (see Table 5). Zucker Hillside patients had the lowest mean and smallest standard deviation (SD) on the MCCB, and the lowest mean on IntegNeuro, at Time 1. At Time 2, Zucker Hillside patients had the lowest mean but not smallest SD on the MCCB, and did not have the lowest mean on IntegNeuro. The mean UPSA score at Zucker Hillside was at about the average of the four sites. These results suggest that, in terms of functional capacity, Zucker Hillside patients were equivalent to patients at other sites, and that they did not consistently demonstrate a pattern of lower cognition scores, lower SDs, or higher missing data rates across tasks and timepoints that could account for observed correlations. Although greater cognitive impairment could indirectly affect the magnitude of correlations (via missing data, low motivation and therefore invalid scores, etc.), it could not, itself, be the reason for the unusually low cognition UPSA correlations at Zucker Hillside. Similarly, while any restriction of range in the Zucker Hillside data could affect the correlation at that site, it would not, by itself, lower the overall correlation across all sites where data from all study subjects were combined. Therefore, to further inspect the data, we created scatterplots depicting the relationships between variables at Zucker Hillside, and then at all other sites combined, for each of the cognitive batteries, at each time point (see Fig. 1). These plots demonstrate clearly that the Zucker Hillside data are not characterized by excessively low scores or restricted range on any of the variables, but they also do not reveal any Fig. 1. Scatterplots depicting the relationships between scores on the two cognitive batteries (MCCB and IntegNeuro) and the UPSA, at Zucker Hillside and then at all other sites combined, at each time point. ZHH=Zucker Hillside; Site 1 3=Cornell, Thresholds, and UMDNJ.

7 S.M. Silverstein et al. / Psychiatry Research 187 (2011) } Fig. 1 (continued). other data properties that could account for the low correlations at that site. Finally, examination of site differences in correlations between cognition scores and MSIF scores provide further evidence that the data from Zucker Hillside are valid. For example, at Time 1, Pearson r values between MCCB composite and MSIF scores were 0.35 for Zucker Hillside, 0.12 for Cornell, 0.12 for Thresholds, and 0.29 for UMDNJ (only the correlation at Zucker Hillside was statistically significant). At Time 2, these values were 0.40, 0.08, 0.18, and 0.34, respectively (again, only the correlation at Zucker Hillside was statistically significant). For IntegNeuro, the r values were 0.17, 0.20, 0.10, and 0.09 at Time 1 and 0.11, 0.03, 0.08, and 0.10 at Time 2, respectively. These data indicate that, with the MSIF, which is an assessment of vocational, residential, and educational functioning as obtained via interview data, cognition functioning relationships were equal to those at the other sites (note none of the r values are significantly different from each other). Interestingly, MSIF and UPSA scores were generally independent, with the overall correlation among patients at r= 0.01, p=0.90, and with no differences across sites: r values at Zucker Hillside, Cornell, Thresholds, and UMDNJ were 0.12, 0.11, 0.12, and 0.36, respectively (only the latter was statistically significant; note higher scores on the UPSA indicate better functioning whereas higher scores on the MSIF indicate greater impairment). In short, the significant cognition functioning relationships demonstrated using the MSIF at Zucker Hillside suggest that a relationship does exist between cognition and functioning among patients at this site, raising once again the question of why it was only at this site that such a relationship was not demonstrated with the UPSA. 4. Discussion Results from these additional analyses of the data from Silverstein et al. (2010) partially clarify why the correlations between cognition and UPSA scores in that study were lower than in previously reported studies. Specifically, it is not because of a difference in how the UPSA was scored. Rather, it is clearly due to site differences in the magnitude of the correlations, with one site in particular demonstrating

8 430 S.M. Silverstein et al. / Psychiatry Research 187 (2011) unusually low correlations. At a deeper level, however, it remains unclear why the correlations at this one site were so different from those from other sites, especially when scores on the dependent variables, and SDs, were similar to those of schizophrenia patients in other studies. Although Zucker Hillside patients did demonstrate the most impaired cognitive functioning, they did not consistently do so; sufficient range was observed in the Zucker Hillside data; and this site did not have the highest missing data rate on both cognitive batteries, ruling out these factors as explanations for the low correlations. Perhaps more importantly, cognition functioning relationships have been observed even among schizophrenia patients who have spent years in a state hospital (Silverstein et al., 1998), and so poor cognitive functioning by itself would not account for low correlations with functional outcome measures. Moreover, given that the means, SDs, and missing data rates at Zucker Hillside were not consistently lowest/highest compared to other sites, and were comparable to those in past studies, this suggests that test administrator error is unlikely to account for the correlational findings. Also arguing against test administrator error is the high experience level of the research staff at Zucker Hillside and the long history of research at that site using measures such as those used in the current study. In addition, the similar correlations between cognition and UPSA scores, on both a fully automated computerized battery (IntegNeuro) and a largely paper-and-pencil battery (MCCB), suggest that scores reflect actual cognitive ability and are not a function of problems in test administration. Finally, analysis of site differences in cognition MSIF correlations indicated that Zucker Hillside was the only site where significant correlations were observed. This suggests that cognition functioning relationships can be observed among patients at this site, even if this did not occur with the UPSA in this study. Therefore, there is no obvious explanation for the unusual results at Zucker Hillside, other than the possibility that the cohort there had characteristics such that the normally tight relationship between cognition and UPSA scores did not exist. One speculation is that, since a significant minority of patients at this site: 1) were in work programs with associated money management requirements and requirements to function independently; 2) regularly used public transportation to get to treatment appointments and work settings, since the hospital is in Queens, NY; and 3) would have typically earned more money than most schizophrenia patients which would allow for greater independence in leisure activities, they may have been able to obtain higher scores on the UPSA (which includes items related to use of public transportation, bill paying, planning leisure activities, etc.) than schizophrenia patients with a similar level of cognitive impairment but who function in different community environments and are not engaged in work. That such a scenario is possible is supported by anecdotal evidence. For example, at UCLA, in the late 1980s, an effort was made to develop a skills training module for chronic schizophrenia patients that focused on effective use of public transportation. However, this was abandoned when it became apparent that the patients knew more about how to navigate the Los Angeles public transportation system than did the researchers who were developing the intervention (C. Wallace, personal communication, June, 1993). In the case of the present study, such a de-coupling of cognitive ability and level of functioning, caused by specific types of life experience, would weaken the typical relationship between cognition and UPSA scores (but not cognition MSIF scores), and reduce the sensitivity of the UPSA to cognitive change. Since not all patients in the Zucker Hillside sample were in work programs, however, the heterogeneity in the sample may have further reduced the magnitude of the correlation at this site. While we cannot definitively clarify whether this speculation is correct, we believe it will be important in future studies to determine whether specific subpopulations of patients may have characteristics or life experiences that change the typical cognition function relationship. In addition, the low correlation between MSIF and UPSA scores indicates that variation in cognition functioning relationships across sites can be a function of the type of outcome measure that is used (e.g., performance-based, informant, interview-based, etc.). Keeping these issues in mind, it is clear that, at three of the four sites in the study, correlations between MCCB and IntegNeuro scores and UPSA scores were close to, or within the range reported in past studies. An interesting finding from this re-analysis was that correlations between UPSA and IntegNeuro scores were higher at the second testing session. This suggests that greater familiarity with the computerized test format leads to more valid scores, less error variance, and higher correlations with variables with which relationships exist (e.g., functioning), and that when using computerized batteries, a more valid assessment of the cognition functioning relationship can be gained by a prior, training/testing session. It is also possible, of course, that the Time 1 IntegNeuro UPSA score relationship is more valid, but it would be difficult to support the claim that cognition scores that reflect greater familiarity with a test (i.e., Time 2 scores) are less valid than initial scores, especially on a battery such as IntegNeuro where alternate stimulus sets are used upon repeat administrations. Moreover, since correlations between the MCCB and UPSA did not differ between Time 1 and Time 2, it is unlikely that practice effects per se affected the size of the correlation. In support of the hypothesis that the Time 2 correlations were more valid is the finding that the Time 2 IntegNeuro UPSA score correlation was more similar to correlation values reported in past studies using other batteries than was the Time 1 correlation. Taken together, these data support past studies in supporting use of the UPSA as a co-primary outcome variable in studies of cognitive change in schizophrenia. Acknowledgments We thank Phil Harvey for thoughtful comments and observations on the Silverstein et al. (2010) paper that led to the additional analyses presented here. The study from which the data were extracted was funded by a contract from AstraZeneca to the first author. References Addington, D., Addington, J., Schissel, B., A depression rating scale for schizophrenics. Schizophrenia Research 3, Birchwood, M., Smith, J., Cochrane, R., Wetton, S., Copestake, S., The Social Functioning Scale: the development and validation of a new scale of social adjustment for use in family intervention programs with schizophrenic patients. The British Journal of Psychiatry 157, Bowie, C.R., Depp, C., McGrath, J.A., Wolyniec, P., Mausbach, B.T., Thornquist, M.H., Thornquist, M.H., Luke, J., Patterson, T.L., Pulver, A.E., Prediction of real-world functional disability in chronic mental disorders: a comparison of schizophrenia and bipolar disorder. The American Journal of Psychiatry 167, Depp, C.A., Mausbach, B.T., Eyler, L.T., Palmer, B.W., Cain, A.E., Lebowitz, B.D., Patterson, T.L., Jeste, D.V., Performance-based and subjective measures of functioning in middle-aged and older adults with bipolar disorder. The Journal of Nervous and Mental Disease 197, First, M.B., Spitzer, R.L., Gibbon, M., Williams, J.B.W., Structured Clinical Interview for DSM IV Axis I Disorders-patient Edition (SCID-I/P Version 2.0). Biometrics Research Department, New York State Psychiatric Institute, New York, New York. Green, M.F., What are the functional consequences of cognitive deficits in schizophrenia? The American Journal of Psychiatry 153, Green, M.F., Kern, R.S., Braff, D.L., Mintz, J., Neurocognitive deficits and functional outcome in schizophrenia: Are we measuring the right stuff? Schizophrenia Bulletin 26, Harvey, P.D., Helldin, L., Bowie, C.R., Heaton, R.K., Olsson, A., Hjärthag, F., Norlander, T., Patterson, T.L., Performance-based measurement of functional disability in schizophrenia: a cross-national study in the United States and Sweden. The American Journal of Psychiatry 166, Jaeger, J., Berns, S.M., Czobor, P., The multidimensional scale of independent functioning: a new instrument for measuring functional disability in psychiatric populations. Schizophrenia Bulletin 29, Juckel, G., Morosini, P.L., The new approach: psychosocial functioning as a necessary outcome criterion for therapeutic success in schizophrenia. Current Opinion in Psychiatry 21, Kay, S.R., Opler, L.A., Fiszbein, A., The positive and negative syndrome scale (PANSS) for schizophrenia. Schizophrenia Bulletin 13, Keefe, R.S.E., Poe, M., Walker, T.M., Harvey, P.D., 2006a. The relationship of the Brief Assessment of Cognition in Schizophrenia (BACS) to functional capacity and realworld functional outcome. Journal of Clinical and Experimental Neuropsychology 28,

9 S.M. Silverstein et al. / Psychiatry Research 187 (2011) Keefe, R.S.E., Poe, M., Walker, T.M., Kang, J.W., Harvey, P.D., 2006b. The Schizophrenia Cognition Rating Scale: an interview-based assessment and its relationship to cognition, real-world functioning, and functional capacity. The American Journal of Psychiatry 163, Keefe, R.S.E., Hurley Fox, K., Harvey, P.D., Cucchiaro, J., Siu, C., Loebel, A., 2009a. Test retest characteristics of the MATRICS Consensus Cognitive Battery in a 29-site schizophrenia clinical trial of Lurasidone HCl Versus Risperidone. Presented at the meeting of The International Society for CNS Clinical Trials and Methodology, San Diego, November Keefe, R.S.E., Siu, C., Hurley Fox, K., Lowe, D.A., Abdullah, A.R., Garabaldi, G., Santarelli, L., Murray, S.R., 2009b. Test retest characteristics of the MATRICS Consensus Cognitive Battery In a 20-site schizophrenia clinical trial of R3487/MEM3454 versus placebo. Schizophrenia Bulletin 35 (Suppl. 1), Leifker, F.R., Patterson, T.L., Heaton, R.K., Harvey, P.D., Validating measures of realworld outcome: the results of the VALERO expert survey and RAND panel. Schizophrenia Bulletin. doi: /schbul/sbp044. Mausbach, B.T., Harvey, P.D., Goldman, S.R., Jeste, D.V., Patterson, T.L., Development of a brief scale of everyday functioning in persons with serious mental illness. Schizophrenia Bulletin 33, McClure, M.M., Bowie, C.R., Patterson, T.L., Heaton, R.K., Weaver, C., Anderson, H., Harvey, P.D., Correlations of functional capacity and neuropsychological performance in older patients with schizophrenia: evidence for specificity of relationships? Schizophrenia Research 89, Miles, A.A., Heinrichs, R.W., Ammari, N., in press. Real world functioning in schizophrenia patients and healthy adults: assessing validity of the Multidimensional Scale of Independent Functioning. Psychiatry Research. Nuechterlein, K.H., Green, M.F., Kern, R.S., Baade, L.E., Barch, D.M., Cohen, J.D., Essock, S., Fenton, W.S., Frese 3rd, F.J., Gold, J.M., Goldberg, T., Heaton, R.K., Keefe, R.S., Kraemer, H., Mesholam-Gately, R., Seidman, L.J., Stover, E., Weinberger, D.R., Young, A.S., Zalcman, S., Marder, S.R., The MATRICS Consensus Cognitive Battery, Part 1: Test selection, reliability, and validity. The American Journal of Psychiatry 165, Patterson, T.L., Goldman, S., McKibbin, C.L., Hughs, T., Jeste, D.V., UCSD Performance-Based Skills Assessment: development of a new measure of everyday functioning for severely mentally ill adults. Schizophrenia Bulletin 27, Paul, R.H., Lawrence, J., Williams, L.M., Richard, C.C., Cooper, N., Gordon, E., Preliminary validity of Integneuro : a new computerized battery of neurocognitive tests. The International Journal of Neuroscience 115, Pietrzak, R.H., Olver, J., Norman, T., Piskulic, D., Maruff, P., Snyder, P.J., A comparison of the CogState Schizophrenia Battery and the Measurement and Treatment Research to Improve Cognition in Schizophrenia (MATRICS) Battery in assessing cognitive impairment in chronic schizophrenia. Journal of Clinical and Experimental Neuropsychology 14, Silverstein, S.M., Schenkel, L.S., Valone, C., Nuernberger, S., Cognitive deficits and psychiatric rehabilitation outcomes in schizophrenia. The Psychiatric Quarterly 69, Silverstein, S.M., Jaeger, J., Donovan-Lepore, A.M., Wilkniss, S.M., Savitz, A., Malinovsky, I., Hawthorne, D., Raines, S., Carson, S., Marcello, S., Zukin, Z.R., Furlong, S., Dent, G., A comparative study of the MATRICS and IntegNeuro cognitive assessment batteries. Journal of Clinical and Experimental Neuropsychology 32, Simpson, G.M., Angus, J.W.S., A rating scale for extrapyramidal side effects. Acta Psychiatrica Scandinavica 212, Twamley, E.W., Doshi, R.R., Nayak, G.V., Palmer, B.W., Golshan, S., Heaton, R.K., Patterson, T.L., Jeste, D.V., Generalized cognitive impairments, ability to perform everyday tasks, and level of independence in community living situations of older patients with psychosis. The American Journal of Psychiatry 159, Ventura, J., Nuechterlein, K.H., Subotnik, K.L., Gutkind, D., Gilbert, E.A., Symptom dimensions in recent-onset schizophrenia and mania: a principal components analysisof the24-item Brief Psychiatric Rating Scale. PsychiatryResearch 97,