The Role of Attention in Wisconsin Card Sorting Test Performance

Transcription

1 Pergamon Archives of Clinical Neuropsychology, Vol. 11, No. 3, pp , 1996 Copyright 1996 National Academy of Neuropsychology Printed in the USA. All rights reserved /96 $ SSDI (95) The Role of Attention in Wisconsin Card Sorting Test Performance Kevin W. Greve, Mary C. Williams, William G. Haas, Richard R. Littell, and Carlos Reir~oso Department of Psychology, University of New Orleans The present study sought to test the hypothesis that the second factor (consisting of Failure-to- Maintain-Set and other scores)found in two recent factor analyses of the Wisconsin Card Sorting Test reflects attentional function. The effect of color overlays (an experimental manipulation known to influence neural systems linked to attention) was examined in 17 normal control and 14 attention-disorded children (ages 8 to 12). Group and Color main effects were found for Factor I (which consists largely of measures of perseveration) and a Color main effect was observed for Factor 2. The Color effect for Factor 2 supported the contention that this factor reflects attentional processes. A hypothesis concerning the relationship between problem solving and attention on the WCST is offered and a means for testing it is discussed. As the role of clinical neuropsychology has shifted from diagnosis to cognitive rehabilitation and patient management, the need to understand the cognitive processes underlying performance on specific tests has increased (Mapou, 1988; Milberg, Hebben, & Kaplan, 1986). Several different cognitive processes contribute to what is referred to as "executive function" (Shallice, 1988; Stuss & Benson, 1986), yet measures commonly used to evaluate the executive system (e.g., Category Test, Wisconsin Card Sorting Test) seem unable to differentiate among those processes (Delis, Squire, Bihrle, & Massman, 1992). One solution to this problem is to develop new measures specifically designed to fractionate executive function into its component processes (e.g., Delis et al., 1992). An alternative technique is to examine the factor structure of existing tests with the hope of uncovering meaningful factors that point to the existence of dissociable cognitive processes underlying test performance. Despite its importance as a clinical measure and its use in research for almost 50 years, the Wisconsin Card Sorting Test (WCST; Grant & Berg, 1948; Heaton, 1981) was not subjected to factor analytic study until recently (Greve et al., 1993; Sullivan et al., 1993). This study was supported in part by a grant to the first author from the University of New Orleans Research Council. Address correspondence to: Kevin W. Greve, PhD. Department of Psychology, University of New Orleans-Lakefront, New Orleans, LA

2 216 K. W. Greve et al. Sullivan et al. (1993) examined the performance of 58 (schizophrenic, alcoholic, normal control) subjects on 11 WCST scores and obtained a three-factor solution, which accounted for 91% of the variance. The first factor, labeled "Perseveration," consisted of 7 of the 11 scores and accounted for 58% of the variance. The second factor, which was named "Inefficient Sorting" and accounted for 19% of the variance, consisted of two scores of which failure-tomaintain-set (FMS) was highest loading. The final factor, which was called "Nonperseverative errors" and accounted for 14% of the variance, consisted of nonperseverative errors and "unique" responses (those that match no dimension). Examining the construct validity of the obtained factors, Sullivan et al. concluded that "Perseveration" (Factor 1) and "Nonperseverative Errors" (Factor 3) reflected executive and memory function (either purely or interacting, depending on subject group). "Inefficient Sorting" (Factor 2) appeared unrelated to either process, despite the fact that her frontal lesion patients performed poorly on it. Greve et al. (1993), examining the performance of 270 patients and normal controls, also reported a factor solution that accounted for 91% of the observed WCST variance. However, their solution differed from that of Sullivan et al. (1993) in reporting only two factors. The fh'st factor, which was called "Problem-solving" and accounted for 70% of the variance, had essentially the same composition as Sullivan's "Perseveration" factor. The second factor, which was called "FMS," accounted for 21% of the variance and is very similar to Sullivan's "Inefficient Sorting" factor. (See Table 1 for a comparison of the two analyses.) Unlike Sullivan, Greve et al. did not directly test the construct validity of these factors but looked to the literature for clues to the nature of their underlying processes. Consistent with Sullivan, Greve interpreted the first factor as reflecting executive function, specifically problem solving. However, where Sullivan could not offer a clear explanation of the second factor, Greve interpreted it as a measure of attentional processes. Support for the suggestion that Factor 2 reflects attentional dysfunction was derived largely from a study of WCST performance in which color overlays were used to manipulate visual attention (Williams, Littell, Reinoso, & Greve, 1994). Several neuropsychological models of attention (Colby, 1991; Posner & Peterson, 1990) predicted that the wavelength (color) of a stimulus would affect processing in the dorsal neural stream, which is believed to mediate key attentional operations. In general, short wavelengths TABLE 1 Factor Structure of the Wisconsin Card Sorting Test Factors Sullivan et al. (1993) Greve et al. (1993) Variance proportion ! Eigenvalues Perseverative errors.99*.06.95* -.12 Perseverative responses.98*.02.93* -.13 % Perseverative errors.98* Total errors.93*.i 7.98*.00 % Conceptual level responses -.90* *.03 Categories completed -.81" *.02 Total correct -.92* * Failure-to-maintain-set *.18.93" Correct- 10/category.47.85* *Indicates that the score had a high and relatively independent loading of that factor.

3 Card Sorting and Attention 217 (blue) were predicted to enhance performance while long wavelengths (red) were expected to cause impairment. This hypothesis has been supported in several studies of the effects of wavelength on such attentional tasks as the Posner Paradigm (Wildzunas, 1993) and the Trail-Making Test (Littell, 1993). In the Williams et al. (1994) study, perseveration scores (those that loaded on Factor 1) were unaffected by color, while FMS (which loaded on Factor 2) was significantly improved by the blue overlays and impaired by the red overlays. The obvious implication of these results is that Factor 2 reflects attentional processes. However, because this interpretation was not based directly upon the results of the Greve et al. (1993) study, but was instead based upon evidence in the literature, the interpretation of these factors is not entirely satisfying and begs for further research directly examining the construct validity of the factors. Thus, the purpose of the present study is to make a direct test of the hypothesis that Factor 2 measures attention. If performance on the scores that comprise Factor 2 reflects attentional function, then the value of Factor 2 should be affected by the wavelength manipulation. Specifically, we would expect the blue overlays to enhance attentional function and reduce the Factor 2 score while the red overlays would have the opposite effect. Further, we would predict that Factor 1, on which the measures of perseveration loaded highly, would be unaffected by the overlay color. To test this hypothesis, we reanalyzed the Williams et al. (1994) data by computing values for each factor based upon both Greve et al.'s (1993) and Sullivan et al.'s (1993) factor analyses. Subjects METHOD Subjects were 31 children between 8 and 12 years of age. Fourteen children were classified as attention disordered (AD) and 17 as normal (NC), based on their performance on the Gordon Diagnostic System (GDS; Gordon, 1989). The groups were matched for age, sex, and IQ (as measured by the Kaufman Brief Intelligence Test; Kaufman & Kaufman, 1990). Specifically, the mean age of the NC group was 10.1 years and that of the AD group was 10.6 years; the difference was not significant, t(30) =.32. The mean IQ of the NC group was 108.2, while the AD group had an average IQ of 100.2; this differences was also not significant, t(30) =.51. There were 11 males and six females in the NC group and 10 males and four females in the AD group. All children scored within the normal to above normal range on intelligence and were from middle-class families. Procedure Four sets of WCST cards were utilized. The cards were inserted into blue, red, gray, and clear plastic jackets, resulting in four complete sets, each constituting a separate color condition. The gray plastic was used as a control for the contrast reduction produced by the colored overlays and the clear plastic served as a placebo control condition. The WCST was administered in standard fashion (Heaton, 1981) four times to each subject, once in each of the four color conditions, with the order of presentation counterbalanced across subjects. The tests were scored with a computer program distributed by Psychological Assessment Resources (Psychological Assessment Resources, 1990). The following seven scores were used in Greve et al.'s (1993) analysis: total errors, perseverative errors, perseverative responses, categories achieved, percent conceptual level responses, failure-to-maintain-set, and total correct. The following four additional scores were used in the Sullivan et al. (1993) factor

4 218 K. W. Greve et al. analysis: total correct minus 10 per category achieved, nonperseverative errors, percent perseverative errors, "unique" responses. Sullivan's Factor 3 score could not be calculated because this data set did not contain the highest loading of its two scores ('Mnique" responses). Two sets of scores representing each factor were computed based upon each of the factor analyses. All WCST scores required conversion to z-scores based on the distribution of the present sample prior to computing the factor scores. The factor scores based upon Greve's analysis were calculated using the standardized factor scoring coefficients presented in Table 2. These will be referred to as Factor Scores. Composite Scores based upon Sullivan's analysis were created using the method reported by Sullivan et al. (1993). Specifically, the z-scores for the highest loading variables on each factor were summed, z-scores of variables with a negative loading were weighted negatively. Each variable was represented in only one composite score. For all scores, higher values reflect poorer performance. RESULTS Developmental research has shown improvement in WCST performance with increasing age and a reduction in WCST differences between normal and attention-disordered subjects as they reach adolescence (Boucugnani & Jones, 1989; Chelune, Ferguson, Koon, & Dickey, 1986; Chelune & Thompson, 1987). Consequently, the factor scores derived via the Greve et al. (1993) and Sullivan et al. (1993) methods were submitted to a two (Group) by four (Color) repeated-measures analysis of covariance (ANCOVA) with age as the covariate. This covariate approach is consistent with that of Williams et al. (1994) in their analysis of the original variables. Factor Score Analysis The ANCOVA on the Factor 1 scores indicated a significant Group main effect [F(1, 28) = 12.09, p <.05] with the attention disordered subjects (M =.328, SD = 1.32) performing more poorly than the control subjects (M = -.257, SD =.52) across all color conditions. There was also a significant Color main effect [F(3, 87) = 3.56, p <.03]. Follow-up dependent t-tests (ix =.01 to protect against Type I error) indicated that performance with the blue overlays was significantly better than performance with the clear overlays. The ANCOVA for the Factor 2 Factor Scores indicated a significant Color main effect [F(3, 87) = 10.28, p =.01]. The follow-up t-tests indicated that the only significant difference was between the Blue and Red conditions; the Clear and Gray did not differ from any condition although the difference between Clear and Blue approached significance (p =.019). Neither the Group main effect TABLE 2 Standardized Factor Scoring Coefficients for the Greve et al. (1993) Factor Analysis Factor 1 2 Total errors Perseverative errors Perseverative responses.i Categories completed % Conceptual level responses Failure-to-maintain-set. i Total correct

5 Card Sorting and Attention 219 for Factor 2 nor the Group by Color interaction for either factor were significant. Age was not a significant covariate in either model [Factor 1: F(1, 28) = 2.06, p =.16; Factor 2: p(1, 28) = 1.06, p =.31]. Table 3 presents the Color main effect means and standard deviations. Composite Score Analysis There was a significant group main effect for the Factor 1 Composite Score [IF(l, 28) = 4.40, p <.05] with the attention disordered (M = 1.85, SD = 7.57) performing more poorly than the normal controls (M = -1.46, SD = 3.00). Neither the Color main effect nor the Group by Color interaction was significant. The Factor 2 Composite Score showed a significant Color main effect [F(3, 87) = 10.76, p <.01]; the follow-up tests indicated that performance in the Red and Clear conditions was significantly worse than in the Blue condition. Clear did not differ from Gray nor Red. Neither the Group nor interaction effects were significant. As in the previous analyses, age was not a significant covariate in either model [Factor I: F(1, 28) = 1.79,p =.19; Factor 2: F(1, 28) = 1.95,p =.17]. DISCUSSION The hypothesis that color would affect performance on Factor 2 was supported; however, contrary to expectation, a weak but significant color effect was observed for Greve's Factor 1 score. Because the wave length manipulation affects attention and the neural system influenced by color has been implicated in attentional disturbances including Attention Deficit Disorder (Heilman, Voeller, & Nadeau, 1991) the finding of a color effect on Factor 2 is consistent with Greve et al.'s (1993) interpretation of that factor as reflecting attentional function. However, because group assignment was based on a direct measure of attention (i.e., the Gordon Diagnostic System) the observation of a group main effect on Factor 1 and its absence on Factor 2 (the "attention" factor) needs explanation as does the unexpected color effect observed on Greve's Factor 1. These explanations may shed some light on the relationship between the cognitive processes underlying performance on the two factors and the value of the factor scores in measuring these processes. Successful performance on the WCST requires that the subject (a) determine the correct response dimension, then (b) maintain responding to that dimension. This is a hierarchical process in that the first component must be achieved before the second is possible. Determining the correct dimension is the problem-solving component of the WCST and TABLE 3 Color Means and Standard Deviations for the Factor and Composite Scores Greve et al. (1993) Sullivan et al. (1993) Factor 1 Factor 2 Factor 1 Factor 2 Blue M b b a c Gray SD M (1.13).118 a.b (.90) a.b (6.44) a (1.46) b'c Clear SD (.93) (I.03) M.247 a.098 a'b (5.41) a (2.00).152 a'b SD (I.03) (.78) (5.86) (1.57) Red M.084 a,b.40 a.388 a.754 a SD (.94) (1.14) (5.39) (2.31) a,b,ccolumn means with the same letter are not significantly different at t~ <.05.

6 220 K. W. Greve et al. requires that a variety of hypotheses be considered and rejected if they prove incorrect. Thus, cognitive flexibility is important. The scores that load most highly on Factor 1 are those that measure the ability to quickly and efficiently test hypotheses and discover the correct dimension. The error scores indicate the selection of an incorrect dimension and the perseveration scores reflect an inability to shift from an incorrect dimension (i.e., to test other hypotheses). In the present study, the attention-disordered children had far more difficulty solving the problem posed by the WCST than did the control children. This is consistent with the findings of numerous other studies of frontal-executive function in attention-disordered children (see Barkley, Grodzinsky, & DuPaul, 1992, for a review). If one cannot solve the problem and discover the correct dimension, then the ability to maintain responding to that dimension is irrelevant. Thus, the normal control children had low scores on Factor 2 because they effectively solved the problem and had the attentional resources to maintain correct responding. On the other hand, the attention-disorder children had low scores on this factor because they had difficulty solving the problem and, thus, had little opportunity for their impaired attentional function to impact their ability to maintain responding. If this explanation is accurate, then one would hypothesize that an intervention that improves the problem-solving ability of the attention-disordered children would improve their scores on Factor 1, thereby reducing or eliminating the group differences. Further, because the impaired children would now know how to respond correctly, pressure would be placed on their attentional resources to maintain correct responding. Because they are impaired attentionally by definition, they should have difficulty in this respect and show a concomitant rise in Factor 2 scores, which would result in the appearance of group differences. There is some evidence in the literature supporting this hypothesis. A number of articles report attempts to teach card sorting to schizophrenic patients, a population that has considerable difficulty on frontal-executive tasks in general and the WCST in particular. Training resulted in a decrease in perseveration (perseverative responses or perseverative errors) and an increase in correct responses and categories completed (Bellack, Mueser, Morrison, Tiemey, & Podell, 1990; Goldberg, Weinberger, Berman, Plishkin, & Podd, 1987; Green, Ganzell, Satz, & Vaclav, 1990; Green, Satz, Ganzell, & Vaclav, 1992). The results of these studies support the first part of the hypothesis in that all these scores load exclusively on Factor 1 (with the exception of total correct, which loads almost equally on both factors in the Greve et al., 1993, analysis). However, because researchers often do not include FMS in their analyses, evidence relating to the second part of the hypothesis is sparse but available. Concomitant with a decrease in perseverative responses after training, Schneider and Asamow's (1987) child schizophrenic patients showed a small, though statistically insignificant, increase in "lost sets" (strings of three to nine correct responses). Further, there was a significant increase in nonperseverative errors among the schizophrenics, which was interpreted as deterioration. However, this may actually reflect a qualitative change in response style from inflexible perseverative responding to a flexible (though apparently ineffective) search for the correct dimension. Thus, it may be that there was no significant increase in lost sets because the training program failed to fully correct the problem-solving deficit. Notably, however, Stuss et al. (1983) reported a significant increase in the number of lost sets (strings of three to five consecutive correct) after training for their leucotomized schizophrenics but not for the normal controls. Thus, these data offer limited support of the hypothesis that removing the problem-solving element of the WCST should provide an opportunity to observe attentional dysfunction manifest as difficultly maintaining set. Taken together, the above data support the assertion that Factor 2 reflects attentional function despite the fact that no group effect was observed on Factor 2.

7 Card Sorting and Attention 221 It was argued above that Factor 1 represents nonattentional problem-solving processes. Yet, the finding that an attentional manipulation influenced performance on this Factor seems at odds with that interpretation. A careful examination of the role of attention in cognition and the statistical procedures used to analyze the data may help explain this discrepancy. Attention is a basic cognitive process and attentional deficits can disrupt even intact higher level functions (Lezak, 1995). This implies that improving attentional function should result in improved behavioral performance based on these higher processes. Regarding the present study, the use of the blue overlay would be expected to improve attentional function generally. The measure that would be most sensitive to this effect would obviously be the one with the largest attentional component (i.e., Factor 2). Nonetheless, improvement on Factor 1 would be expected to the degree that Factor 1 requires intact attentional processing. Because Factors 1 and 2 are orthogonal, the role of attention on Factor 1 would be small relative to Factor 2 and the resulting color effect would also be small. This is exactly what was observed. Of course, the color effect was observed only for Greve's Factor 1 score but not for Sullivan's Factor 1 composite score. This difference probably derives from two sources. First, Sullivan's factor analysis had a subject to variable ratio of 6.4:1 (compared to Greve et al.'s ratio of 38.6:1), which approaches the minimum acceptable ratio of 5:1 and is well below the recommended value of 10:1 (Tabachnick & Fidell, 1989). Smaller subject to variable ratios result in factor structures that are less stable and, thus, introduce more noise when that structure is used to derive factor scores in another sample. Consequently, scores based on Sullivan's analysis may be less sensitive to subtle effects because they are less reliable. The second source of discrepancy is related to how the factor scores were derived. While Greve et al.'s (1993) factors were technically orthogonal, the use of the standardized scoring coefficients meant that each score would contribute a small amount to the computation of the other factor score for the factor on which it had a low loading. In contrast, Sullivan et al. (1993) forced their factor structure to be perfectly orthogonal by giving the high loading variables a coefficient of 1 and the low loading variables a coefficient of 0. The result is that Greve's Factor 1 retains some measure of attention and is, thus, mildly sensitive to the color effect, while Sullivan's is a relatively pure measure of problem solving that is not influenced by color. Thus, the color effect seen for Greve's Factor 1 is not inconsistent with the interpretation of Factor 2 as a measure of attention. In summary, the present study sought to test the hypothesis that Factor 2 of both the Greve et al. (1993) and Sullivan et al. (1993) factor analyses reflects the function of underlying attentional processes. The fact that Factor 2 scores were affected by an experimental manipulation (color overlays), which is known to influence neural systems linked to attention, supported the contention that this factor reflects attentional processes. However, the lack of a group main effect cast some doubt on this conclusion. It was noted, however, that successful problem solving is necessary before difficulty in response maintenance (in the form of higher FMS) can be observed. Thus, removal of the problem-solving element via training or instruction should result in an increase in Factor 2 scores for the attention-disordered children. Such an outcome would provide strong additional support for the construct validity of Factor 2 as an attentional measure. REFERENCES Barkley, R. A., Grodzinsky, G., & DuPaul, G. J. (1992). Frontal lobe functions in attention deficit with and without hyperactivity: A review and research report. Journal of Abnormal Child Psychology, 20, Bellack, A. S., Mueser, K. T., Morrison, R. L., Tierney, A., & Podell, K. (1990). Remediation of cognitive deficits in schizophrenia. American Journal of Psychiatry, 147,

8 222 K. W. Greve et al Boucugnani, L. L., & Jones, R. W. (1989). Behaviors analogous to frontal lobe dysfunction in children with attention deficit hyperactivity disorder. Archives of Clinical Neuropsychology, 4, Chelune, G. J., Ferguson, W., Koon, R., & Dickey, T. O. (1986). Frontal lobe disinhibition in attention deficit disorder. Child Psychiatry and Human Development, 16, Chelune, G. J., & Thompson, L. L. (1987). Evaluation of the general sensitivity of the Wisconsin Card Sorting Test among younger and older children. Developmental Neuropsychology, 3, 8!-89. Colby, C. L. (1991 ). The neuroanatomy and neurophysiology of attention. Journal of Child Neurology, 6, $90-S 118. Delis, D. C., Squire, L. R., Bihrle, A., & Massman, P. (1992). Componential analysis of problem-solving ability: Performance of patients with frontal lobe damage and amnesic patients on a new sorting test. Neuropsychologia, 30, Goldberg, T. E., Weinberger, D. R., Berman, K. E, Plishkin, N. H., Podd, M. H. (1987). Further evidence for dementia of the prefrontal type in schizophrenia? A controlled study of teaching the Wisconsin Card Sorting Test. Archives of General Psychiatry, 44, Gordon, M. (1989). Interpretive guide to the Gordon Diagnostic System. Syracuse, NY: Gordon Systems. Grant, D. A., & Berg, E. A. (1948). A behavioral analysis of degree of reinforcement and ease of shifting to new responses in a Weigl-type card sorting problem. Journal of Experimental Psychology, 38, Green, M. E, Ganzell, S., Sate, E, & Vaclav, J. E (1990). Teaching the Wisconsin Card Sorting Test to schizophrenic patients. Archives of General Psychiatry, 44, Green, M. E, Satz, E, Ganzell, S., & Vaclav, J. E (1992). Wisconsin Card Sorting Test performance in schizophrenia: Remediation of a stubborn deficit. American Journal of Psychiatry, 149, Greve, K. W., Brooks, J., Crouch, J., Rice, W. J., Cicerone, K., & Rowland, L. (1993). Factorial structure of the Wisconsin Card Sorting Test. The Clinical Neuropsychologist, 7, Heaton, R. K. (1981 ). Wisconsin Card Sorting Test manual. Odessa, FL: Psychological Assessment Resources, Inc. Heilman, K., Voeller, K., & Nadeau, S. (1991). A possible pathophysiologic substrate of attention deficit hyperactivity disorder. Journal of Child Neurology, 6, $76-$81. Kaufman, A. S., & Kaufman, N. L. (1990). Kaufman Brief Intelligence Test. Circle Pines, MN: American Guidance Service. Lezak, M. D. (1995). Neuropsychological assessment (3rd ed.). New York: Oxford University Press. Littell, R. R. (1993). The effects of color on the Trail Making Test. Unpublished master's thesis. University of New Orleans, New Orleans, LA. Mapou, R. L. (1988). Testing to detect brain damage: An alternative to what may no longer be useful. Journal of Clinical and Experimental Neuropsychology, 10, Milberg, W. P., Hebben, N., & Kaplan, E. (1986). The Boston Process Approach to neuropsychological assessment. In I. Grant & K. M. Adams (Eds.), Neuropsychological assessment ofneuropsychiatric disorders (pp ). New York: Oxford University Press. Psychological Assessment Resources, Inc. (1990). Wisconsin Card Sorting Test: Scoring program (Version 3.0). Odessa, FL: Author. Posner, M., & Peterson, S. (1990). The attention system of the human brain. Annual Review of Neuroscience, 13, Schneider, S. G., & Asarnow, R. E (1987). A comparison of cognitivejneuropsychological impairments of nonretarded autistic and schizophrenic children. Journal of Abnormal Child Psychology, 15, Shallice, T. (i 988). From neuropsychology to mental structure. Cambridge: Cambridge University Press. Stuss D. T., & Benson, D. E (1986). The frontal lobes. New York: Raven Press. Stuss, D. T., Benson, D. E, Kaplan, E. E, Weir, W. S., Nastier, M. A., Lieberman, I., & Ferrill, D. (1983). The involvement of orbitofrontal cerebrum in cognitive tasks. Neuropsychologia, 21, Sullivan, E. V., Mathalon, D. H., Zipursky, R. B., Kersteen-Tucker, Z., Knight, R. T., & Pfefferbanm, A. (1993). Factors of the Wisconsin Card Sorting Test as measures of frontal-lobe function in schizophrenia and in chronic alcoholism. Psychiatry Research, 46, Tabachnick, B. G., & Fidell, L. S. (1989). Using multivariate statistics (2rid ed.). New York: Harper Collins Publishers. Wildzunas, R. (1993). Transient deficits in the magnoceuular visual subsystem: A possible common etiology for specific reading disability and attention deficit disorder Unpublished doctoral dissertation. University of New Orleans, New Orleans, LA. Williams, M. C., Littell, R. R., Reinoso, C., & Greve, K. W. (1994). The effect of wavelength on the performance of attention-disordered and normal children on the Wisconsin Card Sorting Test. Neuropsychology, 8,