Trait Analyses of the Luria-Nebraska Processes, Motor Functions, and Memory Scales

Transcription

1 Anhiwr of Clinicd Vol. 7. pp l92 $ OO F rinted in the USA. All rights rurrvcd. Copyright National Academy of Nauopsychology Trait Analyses of the Luria-Nebraska Intellectual Processes, Motor Functions, and Memory Scales Stephen N. Macciocchi and Patrick C. Fowler University of Wrginia Health Sciences Center John D. Ranseen University of Kentucky Medical Center Trait analyses of the Luria-Nebraska s Intellectual Processes, Motor Functions, and Memory Scales were perjbrmed using multitrait-multimethod procedures and confirmatory factor analysis. Three measures each of intellectual, motor, and memory functions were administered to 161 neuropsychiatric patients. Intellectual scales (viz, Luria-Nebraska and WAIS-R) were found to have convincing discriminant properties, but motor and memory measures demonstrated visibly less such robustness. Confirmatory factor analyses supported a three-factor model of the 9 x 9 ma@& in which each of the three Luria-Nebraska scales serves as a marker for the factors. Issues related to the neuropsychological constructs, test construction, and construct validity of measurement procedures employed in clinical neuropsychology are discussed. Since publication, the Luria-Nebraska Neuropsychological Test Battery (LNNB) has been the focus of extensive criticism, particularly in regard to its latent structure and construct validity (Adams, 1980a.b; Crossen & Warren, 1982; Delis & Kaplan, 1982; Spiers. 1981). For example, Spiers (1981) argues that the LNNB s scale-names do not reflect the skills they purportedly measure, because most scales contain heterogeneous items that measure a variety of abilities other than the trait (or construct) implied by the names. The Requests for reprints should be mailed to Stephen N. Macciocchi, PhD, Department of Physical Medicine and Rehabilitation, Box 159, University of Virginia Health Sciences Center, Charlottesville, VA

2 542 S. N. Macciocchi et al. research to date which has investigated the trait properties of the LNNB s scales has principally focused on the factor structure of this battery. Moses (1983, 1984a,b), for example, has used varimax rotation to identify... internally consistent orthogonal factors for each LNNB clinical and summary scale... ; and these studies have demonstrated... hierarchical skill groupings predicted by Lurian theory and designers of the test (Moses & Maruish, 1988, p. 7). In this same review, Moses and Maruish argue that most of the studies which fail to support the LNNB s original factor structure (and presumably the trait properties of the battery s individual scales) are methodologically flawed. The authors conclude there is... replicated evidence in support of a stable factorial structure for the LNNB clinical and summary scales across large, heterogeneous samples and item pooling methods... (Moses & Maruish, 1988, p. 9). In spite of these conclusions, the studies that have examined the LNNB typically establish a factor structure for the LNNB s scales and, then, compare these empirically obtained factor models with the theoretical assumptions of Luria (1966). Although agreement between empirically derived factor models and theoretical predictions is important, there are actually very few studies which specifically address the construct validity of the LNNB s individual scales by comparing these with other neuropsychological tests which have known properties. The few studies that have focused on the relationship between specific scales and other neuropsychological tests of similar constructs have produced encouraging, but mixed, results (e.g., Fowler, Macciocchi, & Ranseen, 1986; Larrabee, Kane, Schuck, & Francis, 1985; Shelly & Goldstein, 1982). Most, if not all, of the studies reviewed by Moses and Maruish used factor analytic techniques as a basis for demonstrating the latent structure of the LNNB and its validity, that is, congruence with theoretical postulates. Although factor analytic methods are certainly acceptable for such endeavors, the construct validity of a psychological instrument should be determined in a variety of ways (Cronbach & Meehl, 1955). An alternative method for doing so is the multitrait-multimethod matrix, which enables investigation of convergent and discriminant validity among tests, namely, the adequacy of tests as a measure of a construct (Campbell & Fiske, 1959, p. 100; Cook & Campbell, 1979). In the multitrait-multimethod matrix (MT-MM), several traits (or constructs) are measured by different methods (or instruments), and the resulting correlations are then described in matrix form. When interpreting the matrix, Campbell and Fiske proposed several criteria for evaluating convergent and discriminant validity. For example, if a memory test correlates equally with other memory instruments, intellectual scales, and motor tests, then this test probably does not represent the construct of memory any better than the constructs of intellectual or motor ability. By interpreting method and trait corre-

3 Trait Analyses 543 lation patterns, a general picture of convergent and discriminant validity may well emerge. Using a MT-MM analysis with neuropsychological measures presents a somewhat unique problem in that a completely crossed method and trait matrix is impossible to construct. More recently, however, confirmatory factor analysis (CFA) has been used to analyze such multitrait matrices (Marsh & Hocevar, 1983). CFA can distinguish trait and method variance, provide goodness-of-fit tests, and yield estimates of each measure s uniqueness in the analysis (Marsh & Hocevar, 1983). In addition, CFA enables the investigator to specify, a priori, acceptable interpretive standards which are less subjective than the inspection method proposed by Campbell and Fiske (1959). In order to examine the trait properties of the Luria-Nebraska s Intellectual Processes, Motor Functions, and Memory Scales, we administered several measures of intellectual ability, motor skills, and memory functioning. A correlation matrix was computed and analyzed using Campbell and Fiske s decision rules, as well as submitted to confirmatory factor analysis. We hypothesized that the pattern of correlations in our matrix and the factor analyses would establish trait uniqueness for each of the LNNBs three scales. Subjects METHOD Consecutive referrals to the neuropsychology laboratories at the University of Virginia Health Sciences Center (UVHSC) and Western State Hospital (WSH) in Central Virginia (a training center for the Department of Behavioral Medicine and Psychiatry) were considered as subjects. Only patients who had low education (~6 years) or those unable to adequately complete the assessment were excluded from the study. The final sample included 161 patients who represented the full range of neuropsychiatric referrals for both institutions. UVHSC patients predominant diagnosis was neurological disorder while WSH patients evidenced histories of primary psychiatric illness with a lower frequency of neurological disorder. Patients average age and years of education, sex, handedness, and primary diagnoses are presented in Table 1. The composition of the sample is consistent with the wide range of patients typically referred for neuropsychological assessment in psychiatric settings. Procedures Patients referred for assessment were given neuropsychological evaluations by experienced neuropsychology technicians and postdoctoral fellows in clinical neuropsychology. The length of these examinations varied depending upon the patient s diagnosis, clinical status, and the referral question itself. In all

4 544 S. N. Macciocchi et al. TABLE 1 Sample Characteristics Mean Age 32.1 Education 11.3 SD Sex Handedness Diagnoses: Cerebral Trauma Mild Moderate Severe Seizure Disorder Complex Partial Generalized Psychiatric Disorder Affective Personality Schizophrenic Alcohol Dependency Degenerative Disorder Multiple Sclerosis Alzheimer s Parkinson s Hydrocephalus Cerebrovascular Disorder Right Stroke Left Stroke Systemic Disorder Lupus Mixed Neurological Disorder Diagnosis Not Available Men 68% Right 86% Women 32% Left 14% Median Duration N/A Note. Age, education, and duration of illness given in years. cases, patients were administered the Wechsler Adult Intelligence Scale- Revised (Wechsler, 1981), the Luria-Nebraska s Intellectual Processes, Motor Functions, and Memory Scales (Golden, Hammeke, & Purisch, 1980), the Finger Tapping Test (Reitan, 1958), the Grooved Pegboard Test (Klove, 1963), the Memory-for-Designs portion of the Wechsler Memory Scale

5 Trait Analyses 545 TABLE 2 Neuropsychological Test Scores Raw Scores Test Mean SD WAIS Verbal IQ WAIS Performance IQ Lmia Intellectual Processes Finger Tapping Grooved Pegboard Luria Motor Functions Wechsler Memory Scale Selective Reminding Luria Memory Note. Scores for Finger Tapping and Grooved Pegboard are for the dominant hand. The score for the Wechsler Memory Scale is for the Delayed Visual Memory subtest. The index for the Selective Reminding Test is Continuous Long-term Retrieval. (Wechsler, 1945), and the Selective Reminding Test (Buschke, 1973). Means and standard deviations of the patients raw scores are presented in Table 2. Analyses Prior to analyses, raw scores were corrected for age and education. A correlation matrix of these adjusted variables is displayed in Table 3. The interpretative rules proposed by Campbell and Fiske (1959) were applied to the validity diagonals, heteromethod-monotrait and heteromethod-heterotrait triangles. Prior to computing this composite correlation matrix, separate matrices for the patients from the UVHSC and those from WSH were computed and an index of their similarity was calculated (Gorsuch, 1974). In spite of the dissimilar composition of each group (e.g., neurological versus psychiatric patients), the index of pattern similarity is.95 for the two matrices, suggesting that these matrices are similar enough in terms of their overall pattern to be combined into one composite matrix. The application of confirmatory factor analysis to our correlation matrix most closely follows the example discussed by Kenny (1976) in which there are clearly defined trait factors, while the method factors are represented by distinct, individually-administered neuropsychological tests rather than by separate procedures (e.g., self-ratings vs. test results). In our matrix, then, there are discretely conceived trait factors (intellectual, motor, and memory), while underlying (i.e., latent) variables were the focus of the analysis. The 9 x 9 correlation matrix which is displayed in Table 3 was submitted to LISFUSL VI (Joreskog & Sorbom, 1983). Analyses were performed on the correlation matrix rather than the covariance matrix because the variances of the

6 546 S, h? Macciocchi et al. TABLE 3 Multitrait Matrix Variables VIQ PIQ LIP FT GP Lh#F VSM VM Verbal IQ Performance IQ Lnria Intekxtual Processes -z -74 Finger Tapping -g Grooved Pegboard Lnria h5otor Functions Viiuospatial Memory Verbal Memory Lnria Memory con~ibuting variables were subs~ti~ly disparate (Cudek, 1989). The correlation matrix was analyzed using both maximum likelihood and generalized least squares solutions because of the concerns which have been voiced about the indiscriminant use of maximum likelihood factor analyses (e.g., Boomsma, 1980; Jackson & Chan. 1980). These parallel analyses did not reveal any significant differences in the factor pattern matrices for the two solutions so the maximum likelihood solution was retained for discussion due to its advantages [e.g., asymptotic efficiency of the solution, scale invariance, a goodness-of-fit test for the number of factors, as well as the avoidance of limiting assumptions in parameter distributions (Anderson & Gerbing, 1984)]. In our study, because there is a very favorable ratio between the number of subjects and the number of parameters being estimated, as well as between the number of variables and the number of factors, CFA, which uses designated theoretical values in the factor loading matrix and constrains the correlations between underlying factors, there is the singular benefit of making the model identifiable (Kenny, 1979; Long, 1983; O Crady, 1983).1 In the present instance, the target matrix (see Table 4), which consisted of Is and OS, was constructed so that each of the intellectual, memory, and motor variables was estimated for only one factor. RESULTS As expected, validity diagonal entries (heteromethod-monotrait) were significantly different from zero and large enough to encourage further examina- 1 Anderson and Gerbing (1984) have provided evidence that the X2 goodness-of-fit test is well-behaved in even small samples if the models are not too complex, i.e., the estimates of their parameters can be derived without unusual limiting assumptions. Bentler (1985) has also recommended that a minimum ratio of 5:l hold between the number of subjects and the number of parameters being estimated. In our present study, this ratio is 13.42: 1.

7 Trair Analyses 547 TABLE 4 Target Factor Loading M&lx Variables Intellieence Motor Memorv Verbal IQ Performance IQ Lulia Intellecalal Ekcesses Finger Topic Grooved Pegboard Luria Motor Functions Viiuospatial Memory Verbal Memory LurkMemory tion of the data. The strongest convergent relationship (heteromethod-monotrait) was observed for intellectual measures (viz., VIQ. PIQ, LIP). Intellectual measures also demonstrated the greatest discriminant validity. Validity diagonal values were also consistently greater than values in heteromethod-heterotrait triangles for all intellectual measures. While the Grooved Pegboard s results also displayed a consistent pattern of disc~~n~t validity, the other measures showed inconsistent signs of such validity. In general, though, even these variables displayed patterns consistent with some trait discrimination. On balance, validity diagonals exceeded correlations among monomethodmultitrait variables except for the Luria-Nebraska s scales intercorrelations. The maximum likelihood solution of our 9 x 9 correlation matrix is displayed in Table 5. LISREL s goodness-of-fit index is 38 for this model, X2(24) = 88.04, p <,011, a result that reveals a striking degree of fit for this model to the correlation matrix, even though the model does not fit in absolute (i.e., mathematical) terms.* The factor loading pattern, as displayed in Table 5, shows clear definition for each of the three factors: There are substantial loadings for those variables contributing to the intelligence and memory factors, while two variables con~buting to the motor skills dimension demonstrate somewhat lower loadings (and correspondingly higher unique variances); the factors are substantially correlated with each other (see the note for Table 5). In sum, the conceptual framework for understanding relationships among these variables is nicely realized in the factor pattern of this model as well as, generally, in the level of the loadings and the pattern of intercorrela- 2 I he three-factor model is the best for this data matrix in that other, viz., iterated versions of the model did not improve its fit; LISRJZL s modification indices were used to inform analysis of each of the iterated alternatives. In considering yet other possible alternatives to our three-factor model, both a general (single-factor) model and a twofactor m&i were considered, but under the usual constraints of factor analysis, each of these yields a much poorer fit to the present matrix.

8 548 S. N. Macciocchi et al. TABLE 5 Oblique Factor Pattern Matrix (Standard Errors in Parentheses) Factors Variables Intelligence Motor Memory Unique Variances Verbal IQ Performance IQ Luria Intellectual Processes Finger Tapping Grooved Pegboard Luria Motor Functions Visuospatial Memory Verbal Memory Luria Memory 89 (06) 90 (06) -87 (07) 58 (08) -57 (08) -97 (08) 82 (07) 71 (07) -92 (06) Note. Factor pattern correlations: rt2 =.81, rt3 =.92, 23 =.78. Decimals omitted. 21 (03) 19 (03) 25 (04) 66 (08) 68 (08) 07 (06) 33 (05) 50 (06) 16 (04) tions between the factors. This type of analysis very clearly complements the traditional analyses of such matrices and, given the explicitness with which the model is formulated, the limitations of the criteria proposed by Campbell and Fiske (1959) are avoided to a great degree. DISCUSSION Both the MT-MM analysis and CFA support the inference of construct validity for the Luria-Nebraska s Intellectual Processes, Motor Functions, and Memory Scales. Both analyses suggest that the intellectual measures, including the LIP Scale, demonstrate convergent and discriminant validity, although the results for motor and memory measures are somewhat less consistent. Even though these data analyses support the inference that LNNB Scales have unique trait characteristics, several aspects of the data require discussion. First, the three LNNB Scales are all highly correlated with one another. Second, all of the LNNB Scales correlate somewhat higher with intellectual measures than with tests representing similar constructs, but this is also true of other tests (e.g., WMS, SRT). Finally, in spite of establishing a plausible threefactor model for these data, all factors (viz., intellect, motor, and memory) are highly correlated with each other, a result that suggests that the notion of independence between traits like intelligence and memory must be questioned. While some patients with more circumscribed lesions and minimal diaschesis [e.g., mesial-temporal or thalamic location (Corkin, 1984; Squire, 1987, p. 178)] may demonstrate dissociable performance on intellectual and memory tests, most patients referred for neuropsychological assessment in neuropsychiatric settings display highly correlated levels of intellectual and memory abilities (Matarazzo, 1990; Shelly & Goldstein, 1982).

9 Trait Analyses 549 A different selection of tests may have possibly provided a clearer pattern of discrimination between these traits, but what seems evident from this research, as well as other investigations (e.g., Shelly & Goldstein, 1982), is that most neuropsychological measures are substantially intercorrelated (Chelune, 1982). Therefore, the criticism that all items on individual LNNB scales must somehow represent a specific, unitary construct is not persuasive. On the other hand, Spiers and others criticisms have merit in the sense that performance on the LNNB s Memory and Motor Functions Scales depends heavily on general cognitive ability, probably more so than is the case with other memory (e.g., Selective Reminding Test) and, especially, motor tests (e.g., Finger-Tapping Test). However, few if any motor or memory tests are completely independent with respect to intellectual capacity. This finding is also supported by an examination of items, comprising the LNNB scales, which reveals a cognitive-linguistic component to many items, including those on the Motor Functions Scale. Consistent with theory (Luria, 1966), these scales measure multifaceted aspects of neuropsychological constructs. When considering motor functions, one can argue that the somatosensory, proprioceptive, and regulatory items incorporated in the Motor Functions Scale dilute, or confuse, assessments of motor activity. Nevertheless, in many cases, the clinician is most interested in establishing the basis for motor dysfunction rather than simply assessing motor power or speed which is evaluated by more traditionally respected assessment techniques (e.g., strength of grip, finger-tapping). In any case, our data do support the respective construct validities of the LNNB s Intellectual Processes, Motor Functions, and Memory Scales. Since few construct validity studies using MT-MM matrices or CFA have been reported in the literature for the LNNB s scales, our study underlines the need for such investigations of the trait properties of neuropsychological tests presently in use, especially when certain properties are implied by designers of these tests, assumed by clinicians, but not consistently demonstrated by clinical research. REFERENCES Adams, K. M. (198Oa). In search of Lmia s battery: A false start. Journal of Consulting and Clinical Psychology, 48, Adams, K. M. (198Ob). An end of innocence for behavioral neurology? Adams replies. Journal of Consulting and Clinical Psychology, Anderson, J. D., & Gerbing, D. W. (1984). The effect of sampling error on convergence, improper solutions, and goodness-of-tit indices for maximum-likelihood confiitory factor analysis. Psychometrika, Bentler, P. M. (1985). Theory and implementation of E&S: A structural equations program Los Angeles: BMDP Statistical Software. Boomsma, A. (1980). Nonconvergence, improper solutions, and starting values in LISRIX maximum-likelihood estimation. Psychometrica, 2,

10 550 S. N. Macciocchi et al. Buschke, H. (1973). Selective reminding for analysis of memory and learning. Journal of Verbal Learning and Verbal Behavior, l& Campbell, D. T., & Fiske, D. W. (1959). Convergent and discriminant validation by the multitraitmultimethod matrix. Psychological Bulletin Chelune, G. J. (1982). A reexamination of the relationship between the Luria-Nebraska and Halstead-Reitan Batteries: Overlap with the WAIS. Journal of Consulfing and Clinical Psychology, Cook, T. D., & Campbell, D. T. (1979). Quasi-experimentation: Design and analysis forfield settings. Chicago: Rand McNally. Corkin. S. (1984). Lasting consequence of bilateral medical temporal lobectomy, clinical course, and experimental findings. Seminars in Neurology, Cronbach, L. J., & Meehl, P. E. (1955). Construct validity in psychological tests. Psychological Bulletin, 52, Crosson, B., & Warren, R L. (1982). Use of the Luria-Nebraska neuropsychological battery in aphasia: A conceptual critique. Journal of Consulting and Clinical Psychology, Cudek, R. (1989). Analysis of correlation matrices using covariance structure models. Psychological Bulletin, Delis, D., & Kaplan, E. F. (1982). The assessment of aphasia with the Luria-Nebraska neuropsychological battery: A case critique. Journal of Consulting and Clinical Psychology, Fowler, P. C., Macciocchi, S. N., & Ranseen, J. D. (1986). WAIS-R factors and performance on the Luria-Nebraska s Intelligence, Memory, and Motor Scales: A canonical model of relationships. Journal of Clinical Psychology, 42, Golden, C. J., Hammeke, T. A., & Purisch, A. D. (1980). The Luria-Nebraska Neuropsychological Battery. Los Angeles: Western Psychological Services. Gorsuch, R. (1974). Factor analysis. Philadelphia: Saunders. Jackson, D. N., & Ghan. D. W. (1980). Maximum-likelihood estimation in common factor analysis: A cautionary note. Psychological Bulletin, 2, Joreskog, K. G.. & Sorbom, D. (1983). LISREL: Analysis of linear structural relationships by the method of maximum likelihood-versions V & VI (2nd ed.). Chicago: 1E.S. Kenny, D. A. (1976). An empirical application of confirmatory factor analysis to the multitraitmultimethod matrix. Journal of Expermental Social Psychology, Kenny, D. A. (1979). Correlation and causality. New York: Wiley-Interscience. Klove, H. (1963). Clinical neuropsychology. In E N. Forster (Ed.), The medical clinics of North America. New York: Saunders. Larrabee, G. J.. Kane, R. L., Schuck, J. R., & Francis, D. J. (1985). Construct validity of various memory testing procedures. Journal of Clinical and Experimental Neuropsychology, Long, J. S. (1983). Confirmatory factor analysis: A preface to LISREL. Beverly Hills, CA: Sage. Luria, A. R. (1966). Higher corticalfunctions in man. New York: Basic Books. Marsh, H. W., & Hocevar, D. (1983). Confirmatory factor analysis of multitrait-multimetbod matrices. Journal of Educational Measurement, Matarazzo. J. D. (1990). Psychological assessment versus psychological testing: Validation from Binet to the school, clinic, and courtroom. American Psychologist, 45,9, Moses, J. A., Jr. (1983). An orthogonal factor solution of the Luria-Nebraska neuropsychological battery items: I. Motor, rhythm, tactual, and visual scales. Clinical Neuropsychology, V, Moses, J. A., Jr. (1984a). An orthogonal factor solution of the Lima-Nebraska neuropsychological battery items: II. Receptive speech, expressive speech, writing, and reading scales. International Journal of Clinical Neuropsychology, VI, Moses, J. A., Jr. (1984b). An orthogonal factor solution of the Luria-Nebraska neuropsychological battery items: III. Arithmetic, memory, and intelligence scales. International Journal of Clinical Neuropsychology, VI, Moses, J. A., Jr., & Maruish, M. E. (1988). A critical review of the Luria-Nebraska neuropsychological battery literature: II. Construct validity. International Journal of Clinical Neuropsychology, X, 5-11.

11 Trait Analyses 551 O Grady, K. E. (1983). A confirmatory maximum-likelihood factor analysis of the WAIS-R. Journal of Consulting and Clinical Psychology, Reitan, R. M. (1958). Validity of the trail-making test as an indication of organic brain damage. Perceptual and Motor Skills, 8, Shelly. C.. & Goldstein, G. (1982). Psychometric relations between the Luria-Nebraska and Halstead-Reitan Nemopsychological Test Battery in a neuropsychiatric setting. Clinical Neumpsychology, N, Spiers, T. A. (1981). Have they come to praise Luria or to bury him? The Luria Nebraska Battery controversy. Journal of Consulting and Clinical Psychology, 49, Squire, L. R. (1987). Memory and brain. New York: Oxford University Press. Wechsler, D. (1945). A standardized memory scale for clinical use. Journal of Psychology, Wechsler, D. (1981). WAIS-R manual. New York Psychological Corporation.