Escapable Conclusions: Toomela (2003) and the Universality of Trait Structure

Transcription

1 Journal of Personality and Social Psychology In the public domain 2004, Vol. 87, No. 2, DOI: / Escapable Conclusions: Toomela (2003) and the Universality of Trait Structure Jüri Allik University of Tartu and Estonian Centre of Behavioral and Health Sciences Robert R. McCrae National Institute on Aging A. Toomela (2003) concluded that personality structure may be shaped by a cultural mechanism, word meaning structure, and that the apparent universality of the Five-Factor Model (FFM) was the result of inadequate sampling of concrete thinkers. A reanalysis using targeted rotation showed that the intended structure was generally replicated; small deviations in the most concrete thinkers were probably due to measurement error. The reanalysis authors found no evidence of structural differences in a second sample of adults classified as concrete or abstract thinkers. The FFM structure has been replicated in many samples containing individuals with cognitive or educational limitations; observer ratings may be preferred if these limitations are severe. To date, there is no compelling evidence that culture affects personality structure. In a recent study, Aaro Toomela (2003) examined relations between personality trait structure and the Vygotskian structure of word meaning. 1 On the basis of an analysis of data from 870 respondents (chiefly men from the Estonian military), Toomela claimed that individuals who primarily used everyday concepts in their thinking or who had relatively low levels of cognitive ability did not possess a coherent Big Five personality structure, whereas individuals who primarily used hierarchically organized scientific concepts in their thinking or who had high levels of cognitive ability did. Because most research on the cross-cultural generalizability of the Five-Factor Model (FFM) has been conducted on well-educated samples who presumably use scientific concepts, Toomela argued that the data point to an inescapable conclusion: The five-factor structure of Neuroticism, Extraversion, Openness, Agreeableness, and Conscientiousness is not a human universal (p. 733). Although Toomela raises interesting issues that have rarely been explored, we believe that his data, properly analyzed and conceptualized, do not cast doubt on the universality of the FFM. Our reinterpretation has implications for personality assessment and theory. Toomela correctly noted that there are profound ties between personality assessment and language. All human languages have terms to describe individual differences in personality, and most personality assessments rely on responses to verbal stimuli (but see Paunonen & Ashton, 2002). However, assuming that the respondent understands the language in which the inventory is administered, is there any reason to think that the way people construe word meanings will affect the structure of personality? Jüri Allik, Department of Psychology, University of Tartu, Tartu, Estonia, and Estonian Centre of Behavioral and Health Sciences, Tartu, Estonia; Robert R. McCrae, Laboratory of Personality and Cognition, National Institute on Aging, National Institutes of Health, Department of Health and Human Services, Baltimore, Maryland. Robert R. McCrae receives royalties from the Revised NEO Personality Inventory. Correspondence concerning this article should be addressed to Robert R. McCrae, Gerontology Research Center, Box 3, 5600 Nathan Shock Drive, Baltimore, MD jeffm@lpc.grc.nia.nih.gov Toomela argued that individuals who use everyday concepts would be poorer than those who use scientific concepts in both differentiation and integration of trait information. These opposing deficiencies would have no net effect on the number of factors; the content of factors, however, would be substantially different (p. 725). Toomela did not offer any hypotheses about what new factors might emerge among people for whom word meaning is tied to concrete sensory experiences, nor did he offer any post hoc speculations about why the observed factors emerged. Rather than show how meaningfully different factors occurred in this group, Toomela described the results as relatively chaotic. He apparently interpreted this as evidence of lower factor coherence among individuals whose constructs are not hierarchically organized. To us, incoherence suggests error of measurement rather than substantive differences. A Reanalysis To test his hypotheses, Toomela stratified his sample on the basis of measures of word meaning structure and cognitive ability, which were substantially correlated. The word meaning measure assessed the degree to which definitions were hierarchical, that is, referred to higher level, abstract concepts, and the subgroups ranged from least (H1) to most (H5) hierarchical (or scientific). Five subgroups (CA1 to CA5) were also formed on the basis of a cognitive ability measure that included tests of verbal, mathematical, and spatial abilities. Subgroups had between 152 and 217 participants. Toomela extracted five factors from the intercorrelations of the facet scales of the Estonian Revised NEO Personality Inventory (NEO PI R; Costa & McCrae, 1992; Kallasmaa, Allik, Realo, & McCrae, 2000) in each subgroup and reported the results of a varimax rotation. It is well known that the NEO PI R does not show simple structure (Church & Burke, 1994); consequently, the position of 1 Toomela also analyzed mean level differences in relation to word meaning structure and cognitive ability. We regard this aspect of his article as a valuable contribution to that literature (cf. Austin et al., 2002) but do not discuss it further. 261

2 262 ALLIK AND MCCRAE the factors under varimax rotation is somewhat arbitrary. This is especially likely to be a problem in small samples, in which random perturbations in the data can distort the structure. McCrae, Zonderman, Costa, Bond, and Paunonen (1996) therefore argued that exploratory analyses should be followed by confirmatory analyses, using targeted rotation 2 evaluated by congruence coefficients. We followed that strategy in reanalyzing Toomela s data. This method provides a quantitative evaluation of replicability that can be statistically assessed. These reanalyses should be viewed in the larger context of the enterprise of searching for cross-cultural replicability. Our approach has been to use the same inventory in each culture, whereas others have used indigenous inventories (Katigbak, Church, Guanzon-Lapeña, Carlota, & del Pilar, 2002) or lexical analyses (Saucier, Hampson, & Goldberg, 2000). Replications across instruments or methods provide powerful evidence of universality, but failures to replicate are ambiguous. For example, the failure to find a lexical Openness factor in a culture might have more to do with limitations in the personality lexicon than with trait structure in the members of that culture (McCrae, 1990). However, when the same instrument is compared across cultures, failures to replicate are strong evidence of cultural differences; our design optimizes the falsifiability of claims of universality. Finding a replicable structure when an instrument like the NEO PI R is used does not, however, mean that it is the only universal structure. Indeed, if personality structure itself is truly universal, then any personality instrument with a stable structure in one culture should have a replicable structure across cultures (e.g., Barrett & Eysenck, 1984). Further, any rotation of the factor structure should also be replicable; in fact, when two matrices are compared, the total congruence coefficient is identical regardless of which is taken as the target (McCrae et al., 1996). Some efforts have been made to determine whether variations in the varimax position of the axes are culturally meaningful (Kallasmaa et al., 2000), but this issue is currently unresolved. In the present analyses, as in previous analyses, the American normative structure is used as the target matrix because this structure is best understood and because it corresponds to the lexical findings in many cultures. The varimax-rotated factor columns of Table 1 reproduce the varimax solution of Toomela s principal-components analysis for the least hierarchical word meaning structure subgroup, H1. At first glance it is very difficult to recognize the usual NEO PI R structure in these data. Calculation of congruence coefficients with the American normative sample factors suggests that the best factor matching pairs Factor 1 (reflected) with Neuroticism (congruence coefficient.87), Factor 2 with Conscientiousness (.71), Factor 3 (reflected) with Agreeableness (.49), Factor 4 with Extraversion (.71), and Factor 5 with Openness to Experience (.52). Because congruence coefficients of at least.85 are necessary to infer replication (Haven & ten Berge, 1977), Toomela is surely correct that this does not replicate the intended structure. However, a varimax rotation is only one of an infinite number of mathematically equivalent descriptions of the factor structure. An alternative is provided by orthogonal Procrustes rotation, targeted to the American structure. The Procrustes-rotated factor columns of Table 1 present the results of this rerotation. The observed factor congruences varied from.75 (Agreeableness) to.93 (Extraversion), with the total congruence coefficient of All of these values are far above chance (McCrae et al., 1996), although only Neuroticism, Extraversion, and Conscientiousness are clearly replicated by Haven and ten Berge s (1977) rule-of-thumb. Because the total variance of the factors is unchanged by rotation, it remains the case that the total variance in this subsample is lower than in other subsamples, suggesting a less clearly marked structure. Table 2 summarizes similar reanalyses for all word meaning and cognitive ability subgroups. There is no question that the overall match of the H1 subgroup to the normative American structure is far from perfect and worse than in other groups. Nevertheless, the usual five-factor structure is clearly recognizable even in this extreme everyday concept thinkers subgroup. The lowest cognitive ability subgroup also shows a marginal replication for Agreeableness, but in every other subgroup, every factor is clearly replicated. Appropriate factor rotation substantially eliminates the apparent chaos in factor loadings. Distinguishing Traits From Methods Two of the factors in the H1 group failed to meet the usual criterion for replication. Agreeableness, in particular, with its unexpected loadings from high Openness to Values and low Openness to Ideas, is arguably different from the standard Agreeableness factor. Can we conclude from this that word meaning structure affects personality? No. In fact, even if this finding were replicated in other samples, it would not necessarily imply anything about the structure of personality traits. Instead, it would provide direct evidence only on the structure of self-reports of traits. It is reasonable to hypothesize that individuals who score low on measures of hierarchical word meaning and intelligence lack the cognitive abilities and verbal skills that are necessary for the observation and accurate description of their own personality traits. We know that very young children do not have sufficiently sophisticated mental and verbal tools for the analysis and description of their own personality, but they can be judged by well-informed observers (for example, their parents), whose ratings yield the familiar FFM (Mervielde & De Fruyt, 1999). To be a poor judge of one s own personality does not necessarily mean that one s personality is itself chaotic. In fact, the problem may be as simple as difficulties in understanding the questions on the personality inventory. It has been demonstrated, for instance, that negatively worded items are particularly difficult for young children (Marsh, 1986). The small variations from the standard FFM seen in the H1 group might be due to such methodological problems, not to a substantive difference in the universal structure of personality. It is not surprising that a high level of intelligence also helps in the observation, analysis, and reporting of one s own personality traits. A summary of the relevant literature concluded that cognitive abilities are the main factor determining the ability to understand other people s mental states: Good judges of other persons are generally more intelligent and tend to look at the world in cognitively complex and sophisticated ways (Davis & Kraus, 1997). The clear FFM structures seen in the higher cognitive 2 The use of Procrustes rotation was once viewed with skepticism (Horn, 1967), but orthogonal Procrustes rotation has become widely accepted (e.g., Chan, Ho, Leung, Chan, & Yung, 1999; Terracciano, 2003). 3 The observed factor congruences were only slightly different when data were targeted to the Estonian normative data (Kallasmaa et al., 2000). Factor congruence coefficients ranged from.75 (Agreeableness) to.92 (Extraversion) with the total congruence coefficient of.87.

3 TRAIT STRUCTURE UNIVERSALITY 263 Table 1 Factor Loadings of Revised NEO Personality Inventory Facet Scales in the Least Hierarchical Word Structure Subgroup After Varimax and Procrustes Rotations Varimax-rotated factor Procrustes-rotated factor a Facet F1 F2 F3 F4 F5 N E O A C N1: Anxiety N2: Angry Hostility N3: Depression N4: Self-Consciousness N5: Impulsiveness N6: Vulnerability E1: Warmth E2: Gregariousness E3: Assertiveness E4: Activity E5: Excitement Seeking E6: Positive Emotions O1: Fantasy O2: Aesthetics O3: Feelings O4: Actions O5: Ideas O6: Values A1: Trust A2: Straightforwardness A3: Altruism A4: Compliance A5: Modesty A6: Tender-Mindedness C1: Competence C2: Order C3: Dutifulness C4: Achievement Striving C5: Self-Discipline C6: Deliberation Factor congruence coefficient 89 b 93 b 81 b 75 b 88 b Note. N 158. Varimax loadings from Relationships Between Personality Structure, Structure of Word Meaning, and Cognitive Ability: A Study of Cultural Mechanisms of Personality, by A. Toomela, 2003, Journal of Personality and Social Psychology, 85, pp Copyright 2003 by the American Psychological Association. Reprinted by permission of the author. Decimal points omitted; loadings greater than or equal to.40 in absolute magnitude are given in boldface. F factor; N Neuroticism; E Extraversion; O Openness to Experience; A Agreeableness; C Conscientiousness. a Rotated toward American normative structure (Costa & McCrae, 1992). b Congruence higher than that of 99% of rotations from random data. ability subgroups presumably reflect the quality of the data collected from these respondents. An Attempted Replication Toomela s basic idea that personality structure may vary as a function of one s way of conceptualizing the world is an intriguing notion that has rarely been tested. It would be useful to test it with other operationalizations of thinking style, ideally with measures that are unrelated to cognitive ability. One such measure is the Behavior Identification Form (BIF; Vallacher & Wegner, 1989), which asks respondents to select one of two descriptions for a behavior. As in Toomela s word meaning measure, responses can be classified as concrete or abstract. For example, voting may be described abstractly as influencing the election or concretely as marking a ballot. It seems likely that people who use everyday word meanings would also be more concrete in their action identification. However, BIF scores are unrelated to IQ (Vallacher & Wegner, 1989). The BIF was administered to participants in the Baltimore Longitudinal Study of Aging who had previously completed the NEO PI R (see McCrae, 1993, for details). Only 242 participants had complete data on both instruments, so division into quintiles was not feasible. Instead, we dichotomized the sample at the median. The full range of BIF scores was represented, although a majority of respondents gave predominantly abstract responses. In

4 264 ALLIK AND MCCRAE Table 2 Congruences for Different Word Meaning Structure and Cognitive Ability Subgroups Group Factor N E O A C the high-scoring, abstract-thinking subgroup, factor congruence coefficients with the normative sample ranged from.93 to.98 after varimax rotation and from.94 to.98 after targeted rotation. In the low-scoring, concrete-thinking subgroup, factor congruence coefficients ranged from.82 to.97 after varimax rotation. As in some other samples (see Kallasmaa et al., 2000), the Extraversion and Agreeableness factors were tilted toward intermediate Love and Submission axes. However, after targeted rotation, factor congruence coefficients ranged from.94 to.98. Although different results might have been seen in a group with more extreme concretethinking scores, the present analysis offers no support for the hypothesis that the structure of personality is influenced by the concreteness or abstractness of the respondent s thinking style. 4 Implications for Assessment Total Hierarchical word meaning structure H H H H H Cognitive ability CA CA CA CA CA Note. H1 least hierarchical subgroup; H5 most hierarchical subgroup. CA1 lowest cognitive ability results subgroup; CA5 highest cognitive ability results subgroup. N Neuroticism; E Extraversion; O Openness to Experience; A Agreeableness; C Conscientiousness. Toomela concluded that it should be evident that the Big Five structure of personality is not universal to every culture and to every healthy man (p. 732). It is of course true that the FFM structure has not yet been found in every culture in the world but it has been substantially replicated in every culture in which questionnaire measures of the FFM have been examined (Paunonen & Ashton, 1998; Rolland, 2002). Many of the replication studies have used student samples, but others have used more representative samples (e.g., Lima, 2002, where almost half the sample required oral administration), and we have access to additional unpublished studies that speak to this issue. Trobst et al. (2000) administered the NEO PI R to a sample at high risk for HIV infection. Only half the sample had the sixth-grade reading level required for administration of the test, yet in that half, the FFM structure was clearly recovered. Weiss et al. (2003) replicated the NEO PI R structure in a sample of elderly, functionally impaired Medicare beneficiaries, nearly 40% of whom were not high school graduates. 5 De Fruyt, Mervielde, Hoekstra, and Rolland (2000) reported a fairly good replication in a sample of Dutch adolescents ages 12 and 13. In all these cases, the familiar FFM structure was found despite limitations in cognitive ability or education, and it seems reasonable to continue to investigate self-reported personality traits in a wide range of samples. However, there are certainly some situations in which selfreport personality data may not be trustworthy, and degraded factor structures could be an indication of methodological problems. In dealing with young children, or cognitively impaired adults, 6 or members of preliterate cultures, it may be appropriate to use observer ratings (which also replicate the FFM; e.g., Tupes & Christal, 1961/1992) instead of self-reports. Ideally, all research would include both self-reports and peer ratings, which would facilitate the separation of substantive differences from error of measurement. Implications for Personality Theory The appeal of Toomela s (2003) article for many readers was probably his claim that the structure of personality was modified by a cultural mechanism (i.e., structure of word meaning) and that earlier studies, which seem to support the idea that Big Five personality structure is a biologically determined human universal, suffer from serious sampling problems and insufficient data analyses (p. 723). That conclusion was meant to challenge the fivefactor theory view that personality traits are transcultural universals, rooted in biology (McCrae & Costa, 1997, 1999). We have argued here that the deficiencies in sampling are not so severe as it might appear from his review of the literature and that it was Toomela s varimax analyses that were insufficient. After further targeted rotation, the hypothesized structure was found, only slightly degraded in the subgroup with the most extreme everyday concept meaning structure. Even if factor structures had remained chaotic after targeted rotation, it is not clear that cultural mechanisms would have been responsible. Meaning systems are certainly acquired in a culture, but whether one acquires an everyday or a scientific structure depends in considerable measure on one s level of intelligence. In Toomela s data, the correlation between the Structure of Word Meaning test and the Cognitive Ability Scale was.54. Cognitive abilities, in turn, are substantially heritable (Brody & Crowley, 1995). Thus, it is possible that the distortions to factor structure seen in Toomela s data were due to error of measurement introduced by the operation of endogenous cognitive factors that would have been manifested in any cultural environment. Five-factor theory does not claim that the cultural environment is irrelevant to personality functioning. On the contrary, it is an 4 Surprisingly, even in the very small group of 53 Baltimore Longitudinal Study of Aging participants with predominantly concrete-thinking scores, the five factors could be replicated, with factor congruence coefficients of from.85 to.94 after Procrustes rotation. 5 In the subsample of 254 patients with less than a high school degree, congruence coefficients for the five factors (compared with the normative factor structure) ranged from.86 to.96 after varimax rotation and from.92 to.98 after targeted rotation (personal communication, A. Weiss, November 4, 2003). 6 In a study of traumatic brain injury patients, Trobst replicated the NEO PI R structure in both self-reports (N 187, targeted factor congruences.96 to.98) and observer ratings (N 163, targeted factor congruences.91 to.96; personal communication, K. K. Trobst, November 4, 2003).

5 TRAIT STRUCTURE UNIVERSALITY 265 essential context for the expression of all personality traits. Just as verbal fluency cannot exist without a learned language in which to be fluent, so Extraversion, Openness, and the other personality factors cannot be manifested except in culturally conditioned thoughts, feelings, and behaviors. Personality is expressed through culture, but so far, there is no good evidence that culture shapes the structure of personality. References Austin, E. J., Deary, I. J., Whiteman, M. C., Fowkes, F. G. R., Pedersen, N. L., Rabbitt, P., et al. (2002). Relationships between ability and personality: Does intelligence contribute positively to personal and social adjustment? Personality and Individual Differences, 32, Barrett, P., & Eysenck, S. B. G. (1984). The assessment of personality factors across 25 countries. Personality and Individual Differences, 5, Brody, N., & Crowley, M. J. (1995). Environmental (and genetic) influences on personality and intelligence. In D. H. Saklofske & M. Zeidner (Eds.), International handbook of personality and intelligence (pp ). New York: Plenum Press. Chan, W., Ho, R. M., Leung, K., Chan, D. S. K., & Yung, Y. F. (1999). An alternative method for evaluating congruence coefficients with Procrustes rotation: A bootstrap procedure. Psychological Methods, 4, Church, A. T., & Burke, P. J. (1994). Exploratory and confirmatory tests of the Big Five and Tellegen s three- and four-dimensional models. Journal of Personality and Social Psychology, 66, Costa, P. T., Jr., & McCrae, R. R. (1992). Revised NEO Personality Inventory (NEO PI R) and NEO Five-Factor Inventory (NEO-FFI) professional manual. Odessa, FL: Psychological Assessment Resources. Davis, M. H., & Kraus, L. A. (1997). Personality and empathic accuracy. In W. Ickes (Ed.), Empathic accuracy (pp ). New York: Guilford Press. De Fruyt, F., Mervielde, I., Hoekstra, H. A., & Rolland, J.-P. (2000). Assessing adolescents personality with the NEO PI R. Assessment, 7, Haven, S., & ten Berge, J. M. F. (1977). Tucker s coefficient of congruence as a measure of factorial invariance: An empirical study (Heymans Bulletin 290 EX). University of Groningen, Groningen, the Netherlands. Horn, J. L. (1967). On subjectivity in factor analysis. Educational and Psychological Measurement, 27, Kallasmaa, T., Allik, J., Realo, A., & McCrae, R. R. (2000). The Estonian version of the NEO PI R: An examination of universal and culturespecific aspects of the Five-Factor Model. European Journal of Personality, 14, Katigbak, M. S., Church, A. T., Guanzon-Lapeña, M. A., Carlota, A. J., & del Pilar, G. H. (2002). Are indigenous personality dimensions culture specific? Philippine inventories and the Five-Factor Model. Journal of Personality and Social Psychology, 82, Lima, M. P. (2002). Personality and culture: The Portuguese case. In R. R. McCrae & J. Allik (Eds.), The Five-Factor Model of personality across cultures (pp ). New York: Kluwer Academic/Plenum Publishing. Marsh, H. W. (1986). Negative item bias in ratings scales for preadolescent children: A cognitive-developmental phenomenon. Developmental Psychology, 22, McCrae, R. R. (1990). Traits and trait names: How well is openness represented in natural languages? European Journal of Personality, 4, McCrae, R. R. (1993). Moderated analyses of longitudinal personality stability. Journal of Personality and Social Psychology, 65, McCrae, R. R., & Costa, P. T., Jr. (1997). Personality trait structure as a human universal. American Psychologist, 52, McCrae, R. R., & Costa, P. T., Jr. (1999). A five-factor theory of personality. In L. A. Pervin & O. P. John (Eds.), Handbook of personality: Theory and research (2nd ed., pp ). New York: Guilford Press. McCrae, R. R., Zonderman, A. B., Costa, P. T., Jr., Bond, M. H., & Paunonen, S. V. (1996). Evaluating replicability of factors in the Revised NEO Personality Inventory: Confirmatory factor analysis versus Procrustes rotation. Journal of Personality and Social Psychology, 70, Mervielde, I., & De Fruyt, F. (1999). Construction of the Hierarchical Personality Inventory for Children. In I. Mervielde, I. Deary, F. De Fruyt, & F. Ostendorf (Eds.), Personality psychology in Europe: Proceedings of the Eighth European Conference on Personality Psychology (pp ). Tilburg, the Netherlands: Tilburg University Press. Paunonen, S. V., & Ashton, M. C. (1998). The structured assessment of personality across cultures. Journal of Cross-Cultural Psychology, 29, Paunonen, S. V., & Ashton, M. C. (2002). The nonverbal assessment of personality: The NPQ and the FF-NPQ. In B. De Raad & M. Perugini (Eds.), Big Five assessment (pp ). Göttingen, Germany: Hogrefe & Huber. Rolland, J.-P. (2002). Cross-cultural generalizability of the Five-Factor Model of personality. In R. R. McCrae & J. Allik. (Eds.), The Five- Factor Model of personality across cultures (pp. 7 28). New York: Kluwer Academic/Plenum Publishers. Saucier, G., Hampson, S. E., & Goldberg, L. R. (2000). Cross-language studies of lexical personality factors. In S. E. Hampson (Ed.), Advances in personality psychology (Vol. 1, pp. 1 36). Philadelphia: Taylor & Francis. Terracciano, A. (2003). The Italian version of the NEO PI R: Conceptual and empirical support for the use of targeted rotation. Personality and Individual Differences, 35, Toomela, A. (2003). Relationships between personality structure, structure of word meaning, and cognitive ability: A study of cultural mechanisms of personality. Journal of Personality and Social Psychology, 85, Trobst, K. K., Wiggins, J. S., Costa, P. T., Jr., Herbst, J. H., McCrae, R. R., & Masters, H. L., III. (2000). Personality psychology and problem behaviors: HIV risk and the Five-Factor Model. Journal of Personality, 68, Tupes, E. C., & Christal, R. E. (1992). Recurrent personality factors based on trait ratings. Journal of Personality, 60, (Original work published 1961) Vallacher, R. R., & Wegner, D. M. (1989). Levels of personal agency: Individual variations in action identification. Journal of Personality and Social Psychology, 57, Weiss, A., Costa, P. T., Jr., Karuza, J., Duberstein, P. R., Friedman, B., & McCrae, R. R. (2003). Personality stability and change in patients and caregivers aged 65 to 100. Unpublished manuscript, Gerontology Research Center. Received November 19, 2003 Revision received February 5, 2004 Accepted March 5, 2004