This MS is the final prepublication (open access) version of the published article:

Transcription

1 Who took the out of expectancy-value theory? 1 This MS is the final prepublication (open access) version of the published article: Nagengast, B., Marsh, H. W., Scalas, L. F., Xu, M. K., Hau, K.-T., & Trautwein, U. (2011). Who took the out of expectancy-value theory? A psychological mystery, a substantive-methodological synergy, and a cross-national generalization. Psychological Science, 22(8), doi: / This article may not exactly replicate the final published version in the journal. It is not the copy of record and readers are encouraged to obtain the copy of record through their university or local library using the article s DOI (digital object identifier). Running Head: Who took the out of expectancy-value theory? Who Took The Out Of Expectancy-Value Theory? A Psychological Mystery, A Substantive-Methodological Synergy, And A Cross-National Generalization Benjamin Nagengast, University of Oxford, UK Herbert W. Marsh, University of Oxford, UK, and University of Western Sydney L. Francesca Scalas, University of Oxford, UK, and University of Cagliari, Italy Man Xu, University of Oxford, UK, and University of Cambridge, UK Kit-Tai Hau, The Chinese University of Hong Kong, Hong Kong Ulrich Trautwein, University of Tübingen, Germany 26 November 2010 Revision: 23 March 2011 Author note

2 Who took the out of expectancy-value theory? 2 This research was supported in part by a grant to the second author from the UK Economic and Social Research Council. Requests for further information about this investigation should be sent to Benjamin Nagengast, Department of Education, University of Oxford, 15 Norham Gardens, Oxford OX2 6PY UK; benjamin.nagengast@education.ox.ac.uk

3 Who took the out of expectancy-value theory? 3 Abstract Expectancy-value theory (EVT) is a dominant theory of human motivation. Historically, the expectancy-by-value (E V) interaction was central to EVT; motivation is high only if both expectancy and value are high. However, the E V interaction mysteriously disappeared more than 25 years ago. Apparently solving the mystery, we tested E V interactions using evolving latent-variable models with interactions, based on large representative samples of 15-year-olds (N = 398,750) from 57 diverse countries. The effects of expectancy (science self-concept), value (enjoyment), and the E V interaction were all statistically significant and positive for both engagement in science activities and intentions of pursuing scientific careers for the total sample and for nearly all of the 57 countries considered separately. This, apparently the strongest cross-national test of EVT ever undertaken, supports the generalizability of EVT predictions including the lost E V interaction. The culprit (weak statistical methodology) was identified, re-establishing the E V interaction to its rightful role in EVT.

4 Who took the out of expectancy-value theory? 4 We are about to embark on a psychological mystery a real-life academic thriller. In the not so ancient history of psychology, the dominant theory of human motivation was expectancy-value theory (EVT). EVT had a proud tradition based on strong theory and empirical support from both early animal research (e.g., Tolman, 1938; 1955) and subsequent human research (Atkinson, 1957; see review by Feather, 1959). The cornerstone of EVT was the critical role of the expectancy-by-value interaction (E V). As depicted in the right panel of Figure 1, EVT predicted a multiplicative pattern of relations between expectancy, value, and resulting motivation. To achieve a high level of motivation, both expectancy and value had to be high. If either expectancy or value were low, high values on the other dimension were of little or no consequence for motivation and behavior. The centrality of the E V interaction to EVT was evident into the early 1980s; Feather s (1982) comprehensive review concluded that EVT models assume that expectations and subjective values combine multiplicatively to determine force (p. 414). Then, apparently without warning, the in EVT mysteriously disappeared. Beginning in the mid-1980s, the E V term stopped being reported in tests of modern versions of EVT of achievement motivation and the focus of the theoretical models was shifted away from a multiplicative relation of expectancy and value to purely additive models (see the left panel of Figure 1 compared to the right panel). A striking example is the influential sociocultural EVT-model of achievement motivation in educational psychology (Eccles (Parsons), 1983; Wigfield & Eccles, 2000). While it clearly extends earlier EVT-models, e.g. by differentiating the value domain into four dimensions (interest-enjoyment value, attainment value, utility value and cost) and by identifying pre-cursors of expectancy and value dimensions, its core component is the joint prediction of achievement-related motivation and choices and ultimately performance by expectancy of success and value of the outcome.

5 Who took the out of expectancy-value theory? 5 However, the initial presentation (Eccles (Parsons), 1983), but also more recent reviews (e.g., Wigfield & Eccles, 2000; Wigfield, Tonks, & Eccles, 2004) of this model only allude to the positive correlation between expectancy and value. There is no explicit mention of an E V interaction that had been the main focus in Atkinson s (1957) original model. Does the omission of the multiplicative relation between expectancies and value matter? Figure 1 illustrates that an additive model (left panel) as tacitly implied by current EVT-models and a multiplicative relation implied by the original EVT-models (right panel) lead to fundamentally different predictions. In the additive model (left panel), the effects are compensatory; low scores in one domain (e.g., a small expectancy of success) can be offset by corresponding larger scores in the other (e.g., a high value attributed to the outcome). However, in the multiplicative model (right panel), the effects of expectancy and value are non-compensatory; both expectancy and value have to be high in order to instigate motivated behavior. This fundamental difference between the two models has important implications for the application of EVT in policy and practice. Mysteriously, the multiplicative relation between expectancy and value seems to have vanished from EVT of achievement motivation without a trace. There seems to have been no explicit theoretical argument for dropping it and hardly any reported empirical tests of it. How could such a prominent theoretical feature of EVT have been so central in 1982 (Feather, 1982), and then be simply ignored for the next quarter century and more? Surely someone must have noticed that it went missing! Surely there must have been a good reason for discarding the cornerstone of EVT! Who was the culprit? The Culprit: Weak statistical models for testing interaction effects Our main suspect the culprit who is responsible for the missing E V interaction is weak statistical models and methodology. Appropriate tests of multiplicative relations between variables require the test of interaction effects using a statistical model that includes

6 Who took the out of expectancy-value theory? 6 both the corresponding main or first-order effects of the predictors as well as the effect of their product (Arnold & Evans, 1979; Blanton & Jaccard, 2006; Busemeyer & Jones, 1983; Cohen, 1978). Especially in psychology with its emphasis on individual differences, many theoretical models like EVT explicitly posit interaction effects; examples include aptitude-treatment interactions (Cronbach & Snow, 1977), importance-weighted average models (Marsh, 1993, 2008), theoretical accounts of attitude behavior-relations (Ajzen, 1987), and models of person-environment fit. Even though they are a critical part of many psychological theories, interaction effects, particularly in observational studies, are typically small, non-significant, or not easily replicated and hence easily overlooked and forgotten. Part of the problem is that multiple regression, even if it uses a correctly specified model, falls short when predictor variables are measured with error: The measurement errors in the original variables combine multiplicatively in the product variable used for testing the interaction effect. This leads to an underestimation of the size of the interaction effect that is even stronger than the corresponding attenuation of the main effects (Busemeyer & Jones, 1983). 1 Alternative approaches based on structural equation models (SEMs) have been evolving during the last decade, but have often not been followed in applications (Marsh, Wen, Nagengast, & Hau, in press-b). The E V interaction disappeared partly as a consequence of the shift from a focus on behavioral choice in laboratory studies prevalent in the historical approaches to tests of EVT to the assessment of expectations of success and value with surveys and questionnaires in real-world contexts (see Busemeyer & Jones, 1983; Mitchell, 1974). In laboratory studies, EVT constructs were operationally defined and experimentally manipulated; this resulted in a small number of levels for each dimension that were free of measurement error and were independent of each other. Using within-person designs, the interaction hypothesis could be tested with analysis of variance. The analysis of surveys and

7 Who took the out of expectancy-value theory? 7 questionnaires shifted the focus from within-person differences in motivation to engage in different tasks to between-person (interindividual) comparisons of motivation to engage in identical subjects or tasks. Multiple regression and path analysis with scale scores (e.g., Eccles (Parsons), 1983) and linear SEMs (e.g., Meece, Wigfield, & Eccles, 1990) became the analytical techniques of choice. Although in principle appropriate for the analysis of interaction effects, both methods pose obstacles for detecting interaction effects: multiple regression with scale scores will underestimate interaction effects due to measurement error in the variables (Busemeyer & Jones, 1983); linear SEM approaches that control measurement error could traditionally only model additive relations between latent variables (see Marsh et al., in press-b). Consistent with this perspective, Eccles (9 March, 2011, personal communication) noted that she and colleagues sought evidence for E V interactions, but did not report them because they were consistently non-significant. So we replaced the symbol with a dash in order to shift the focus away from the multiplicative function. Note that we did not replace the with a + sign but we did stop testing for a significant interaction because we so rarely found it to be significant in our studies. So one could certainly infer from our regression models that we were focused on an additive model. Hence, although Eccles published work has emphasized the additive component of EVT that is the focus of modern EVT, her personal account provides a richer picture. In summary, our search seems to have narrowed down the list of likely suspects to weak statistical models as the main culprit. E V interactions in traditional multiple regression were plagued by measurement error. Linear SEMs could not be used to test interaction effects. Latent variable models with interactions became available in principle in the 1980s (e.g., Kenny & Judd, 1984). Despite the widespread use of SEMs and the importance of interaction

8 Who took the out of expectancy-value theory? 8 effects, practically no substantive researcher used them. The paucity of such applications is not due to a lack of relevant substantive applications that require tests of interaction terms. Rather, as noted by Rigdon, Schumacker and Wothke (1998), difficulties in specifying complicated constraints necessary for SEMs with latent interactions have led researchers to pursue other approaches. However, these models have recently become more easily accessible to applied researchers (Klein & Moosbrugger, 2000; Marsh, et al., in press-b; Marsh, Wen, & Hau, 2004). Here, we apply them to large nationally representative databases to test for E V interactions. The Present Investigation Despite wide acknowledgment that science skills are fundamental to socio-economic development in technology-based societies, the number of students pursuing careers in science is declining worldwide (OECD, 2007). Schools fail at their core academic business of fostering engagement and educational aspirations in this field. While schools focus primarily on academic achievement, the fundamental problem is one of motivation to engage in science activities and to pursue a science-related career path. The first step in reversing this trend is a better understanding of the key motivational determinants of career choices and their interplay, as well as the underlying motives to students growing disengagement in science subjects. What is needed to tackle this problem is a synergy between good substantive theory (EVT) and strong statistical methodology (latent-variable models of interaction effects). Understanding the interplay of expectancy and value in bringing about engagement and career motivation in science is critical for the development of strategies to address this shortage. We analyze data from the OECD Program for International Student Assessment (PISA) 2006 that focused on achievement and motivational outcomes in science. In line with the socio-cultural EVT-model of achievement motivation (Eccles (Parsons), 1983; Wigfield

9 Who took the out of expectancy-value theory? 9 & Eccles, 2000), we use academic self-concept in science (Marsh, 2006; representing expectancy of success, see Eccles & Wigfield, 1995) and enjoyment of science (representing intrinsic value) to predict extra-curricular engagement in science (a behavioral measure of motivation) and career aspirations in science (a proxy measure for choice) using a SEM with latent interactions. The international comparability and the use of nationally-representative samples in each of the participating countries make PISA the strongest available resource for testing the cross-national generalizability of EVT, including the apparently lost E V interaction. The large and diverse sample of 57 countries that participated in PISA allows demanding tests of the cross-national generalizability of the findings. Although the EVTmodel explicitly includes cultural background variables and hypothesizes that they influence academic motivation and choice (Wigfield & Eccles, 2000), most tests of the model have been conducted in Western cultures (Wigfield, et al., 2004). The few international comparisons to-date have addressed differences and similarities in the factor structure of expectancy constructs and compared the absolute levels of components of expectancy and value, showing e.g., that competence perceptions are higher in Western cultures, but providing only limited results for value components of EVT (see the review in Wigfield et al., 2004). However, these studies have not reported E V interactions and were typically based on convenience samples instead of nationally-representative data available in the PISA 2006 database. Methods Participants We used data collected as part of PISA The sample consisted of 15-year-old students (N = 398,750, 50.5% female) from 57 countries (OECD, 2007, 2009). A complex

10 Who took the out of expectancy-value theory? 10 two-stage sampling process was used to guarantee the representativeness of the samples for the national populations of 15-year-old students (for details see OECD, 2009). Measures The measures of the expectancy-value constructs and the motivational outcomes were selected from the student background questionnaire. All scales were based on positively worded items with a 4-point Likert answering scale ranging from Agree completely to Disagree completely, if not otherwise mentioned. All answers were recoded so that higher scores indicated a higher value on the underlying construct. All scales have been validated extensively and possess good measurement properties (OECD, 2009; also see the supplemental appendix). Complete descriptions of the scales, items and their properties are given in OECD (2009). Science self-concept was used to operationalize students expectancy of success. The scale consisted of six items assessing students perception of their competencies in science (e.g. <School science> topics are easy for me. ). Enjoyment of science was used to represent one of the subdomains of value (interestenjoyment value). The scale consisted of five items that assessed the enjoyment students experienced when engaging in science-related activities (e.g., I generally have fun when I am learning <broad science> topics. ). Science-related extra-curricular activities were used as a behavioural outcome. The scale consisted of six items assessing the frequency of participation in non-compulsory and after-school science activities, e.g., watching TV programs, reading magazines or attending science clubs. The labels of the answering categories ranged from never or hardly ever to very often. Career aspiration in science was used as a proxy measure for long-term academic choice. The scale consisted of four items that assessed the students intentions of studying

11 Who took the out of expectancy-value theory? 11 science after school and taking up a science-related career (e.g., I would like to work in a career involving <broad science> after <secondary school>. ). Data Analysis We tested the joint influence of science self-concept, science enjoyment and their latent interaction on science career aspirations and extracurricular activities (see the path diagram Figure 2) implementing the unconstrained approach (see the supplemental technical appendix) to latent interactions in Mplus 5.21 (Muthén & Muthén, ) using robust standard errors and test statistics to account for non-normality of the indicators. It is important to emphasize that appropriately specified latent interaction models include both main effect variables as well as the product term (Cohen, 1978, Cronbach, 1987; Marsh, et al., in press-a; see also footnote 1 and supplemental materials). All analyses used the standardized student weight to obtain unbiased estimates of population parameters and the complex design correction in Mplus to control standard errors for the nesting of students within schools (see supplemental materials, for more details on the model and its implementation including the standardization of variables, treatment of missing data and criteria for the assessment of model fit). Two sets of analyses were conducted: (1) In the total group analysis, the latent interaction model was fitted to the full international sample (see also the Mplus syntax in the supplemental material); (2) Using the 57 countries as separate groups, we then tested a sequence of increasingly restrictive multigroup models to test the invariance of the factor loadings for the first-order constructs across the 57 countries (Meredith, 1993). After that the multigroup SEM with latent interactions was fitted simultaneously in each of the 57 countries. In order to test whether the structural effects varied across the countries, the path coefficients in the structural model were fixed to equality across countries and the fit of this model was compared to the model in which these effects were freely estimated.

12 Who took the out of expectancy-value theory? 12 Results Total group analysis The SEM with science self-concept, science enjoyment, their latent interaction as predictors for career aspirations and extra-curricular activities fit the data in the total sample well (χ 2 (289) = , CFI = 0.975, TLI = 0.971, RMSEA = 0.021). The path coefficients of the completely standardized solution and correlations between the latent variables are shown in Figure 2. For both dependent measures, significant main effects of the latent predictor variables and their latent interaction emerged (see Table 1). A higher science self-concept led on average to more engagement in science-related activities and to higher career aspirations in science. Similarly, a higher enjoyment of science led on average to more engagement in science-related activities and to higher career aspirations in science. In line with previous research on the joint effects of expectancy and value (see Wigfield & Eccles, 2000), enjoyment proved to be a relatively more important predictor for engagement and choice; the path coefficients for enjoyment were larger than those for self-concept. Of central importance, the critical E V interaction was statistically significant for both the pursuit of science-related activities and science career aspirations. The effect of selfconcept varied with the level of enjoyment and the effect of enjoyment varied with the level of self-concept. The effect of the latent interaction variable on science-related behaviours was positive and highly significant (b = 0.073, s.e. = 0.003, p < 0.001) as was the effect on career aspirations in science (b = 0.053, s.e. = 0.002, p < 0.001). Simple slopes plots (Bauer & Curran, 2005; Preacher, Curran, & Bauer, 2006; see Figure 3) show the effect of science selfconcept on the respective outcome variables at the mean of enjoyment (the main effect) and at one and two SD below and above the mean. The non-linear, multiplicative relation is clearly evident from the graphs; the effect of self-concept on outcomes becomes more

13 Who took the out of expectancy-value theory? 13 positive when enjoyment is high and the effect of enjoyment becomes more positive when self-concept is high. This interaction is particularly evident for engagement in science-related activities such that at low levels of enjoyment, self-concept has no effect whereas at high levels of enjoyment, an increase of one SD in self-concept is associated with about.4sd change in science-related activities. In supplemental analyses (see supplemental materials) we tested the generalizability of the results over gender. The results showed that these main and interaction effects of self-concept and enjoyment were similar for boys and girls. Multicountry Analysis We now turn to the evaluation of the generalizability of EVT effects across the 57 countries represented in PISA First, we established the invariance of factor loadings (Meredith, 1993) across the 57 countries. This was accomplished by comparing a multigroup confirmatory factor analyses model in which all factor loadings of the indicators were fixed (factorial invariance) to a model in which the factor loadings could vary between countries (configural invariance). The unrestricted model showed a good fit to the data (χ 2 (10431) = , CFI = 0.967, TLI = 0.962, RMSEA = 0.035). Restricting the factor loadings to be equal across countries did not reduce the fit substantially (χ 2 (11383) = , CFI = 0.957, TLI = 0.954, RMSEA = 0.039), supporting the invariance of the measurement model across countries (e.g., Chen, 2007; Cheung & Rensvold, 2002) and fulfilling a sufficient condition for comparability of path coefficients of the latent interaction model across countries. Next we tested a multigroup SEM that included the main effects of science selfconcept and enjoyment on career-aspirations and extracurricular activities as well as their latent interaction (E V). All factor loadings for the latent predictor and outcome variables and the latent interaction were held invariant across countries. The model demonstrated good fit to the data (χ 2 (17649) = , CFI = 0.955, TLI = 0.953, RMSEA = 0.032).

14 Who took the out of expectancy-value theory? 14 The main effects of science self-concept were statistically significant for all 57 countries on career aspirations (M = 0.263, SD = 0.075), and for all but one country on extra-curricular activities (M = 0.263, SD = 0.075; the exception being Liechtenstein with a sample size of 339). The main effects of enjoyment on extra-curricular activities (M = 0.503, SD = 0.108) and on career aspirations (M = 0.481, SD = 0.063) were statistically significant in all 57 countries. For the latent interaction between science self-concept and enjoyment (E V) the following pattern emerged: The latent interaction effect on science-related activities (M = 0.067, SD = 0.017) was positive and statistically significant in all countries. The latent interaction effect on career aspirations was statistically significant (M = 0.055, SD = 0.023) in 49 countries; it was non-significant in 8 counties (see individual country parameters in Table S1, supplemental materials) Finally, we explored the cross-cultural generalizability of our results by constraining the structural coefficients path coefficients leading from EVT constructs to outcomes to be invariant across countries. The fit indices for the restricted model did not change substantively (χ 2 (17985) = , CFI = 0.953, TLI = 0.952, RMSEA = 0.032); in fact, changes in these fit indices were less than cut-off criteria typically used ( CFI = 0.002, TLI < 0.001, RMSEA < 0.001) to support the invariance of parameters over multiple groups (see Chen, 2007; Cheung & Rensvold, 2002). These results suggest that the restricted model fit the data nearly as well as the unrestricted model, and support the generalizability of the firstorder effects as well as the critical E V interaction effect across the diverse set of 57 countries participating in PISA Discussion Consistent with most good psychological mysteries, we began with an apparent crime (the missing E V interaction), identified the culprit (weak statistical methodology), gathered evidence to support our speculations, and returned the missing E V interaction to its rightful

15 Who took the out of expectancy-value theory? 15 place at the heart of EVT. In accomplishing this task, we demonstrated apparently the strongest support for the cross-national generalizability of EVT predictions ever undertaken. Based on nationally representative samples of 15-year-olds from 57 diverse countries, there was good support for the generalizability of the effects of self-concept, enjoyment, and their interaction for both engagement in science activities and plans to pursue science careers. Although it might be premature to claim the universality of EVT predictions including the lost E V interaction the support is very strong. The presented research is a prime example for a substantive-methodological synergy (Marsh & Hau, 2007): The application of new and emerging quantitative research methodology SEMs with latent interactions to a long-standing and unresolved substantive issue the missing E V interaction. Not only do our findings have important theoretical and practical implications for applied motivational researchers, they also demonstrate evolving methodology to test interaction effects in observational, non-experimental data (McClelland & Judd, 1993). SEMs with latent interactions based on large Ns allow strong tests of hypothesized multiplicative relations with models that are correctly specified to include both main effects and the interaction effect (Blanton & Jaccard, 2006; Cohen, 1978; Marsh et al., in press-a; also see footnote 1). In supplemental power analyses, we found that sample sizes of at least N=1000 that are typical in some large-scale studies are needed to consistently identify latent E V interactions similar to those in the international sample. However, with sample sizes of less than 500 that are typical in many applied studies, power was insufficient less than a 50% chance of identifying such E V interactions (see Technical Appendix for further discussion). Our findings also have direct implications for policy-practice and interventions that seek to increase long-term engagement and pursuit of careers in science. The essence of the non-compensatory, multiplicative relation between expectancy and value is that both have to

16 Who took the out of expectancy-value theory? 16 be high. It is not sufficient to either enhance academic self-concept or to enhance value; teachers supported by appropriate policy-practice must be sufficiently skilled to simultaneously enhance both constructs. If teachers focus on one to the exclusion of the other, then the influence of each is undermined. We have been successful in re-establishing the E V interaction to its rightful position at the centre of EVT, but much work is still needed. Within modern approaches to EVT of achievement motivation (e.g., Wigfield & Eccles, 2000), the value component is multifaceted, consisting of extrinsic value and cost in addition to intrinsic value represented by enjoyment in our study. Hence further research is needed in order to disentangle the interplay among these value components in motivating achievement-related behavior and choices. Our supplemental analyses show that support for the E V interaction generalizes over gender for science, but further research is warranted to evaluate its cross-cultural generalizability in other disciplines. More generally, evaluations of EVT require longitudinal data to provide stronger tests of the implicit assumptions of the causal ordering of the EVT constructs in relation to a wider variety of short- and long-term outcomes. However, based on the present investigation, we posit that E V interactions will continue to play an important role in such extensions of EVT.

17 Who took the out of expectancy-value theory? 17 References Ajzen, I. (1987). Attitudes, traits, and actions: Dispositional prediction of behavior in personality and social psychology. In L. Berkowitz (Ed.), Advances in experimental social psychology (Vol. 20, pp. 1-63). New York: Academic Press. Arnold, H. J., & Evans, M. G. (1979). Testing Multiplicative Models Does Not Require Ratio Scales. Organizational Behavior and Human Performance, 24(1), Atkinson, J. W. (1957). Motivational Determinants of Risk-Taking Behavior. Psychological Review, 64(6), Bauer, D. J., & Curran, P. J. (2005). Probing interactions in fixed and multilevel regression: Inferential and graphical techniques. Multivariate Behavioral Research, 40(3), Blanton, H., & Jaccard, J. (2006). Tests of multiplicative models in psychology: A case study using the unified theory of implicit attitudes, stereotypes, self-esteem, and selfconcept. Psychological Review, 113(1), doi: / x Busemeyer, J. R., & Jones, L. E. (1983). Analysis of Multiplicative Combination Rules When the Causal Variables Are Measured with Error. Psychological Bulletin, 93(3), Chen, F. F. (2007). Sensitivity of goodness of fit indexes to lack of measurement invariance. Structural Equation Modeling, 14(3), Cheung, G. W., & Rensvold, R. B. (2002). Evaluating goodness-of-fit indexes for testing measurement invariance. Structural Equation Modeling, 9(2), Cohen, J. (1978). Partialed products are interactions; partialed powers are curve components. Psychological Bulletin, 85, Cronbach, L. J. (1987). Statistical tests for moderator variables. Flaws in analyses recently proposed. Psychological Bulletin, 102,

18 Who took the out of expectancy-value theory? 18 Cronbach, L. J., & Snow, R. E. (1977). Aptitudes and instructional methods: A handbook for research on interactions. Oxford, UK: Irvington. Eccles (Parsons), J. S. (1983). Expectancies, values, and academic behaviours. In J. T. Spence (Ed.), Achievement and achievement motivation (pp ). San Francisco, CA: W. H. Freeman. Eccles, J. S., & Wigfield, A. (1995). In the mind of the actor: The structure of adolescents' achievement task values and expectancy-related beliefs. Personality and Social Psychology Bulletin, 21, Feather, N. T. (1959). Subjective probability and decision under uncertainty. Psychological Review, 66(3), Feather, N. T. (1982). Expectancy-value approaches: Present status and future directions. In N. T. Feather (Ed.), Expectations and actions. Expectancy-value models in psychology (pp ). Hillsdale, N.J.: Lawrence Erlbaum. Kenny, D. A., & Judd, C. M. (1984). Estimating the nonlinear and interactive effects of latent. Psychological Bulletin, 96(1), Klein, A. G., & Moosbrugger, H. (2000). Maximum likelihood estimation of latent interaction effects with the LMS method. Psychometrika, 65(4), Marsh, H. W. (1993). Relations between global and specific domains of self the importance of individual importance certainty, and ideals. Journal of Personality and Social Psychology, 65(5), Marsh, H. W. (2006). Self-concept theory, measurement and research into practice: The role of self-concept in educational psychology. Leicester, UK: British Psychological Society.

19 Who took the out of expectancy-value theory? 19 Marsh, H. W. (2008). The elusive importance effect: More failure for the Jamesian perspective on the importance of importance in shaping self-esteem. Journal of Personality, 76(5), doi: /j x Marsh, H. W., & Hau, K. T. (2007). Applications of latent-variable models in educational psychology: The need for methodological-substantive synergies. Contemporary Educational Psychology, 32(1), doi: /j.cedpsych Marsh, H.W., Hau, K.-T, Wen, Z., Nagengast, B., & Morin, A.J.S. (in press-a). Moderation. In T.D. Little (Ed.): The Oxford Handbook of Quantitative Methods. New York: Oxford University Press. Marsh, H. W., Wen, Z. L., & Hau, K. T. (2004). Structural equation models of latent interactions: Evaluation of alternative estimation strategies and indicator construction. Psychological Methods, 9(3), doi: / x Marsh, H. W., Wen, Z., Nagengast, B., & Hau, K. T. (in press-b). Structural equation models of latent interactions. In R. H. Hoyle (Ed.), Handbook of structural equation modeling. New York: Guilford. McClelland, G. H., & Judd, C. M. (1993). Statistical difficulties of detecting interactions and moderator effects. Psychological Bulletin, 114(2), Meece, J. L., Wigfield, A., & Eccles, J. S. (1990). Predictors of math anxiety and its influence on young adolescents course enrolment intentions and performance in mathematics. Journal of Educational Psychology, 82(1), Meredith, W. (1993). Measurement invariance, factor-analysis and factorial invariance. Psychometrika, 58(4), Mitchell, T. R. (1974). Expectancy models of job-satisfaction, occupational preference and effort - Theoretical, methodological, and empirical appraisal. Psychological Bulletin, 81(12),

20 Who took the out of expectancy-value theory? 20 Muthén, L. K., & Muthén, B. O. ( ). Mplus User's Guide (5 ed.). Los Angeles, CA: Muthén & Muthén. OECD. (2007). PISA 2006 Science Competencies for Tomorrow's World. Paris, France: OECD. OECD. (2009). PISA 2006 Technical Report. Paris, France: OECD. Preacher, K. J., Curran, P. J., & Bauer, D. J. (2006). Computational tools for probing interactions in multiple linear regression, multilevel modeling, and latent curve analysis. Journal of Educational and Behavioral Statistics, 31(4), Rigdon, E. E., Schumacker, R. E., & Wothke, W. (1998). A comparative review of interaction and nonlinear modeling. In R. E. Schumacker & G. A. Marcoulides (Eds.), Interaction and nonlinar effects in structural equation modeling. Mahwah, NJ: Lawrence Erlbaum. Tolman, E. C. (1938). The determiners of behavior at a choice point. Psychological Review, 45(1), Tolman, E. C. (1955). Principles of performance. Psychological Review, 62(5), Wen, Z., Marsh, H. W., & Hau, K. T. (2010). Structural equation models of latent interactions: an appropriate standardized solution and its scale-free properties. Structural Equation Modeling, 17(1), Wigfield, A., & Eccles, J. S. (2000). Expectancy-value theory of achievement motivation. Contemporary Educational Psychology, 25(1), Wigfield, A., Tonks, S., & Eccles, J. S. (2004). Expectancy-value theory in cross-cultural perspective. In D. M. McInerney & S. V. Etten (Eds.), Big theories revisited (pp ). Charlotte, N.C.: Information Age.

21 Who took the out of expectancy-value theory? 21 Footnotes 1 Another common mistake in specifying models to test multiplicative effects is the omission of main effects (see e.g., Blanton & Jaccard, 2006; Cohen, 1978) and exclusive tests of the product variable. The effect of the product variable alone is a mixture of main and interaction effects (Cohen, 1978) and can incorrectly lend support to a multiplicative model even though an additive model holds (Blanton & Jaccard, 2006; Cohen, 1978; Cronbach, 1987). Appropriate (latent and manifest) models for tests of interactions must always include both main effects and the product variable (Cohen, 1978, see also Marsh, Hau, Wen, Nagengast, & Morin in press-a).

22 Who took the out of expectancy-value theory? 22 Table 1 Path coefficients of the latent interaction models for the total international sample and average path coefficients of the multigroup analysis Total sample Multigroup analysis Predictor B s.e. p M SD Extracurricular activities Self-concept < Enjoyment < SCxEnj < Career aspirations Self-concept < Enjoyment < SCxEnj < Note: All parameter estimates were obtained from the fully standardized solution in Mplus 5.21, interaction effects have been restandardized according to Wen, Marsh and Hau (2010). The reported values for the multigroup analysis are means and standard deviations of the standardized parameter estimates over 57 countries (using the variance from the total international sample), so no meaningful p-values can be reported. SCxEnj = interaction of self-concept and enjoyment, b = standardized parameter estimate, s.e. = standard error, p = p- value, M = mean of standardized parameter estimate over 57 countries; SD = standard deviation of parameter estimate over 57 countries.

23 Who took the out of expectancy-value theory? 23 Figure 1. Hypothetical relations between expectancy and value in predicting motivation. The left panel shows the implication of an additive relation, the right panel shows the implications of a multiplicative relation. Theoretical relations between expectancy, value and motivation Additive: Expectancy + Value Multiplicative: Expectancy x Value Value = ++ Value = + Value = 0 Value = Value = Motivation Motivation Expectancy Expectancy

24 Who took the out of expectancy-value theory? 24 Figure 2. Path diagram and parameter estimates of the structural equation model with a latent interaction in the total sample of PISA 2006.

25 Who took the out of expectancy-value theory? 25 Figure 3. Simple slope plots of the relation between self-concept and enjoyment jointly predicting extra-curricular activities in science (left plot) and science career aspirations (right plot) in the total sample of PISA Latent Interaction Plots Extra Curricular Activities Career Aspirations 1 SD 0 1 SD 1 SD 0 1 SD Enjoyment = 2 SD Enjoyment = 1 SD Enjoyment = 0 Enjoyment = 1 SD Enjoyment = 2 SD 2 SD 1 SD 0 1 SD 2 SD Self Concept 2 SD 1 SD 0 1 SD 2 SD Self Concept

26 Who took the out of expectancy-value theory? 26 Technical Appendix Description of PISA database and Statistical Analyses In this technical appendix, we give more information on the PISA 2006 sample and the items that were used to measure the latent variables used in our research. We then describe some details of the implementation of structural equation models with latent interactions that we used to analyze the expectancy-value interaction. PISA 2006 The Program for International Student Assessment (PISA) is an ongoing project commissioned by the Organization for Economic Co-Operation and Development (OECD). Every three years (starting in 2000) PISA collects cross-sectional data on achievement, motivation and policy-relevant background data from nationally representative samples of 15-year-olds an age when students in most countries approach the end of their compulsory schooling. PISA aims to assess not only knowledge on a specific academic subject, but also the extent to which students can use their knowledge to solve problems encountered in real life. For PISA 2006, the data collection focuses on achievement and background data for science based on responses of 398,750 students from 57 countries (30 OECD countries and 27 partner countries). PISA used a two-stage sampling model in each country; a nationally representative sample of at least 150 schools enrolling 15 year-olds (sampling probabilities were proportional to school size) followed by a random selection of about year-old students from each school. PISA measures were compiled by international experts in the respective fields working in close collaboration with the participating countries. The crosscultural validity of the assessment material and the background measures are further ensured by employing state-of-the-art translation and back-translation protocols, and by extensive pre-testing in field trials before the main data collection (for further discussions see Marsh, Hau, Artelt, Baumert, & Peschar, 2006; OECD, 2007; OECD, 2009)

27 Who took the out of expectancy-value theory? 27 Measures In our analysis of the EVT-model, we used the following scales. All scales, unless otherwise indicated, were measured with positively worded items using 4-point Likert scales ranging from strongly agree to strongly disagree. For the analysis, the variables were recoded so that higher numeric values indicated higher values on the corresponding construct. Expectancy. The scale Science Self-Concept was used to operationalize students expectancy of success. The scale consisted of six items assessing students perception of their competencies in science (e.g. <School science> topics are easy for me. ). The median reliability in the OECD countries was α = 0.92 and in the non-oecd-countries α = 0.87 (OECD, 2009). Value. The scale Enjoyment of Science was used to represent intrinsic value of science. The scale consisted of five items that assessed the enjoyment students experienced when engaging in science-related activities (e.g., I generally have fun when I am learning <broad science> topics. ). The median reliability in the OECD countries was α = 0.92 and in the non-oecd-countries α = 0.87 (OECD, 2009). Behavioral Outcome. The scale Science-Related Extra-Curricular Activities was used as a behavioural outcome measure. The scale consisted of six items assessing the frequency of participation in non-compulsory and after-school science activities, e.g., watching TV programmes, reading magazines or attending science clubs. The labels of the answering categories ranged from never or hardly ever to very often. The median reliability in the OECD countries was α = 0.78 and in the non-oecd-countries α = 0.76 (OECD, 2009). Career Aspirations. The scale Future-Oriented Motivation in Science that measured career aspiration in science-related fields was used as a proxy measure for long-term academic choice. The scale consisted of four items that assessed the students intentions of

28 Who took the out of expectancy-value theory? 28 studying science after school and taking up a science-related career (e.g., I would like to work in a career involving <broad science> after <secondary school>. ). The median reliability in the OECD countries was α = 0.92 and in the non-oecd-countries α = 0.90 (OECD, 2009). Structural equation models with latent interactions (SEM-LIs). SEM-LIs have only recently become available to the applied social science research community, e.g. in form of distribution-analytic approaches such as the Latent Moderated Structural Equations model (LMS, Klein & Moosbrugger, 2000) or Quasi-Maximum Likelihood (QML, Klein & Muthen, 2007) and product-indicator approaches such as the unconstrained approach (Marsh, Wen, & Hau, 2006; Marsh, Wen, & Hau, 2004). For the analyses in this paper, we adapted the unconstrained approach to latent interactions (Marsh, et al., 2004) for cross-cultural studies. SEM-LIs seek to estimate the regression of the latent outcome variable η (achievement motivation in our case) on the latent predictor variables ξ (expectancy in our case) and ξ 2 (value in this case) and their cross-product ξ 1 ξ 2, representing the interaction effect. The inclusion of the cross-product distinguishes these models from conventional structural equation models (e.g., Jöreskog, 1970; McDonald, 1978) and the best way to estimate this effect has been an issue of longstanding debate (e.g., Marsh, Wen, et al., 2006; Marsh, et al., 2004; Schumacker & Marcoulides, 1998). The SEM-LI with two latent predictors is typically specified as: η = γ 1 ξ 1 + γ 2 ξ 2 + γ 3 ξ 1 ξ 2 + ζ, (1) where γ 1, γ 2 and γ 3 are the partial regression coefficients of the latent predictor variables and their cross-product and ζ is the structural model residual. The latent predictors ξ 1 and ξ 2 as well as the latent outcome variable η are each inferred from at least two indicators as specified in the corresponding measurement models

29 Who took the out of expectancy-value theory? 29 xij = λ xiξi + δij yk λ ykη + ε k =, (2) where xij is the jth indicator of the ith latent predictor variable ξ i, λxi is the corresponding factor loading and δij is the corresponding residual, y k is the kth indicator of the latent outcome variable η, λyk residual. is the corresponding factor loading, and ε k is the corresponding Product-indicator approaches such as the unconstrained approach identify the latent cross-product ξ 1 ξ 2 by products of indicators of the latent predictor variables ξ 1 and ξ 2 (Marsh, Wen, et al., 2006; Marsh, et al., 2004) according to the following measurement model x x 1i 2l λ 1i 2lξ1ξ 2 + δ1 i2l =, (3) where x 1 i is the ith indicator of ξ 1 and x 2 l is the lth indicator of ξ 2, 1i2l λ is the corresponding factor loading on the latent product variable and δ 1i2l is the corresponding residual (for details on selecting the product indicators see, Marsh, Wen, et al., 2006; Marsh, et al., 2004). In order to minimize the number of necessary constraints in the model, all indicators of the latent variables are standardized before the product-indicators are computed (Marsh, et al., 2004; Marsh, et al., 2007). Marsh et al. (2004) showed that it is only necessary to constrain the mean of the latent product variable to be equal to the covariance of the latent predictor variables, i.e., E ( ξ ) 1 ξ 2 = cov(ξ 1,ξ 2 ), (4) in order to identify the model. Estimation is carried out with a conventional statistical package for structural equation modelling (e.g., Mplus, L. K. Muthén & Muthén, ). Model estimation in Mplus was based on the robust maximum likelihood estimator (Yuan & Bentler, 2000) that provides standard errors and fit statistics that correct for nonnormality of the product-indicators. This estimator has been shown to provide good results

30 Who took the out of expectancy-value theory? 30 when indicators are based on Likert-scales with at least four answer categories (Beauducel & Herzberg, 2006; DiStefano, 2002; Dolan, 1994; Muthén & Kaplan, 1985). In order to test the cross-cultural generalizability of the EVT interaction, we extended the unconstrained approach to multiple groups (defined by the individual countries in the PISA 2006 data). Multiple group structural equation models specify separate measurement and structural models (i.e., equations (1) (4)), for each country g, exemplified here with the structural model including the interaction effect η = γ ξ + γ ξ + γ ξ ξ + ς (5) ( g ) ( g ) ( g ) ( g ) ( g ) ( g ) ( g ) ( g ) ( g ) and testing parameter invariance constraints across countries (see Marsh, et al., 2009, for a full taxonomy of invariance models and tests). Appropriate Model Specification for Tests of Interaction Effects A common mistake in specifying models to test multiplicative effects is the omission of main effects (see e.g., Blanton & Jaccard, 2006; Cohen, 1978) and exclusive tests of the product variable. The effects of the product variable alone are a mixture of main effects and interaction effects (Cohen, 1978) and can incorrectly lend support to a multiplicative model even though an additive model holds (Blanton & Jaccard, 2006; Cohen, 1978; Cronbach, 1987). Appropriate models for tests of interaction effects must always include both main effects and the product variable (Cohen, 1978, see also Marsh, Hau, Wen, Nagengast, & Morin in press). This appropriate model specification applies to latent interaction models like those presented here as well as multiple regression models based on manifest effects like those described by Cohen (1978) and Cronbach (1987). Standardization As recommended by Marsh et al. (2004), all manifest variables were standardized for the total group before the analysis. Product indicators for the latent interactions were formed on the basis of matched pair-strategy and not re-standardized in line with the

31 Who took the out of expectancy-value theory? 31 recommendations given in Marsh et al. (2004). Because of the different number of indicators for the two latent predictors (Self-concept: six; Enjoyment: five), two self-concept items were averaged for the construction of the product indicators. For the multiple group analysis, the standardized manifest indicators were centred (but not re-standardized) around their countryspecific mean before computing the product-indicators for the latent interaction variable. Assessment of model fit Following Marsh, Balla, and Hau (1996; also see Marsh, Balla, & McDonald, 1988; Marsh, Hau, & Wen, 2004) we considered the Tucker-Lewis index (TLI), the comparative fit index (CFI), the root mean square error of approximation (RMSEA), and the Standardized Root Mean Square Residual (SRMR) to evaluate goodness of fit, as well as the χ 2 -test statistic and an evaluation of parameter estimates. The TLI and CFI vary along a 0-to-1 continuum in which values greater than.90 and.95 are typically taken to reflect acceptable and excellent fits to the data, respectively. SRMR values below.09 indicate good fit. RMSEA values of less than.06 are taken to reflect a reasonable fit. Whereas, RMSEA values greater than.10 are unacceptable, although no golden rule exists (Chen, Curran, Bollen, Kirby, & Paxton, 2008; Hu & Bentler, 1999; Marsh, Hau & Wen, 2004). The CFI contains no penalty for a lack of parsimony, so that improved fit due to the introduction of additional parameters may reflect capitalization on chance, whereas the TLI and RMSEA contain penalties for a lack of parsimony (for further discussion see Cheung & Rensvold, 2002; Hu & Bentler, 1999; Marsh et al., 2004). For comparison of nested models by differences in fit indices, we followed the recommendations given by Chen (2007) and Cheung and Rensvold (2002) who suggested that a decrease in fit for the more parsimonious model of less than.01 for incremental fit indices like the CFI, should be treated as support for this model. Chen (2007) suggested that when the RMSEA increases by less than.015 there is support for the more constrained model.