Measuring a Mastery Goal Structure Using the TARGET Framework

Transcription

1 Original Article Measuring a Mastery Goal Structure Using the TARGET Framework Development and Validation of a Classroom Goal Structure Questionnaire Marko Lüftenegger, 1 Ulrich S. Tran, 2 Lisa Bardach, 1 Barbara Schober, 1 and Christiane Spiel 1 1 Department of Applied Psychology: Work, Education and Economy, Faculty of Psychology, University of Vienna, Austria 2 Department of Basic Psychological Research and Research Methods, Faculty of Psychology, University of Vienna, Austria Abstract: In prior research, goal structures have been measured as macroscopic and holistic constructs referring to all activities in the classroom setting associated with learning and performing on a meta-level. A more comprehensive approach for identifying concrete classroom structures that should foster students mastery goals is provided by the multidimensional TARGET framework with its six instructional dimensions (Task, Autonomy, Recognition, Grouping, Evaluation, Time). However, measurement instruments assessing students perceptions of all TARGET dimensions are largely lacking. The main aim of this study was to develop and validate a new student questionnaire for comprehensive assessment of the perceived TARGET classroom structure (the Goal Structure Questionnaire GSQ). Scales were constructed using a rational-empirical strategy based on classical conceptions of the TARGET dimensions and prior empirical research. The instrument was tested in a study using a sample of 1,080 secondary school students. Findings indicate that the scales are reliable, internally valid, and externally valid in terms of relationships with students achievement goals. More concretely, analyses revealed that the TARGET mastery goal structure positively predicts mastery goals, performance approach goals, and an incremental implicit theory of intelligence. No associations were found with performance avoidance goals. Keywords: motivation, achievement goals, classroom, goal structure, TARGET framework The most important attitude that can be formed is that of desire to go on learning John Dewey (1938, p.48). One of the most important objectives of school is to enable, empower, and equip children to become lifelong learners (Schober, Lüftenegger, Wagner, Finsterwald, & Spiel, 2013). This is crucial for both personal development across the life span as well as the necessity of being able to handle constant change and transition as a result of rapid technological and scientific changes, organizational innovation, and global competition. Schools and in particular contextual characteristics in the classroom can promote and children can adopt profoundly different definitions of what teaching and learning are about: mastery or performance goals. Mastery goals are related to developing new skills and improving one s level of competence, whereas performance goals focus on demonstrating competence and ability in comparison to others (approach focus), or avoiding failure and unfavorable judgments of one s ability by others (avoidance focus) (Elliot, 2005). Studies demonstrated that students show the most beneficial motivational and cognitive patterns when they focus on mastery goals (see Meece, Anderman, & Anderman, 2006). Consequently, researchers investigated characteristics of the classroom context in order to describe the extent and the way in which environmental factors support the adoption of different goals. One prominent example of these efforts is the TARGET framework (Epstein, 1988) with its six instructional strategies or dimensions (Task, Authority, Recognition, Grouping, Evaluation, Time). Different types of achievement goals can be stressed in classrooms along any of these dimensions, and students tend to adopt these goals (Meece et al., 2006). However, measurement instruments assessing students perceptions of the TARGET dimensions are largely lacking. Of particular importance, a comprehensive measurement instrument assessing all six TARGET dimensions that represent a mastery goal structure is missing so far. Additionally, the majority of studies examining consequences of goal structures have mainly relied on college or high school student samples. Zeitschrift für Psychologie (2017), 225(1), DOI: / /a Ó 2017 Hogrefe Publishing

2 M. Lüftenegger et al., Classroom Goal Structure Questionnaire 65 Thus, we know less about the adoption of achievement goals in younger students, particularly immediately after the transition to secondary school. Therefore, the aim of the present study is to develop a measurement instrument to adequately assess a perceived mastery goal structure following the six TARGET dimensions. Moreover, another aim is to test the associations between the perceived TARGET mastery goal structure and personal achievement goals. Personal Achievement Goals In educational settings, student goals are the purposes or reasons for engaging, choosing, and persisting in different learning activities or achievement tasks (Pintrich, 2003). Research on students achievement goals has a long tradition in educational research and has resulted in the development of various conceptual models. The dichotomous model (Ames, 1992; Dweck & Leggett, 1988; Maehr, 1989; Nicholls, 1984) distinguishes between mastery goals (to develop competence) and performance goals (to demonstrate competence). Central for individuals pursuing mastery goals is a focus on developing new skills, improving one s level of competence, and trying to understand new learning subjects. In contrast, individuals with performance goals focus on demonstrating their competence and ability in comparison to others (approach focus). The dichotomous model was extended by bifurcating performance goals into performance approach and performance goals (focus on avoiding failure and unfavorable judgments of one s ability by others, respectively) (Elliot, 2005). This trichotomous achievement goal model was further expanded into a 2 2 model by also bifurcating the mastery goal construct into approach and avoidance goal types (Elliot, 2005). In addition to the expansion of the model, there is an active debate about the precise definition of achievement goals (see Hulleman, Schrager, Bodmann, & Harackiewicz, 2010; Kaplan & Maehr, 2007). In this study, we confine ourselves to the trichotomous achievement goal model as a theoretical basis because the mastery-avoidance construct has received little empirical support at the early secondary education level (junior high school) so far. Young children and adolescents are still improving their competences, wherefore mastery-avoidance goals may be of greater importance among older populations (Lee & Bong, 2016). The effects of students achievement goals have been extensively studied in experimental and correlational research designs over the last three decades. Most studies have relied on samples from a higher education context (college, university). We focus on empirical findings within the school context. The findings for mastery goals have been consistent and mostly favorable. Mastery goals are positively associated with greater effort and persistence in learning (Wolters 2004), deep-level learning strategies (Greene, Miller, Crowson, Duke, & Akey, 2004), selfefficacy (Bong, 2009), and well-being (Kaplan & Maehr, 1999). The association between mastery goals and performance is more complex, and the empirical evidence is mixed so far (Bergsmann, Lüftenegger, Jöstl, Schober, & Spiel, 2013; Bong, 2009; Hulleman et al., 2010; Kaplan& Maehr, 1999). Performance avoidance goals are typically associated with negative consequences like self-handicapping (Midgley & Urdan, 2001; Urdan, 2004), anxiety (Federici, Skaalvik, & Tangen, 2015), procrastination and low persistence (Wolters, 2004). In contrast, empirical findings on performance approach goals are mixed and possible consequences are subject to controversial discussion (see Senko, Hulleman, & Harackiewicz, 2011). For example, this applies to the relation between performance approach goals and academic performance: Performance approach goals have been found to be positively related to academic performance (Bong, 2009) or not related(paulick, Watermann, & Nückles, 2011). These mixed results with regard to performance can be partly explained by the different conceptualizations of performance approach goals (focus on normative comparison vs. demonstration of competence; see Senko et al., 2011). For instance, normative performance approach goals are positively associated with performance, whereas competence demonstration goals are not (see Hulleman et al., 2010). Goal Structure and TARGET Framework From the beginnings of achievement goal theory, researchers have highlighted the idea that students adoption of personal achievement goals may be influenced by what happens in the classroom (Ames 1992; Maehr & Midgley, 1996). Classroom structure has the potential to be managed and modified by the actions of those within the classroom (both teachers and students) and can make different achievement goals salient on a contextual level. Children tend to adopt the goals that are stressed in their classroom as their own guiding purposes. More accurately, their perception of these goal structures (Meece et al., 2006), also known as classroom goal structures or classroom goals, should determine the adoption of personal achievement goals. To avoid ambiguities, we use the term goal structures consistently throughout this paper. Empirical findings are in agreement and strongly suggest that perceived goal structures are related to students personal achievement goals (e.g., Bergsmann et al., 2013; Church, Eliott, & Gable, 2001; Federici et al., 2015; Greene et al., 2004; Lau& Nie, 2008; Lüftenegger, van de Schoot, Schober, Finsterwald, & Spiel, 2014; Urdan, 2004; Wolters, 2004). Ó 2017 Hogrefe Publishing Zeitschrift für Psychologie (2017), 225(1), 64 75

3 66 M. Lüftenegger et al., Classroom Goal Structure Questionnaire However, which classroom structures exactly shape students achievement goals? Researchers have identified core dimensions of instructional practices in classrooms involved in the shaping of students personal achievement goals. Joyce Epstein (1988) used the acronym TARGET for a prominent systematization of key classroom dimensions that affect students development and learning: Task design, distribution of Authority/autonomy, Recognition/rewards of students, Grouping arrangements, Evaluation practices, and Time allocation. Carole Ames (1992) used these six classroom dimensions to describe how personal mastery goals should be facilitated. Therefore, the TARGET dimensions were conceptualized to represent a mastery goal structure. The task dimension concerns the design of classwork and homework. Appropriate tasks include a focus on learning, moderate challenges, curiosity, and active involvement. Authority refers to the opportunity to participate actively in making decisions in the classroom that are relevant to instruction. Teachers share authority over instructional decisions with students, taking into account their needs and feelings. The opportunity to decide for oneself what exercises and tasks one should complete in a certain subject area is also included in this dimension, as is shared responsibility in social decision-making processes, for instance enacting class rules. To better reflect the theoretical description of this classroom dimension, we decided to use autonomy instead of authority throughout the manuscript. The recognition dimension concerns the formal or informal provision of recognition through incentives, rewards, or feedback. Rewards can be useful for students if they provide information about their progress or competence. The grouping dimension involves the use of heterogeneous cooperative groups and peer interaction to encourage working with others (Ames, 1992). The evaluation dimension focuses on methods that assess progress and improvement while avoiding the establishment of a competitive environment. Students should experience that it is normal to make mistakes and that these are allowed in the classroom (Steuer, Rosentritt-Brunn, & Dresel, 2013). Time encompasses the appropriateness of workload, the pace of instruction, and the time allotted for students to introduce their own topics and interests. The time dimension is closely linked to the design of tasks and autonomy. In some concepts and studies, time and task are treated as a joint single dimension (Ames, 1992; Greene et al., 2004). Against the background of the manifold nature of the aspects covered, we assume that the six described dimensions are distinguishable, but nevertheless interrelated subdimensions of students perception of their goal structure in line with the TARGET framework (Lüftenegger et al., 2014). We assume that each of the dimensions contributes to the perception of the overall TARGET goal structure in the classroom. Therefore, we assume that perceptions of the six subdimensions vary rather consistently between learners and that one superordinate uniform factor of TARGET can be conceptualized. Assessment of Perceived Goal Structure and TARGET In line with tradition among achievement goal researchers, we argue that students subjective perception of their learning environment is the appropriate source of data (e.g., Maehr & Midgley, 1996; Schwinger & Stiensmeier-Pelster, 2011; but also Lüdtke, Robitzsch, Trautwein, & Kunter, 2009).It is not what the assumed objective outsider (e.g., the teacher or the researcher) sees that is the immediate cause of student attitudes, behavior, or goal adoption. Students perceptions of the goals emphasized in the classroom affect thoughts, beliefs, and attitudes associated with the personal investment in learning. Different classrooms have different learning environments with different classroom goal messages that influence students goal adoption. Therefore, it is important to examine the degree to which perceptions in classrooms are shared. A broad array of empirical evidence indicates that a mastery goal structure is linked to students adoption of a personal mastery goal orientation (e.g., Lau & Lee, 2008; Lüftenegger et al., 2014; Urdan, 2004). Studies examining the relation between a perceived mastery goal structure and students personal performance approach and avoidance goals reveal contradictory results: For instance, one study showed that the perception of a mastery goal structure was positively related to both types of performance goals (Federici et al., 2015), whereas no significant relations between mastery goal structure and the two performance goal orientations were found in another study (Midgley & Urdan, 2001). Only including performance approach goals in their analyses, Lau and Lee (2008) showed a positive relationship between the perception of a mastery goal structure and performance approach goals. Other studies report that a mastery goal structure negatively predicts performance approach goals or found no significant relation with performance avoidance goals (Urdan, 2004; Wolters, 2004). In summary, reported studies cover the whole range of possible relations between a mastery goal structure and the two types of performance goals and no consistent pattern explaining these differing results could be found. It should be noted that the reported studies differ with respect to the complexity of their statistical analyses. However, the inconsistent empirical evidence could also possibly be traced back to the measurement instruments used and their underlying conceptualizations of performance Zeitschrift für Psychologie (2017), 225(1), Ó 2017 Hogrefe Publishing

4 M. Lüftenegger et al., Classroom Goal Structure Questionnaire 67 approach and avoidance goals. Instruments assessing achievement goals often vary in their theoretical conceptions and operational definitions of performance goals (see e.g., Hackel, Jones, Carbonneau, & Mueller, 2016). Regarding the operationalization of performance approach and avoidance goals, two critical elements can be distinguished: the desire to demonstrate competence (e.g., Kaplan & Maehr, 2007) ortooutperformothers(e.g., Elliot, 2005). Therefore, Senko and colleagues (2011) suggest refining the constructs of both performance goals by distinguishing between normatively-focused performance goals (outperforming others) and appearance-based performance goals (demonstrating competence). Current measurements of achievement goals tend to either mix the normative and appearance components in their scales for performance approach and performance avoidance goals (e.g., SELLMO; Spinath, Stiensmeier-Pelster, Schöne, & Dickhäuser, 2002), or use items that assess solely normative (e.g., Achievement Goals Questionnaire, AGQ; Elliot & Murayama, 2008) or appearance aspects (e.g., Patterns of Adaptive Learning, PALS; Midgley et al., 2000). So far, goal structures in the classroom have been measured as a macroscopic and holistic construct that refers to all activities in the classroom setting associated with learning and performing on a meta-level (e.g., PALS, Midgley et al., 2000; Schwinger & Stiensmeier-Pelster, 2011). Research on the TARGET framework has been mostly conducted in empirical studies where individual dimensions were investigated separately (Church et al., 2001; Greene et al., 2004; Lau & Lee, 2008; Tapola & Niemivirta, 2008), or where a few dimensions representing a mastery goal structure were observed together (Bergsmann et al., 2013). Examining singular dimensions, therefore, presumably leads to different results than considering classroom instruction in its entire complexity (Church et al., 2001). Particularly, the systematic promotion of mastery goals possibly necessitates a classroom structure that does not singularly focus on a set of strategies or a particular instructional method, but rather the engagement of a complete constellation of strategies that are conceptually related, such as the system theoretically provided by the TARGET framework. Lüftenegger and colleagues (2014) considered all six TARGET dimensions and could show in a longitudinal design that a mastery goal structure following TARGET has a causal effect on junior high school students personal mastery goals. However, the measurement of several dimensions was very limited in terms of content validity due to the use of single items. So far, to the best of our knowledge, no instrument has been presented that comprehensively assesses all six TARGET dimensions and has been rigorously validated. Due to the comprehensive conceptualization of the TARGET dimensions in the classroom, it is not sufficient to include some items in other instruments of mastery goal structure (e.g., PALS, Midgley et al., 2000) or existing operationalizations of TARGET dimensions (Church et al., 2001; Greene et al., 2004; Lau & Lee, 2008; Tapola & Niemivirta, 2008; Lüftenegger et al., 2014). Instead, a comprehensive assessment of all six dimensions seems necessary to ensure the content validity of scores regarding the overall TARGET goal structure. The Present Investigation Based on prior work on favorable goal structures, the main purpose of the present study was to develop a reliable measurement instrument to assess perceived goal structure based on the TARGET framework with its six subdimensions. Furthermore, we aimed to analyze the effects of this perceived TARGET goal structure on students achievement goals and implicit theories. Specifically, the present research is designed to investigate the structural validity of the measurement instrument and to examine links with achievement goals and implicit theories that have been shown to be both conceptually and empirically important in prior work on goal structure and personal achievement goals more generally. Based on theory and prior research (Lüftenegger et al., 2014), we expected the TARGET mastery goal structure to be a multidimensional classroom characteristic. We expected that the TARGET goal structure has multiple, interrelated subdimensions that, in concert, constitute a superordinate and uniform overall perceived TARGET mastery classroom structure (Hypothesis 1a). This varies between classrooms (Hypothesis 1b). The second objective was to investigate how the TARGET classroom goal structure is related to students personal achievement goals and implicit theories of intelligence. Prior research (Lüftenegger et al., 2014) found that a perceived TARGET mastery goal structure influenced students mastery goals over time. To our knowledge, no study has investigated the effects on other achievement goals. Moreover, we focused on implicit theories of intelligence, which have been shown to be conceptually and empirically important for the adoption of achievement goals (Dweck & Leggett, 1988). To our knowledge, studies focusing on goal structures as possible antecedents of implicit theories are lacking so far. As such, it would seem promising to include implicit theories in the goal structure s nomological network. Perceived TARGET mastery goal structure is expected to positively predict personal mastery goals (Hypothesis 2a). Perceived TARGET mastery goal structure is expected to predict implicit theories(hypothesis 2b). Due to inconsistent research results, interrelations with performance goal types Ó 2017 Hogrefe Publishing Zeitschrift für Psychologie (2017), 225(1), 64 75

5 68 M. Lüftenegger et al., Classroom Goal Structure Questionnaire (approach normative, approach appearance, avoidance normative, avoidance appearance) are indeed expected, but their direction cannot be specified (Hypothesis 2c). Method Sample The survey was conducted with 1,080 Austrian students in May Participation was voluntary, and only students with active parental consent participated in the study. Less than 1% of students were not allowed to participate by their parents. Students completed the questionnaire during normal classroom hours and were instructed by trained research assistants. The students did not receive compensation for their participation in the study. The data were collected in five academic-track secondary schools (Gymnasium schools) in Vienna. In line with the typical composition of academic-track schools in Austria, girls were slightly overrepresented in this study (53.2%ofthe sample). Students mean age was 12.8 years (SD = 1.01) andthey were enrolled in grades six (34%), seven (37.1%), and eight (28.9%). The average number of children per classroom was (SD = 3.29). The subject for each class was determined randomly prior to data collection. The subjects investigated were German (51.8%) and mathematics (48.2%). In Austria, the new school year begins in September; therefore, sufficient time had clearly passed for goal structures to be established. Measures Personal achievement goals, the TARGET goal structure, and implicit theories were all assessed with a questionnaire. All questionnaire items, except those measuring implicit theories, used a 6-point scale ranging from 1 (= strongly disagree) to 6 (= strongly agree) and referred to specific school subjects, that is, mathematics and German language class. Mathematics and German language were chosen as the focus of study, because previous research on goals in secondary schools has found these to be particularly important domains of inquiry (e.g., Midgley & Urdan, 2001; Murayama, & Elliot, 2009; Schwinger & Stiensmeier- Pelster, 2011; Wolters, 2004). Personal Achievement Goals Following the trichotomous achievement goal conceptualization, we assessed mastery goals, performance approach goals, and performance avoidance goals with the respective subscales of the well-validated German achievement goal questionnaire SELLMO-S (Spinath et al., 2002). The scales reflect several dimensions of achievement goals classified in a meta-analysis of achievement goal measures (Hulleman et al., 2010). The items for mastery goals mainly focus on a preference for challenging activities, but also represent the motivation to master a task in order to gain new knowledge. The operationalization of performance approach goals and performance avoidance goals focuses on both the normative and appearance components of performance goals. We divided both performance goals scales into two subscales, one with a focus on appearance and the other on normative comparisons, resulting in four performance goal scales (approach normative, approach appearance, avoidance normative, avoidance appearance). All items were introduced with the phrase In my German/math class, I personally strive... followed by the statements referring to the respective goal type. Examples are...tolearnasmuchaspossible for mastery goals (eight items),...to get my work done better than others for performance approach goals with a normative focus (four items),...to demonstrate what I can and what I know for performance approach goals with an appearance focus (three items),...not to stand out by stupid questions for performance avoidance goals with an appearance focus (three items) and...to that other students don t consider me stupid for performance avoidance goals with a normative focus (five items) (see Schwinger & Stiensmeier-Pelster, 2011, for a complete list of items). Internal consistencies of the five subscales were good (α =.70.86; CR =.71.86). Implicit Theory of Intelligence Students implicit theory of intelligence was measured using a subscale of a well-established German instrument for the assessment of subjective beliefs about factors underlying learning and achievement (SE-SÜBELLKO; Spinath & Schöne, 2003). The scale consists of three items in the form of statements about the nature of intelligence. Students completed these statements by indicating the degree of malleability they believe in on a 6-point semantic differential (sample item: You have a certain amount of intelligence that cannot be changed vs. that can be changed. ). Higher values represent higher endorsement of an incremental theory (α =.80; CR =.80). TARGET Goal Structure TARGET goal structure was assessed via student perceptions of the six proposed dimensions. These comprised Task, Autonomy, Recognition, Grouping, Evaluation, and Time (the items are provided in the Electronic Supplementary Material, ESM 1). The development of the measurement instrument comprised several steps. First, we newly formulated items for each TARGET dimension that were derived from the conceptual understanding of the respective dimension. Second, to ensure content validity, we revised these items using expert judgments from members of our research group. In the next step, we selected the ten items with the best representation of the conceptual understanding of the respective dimension Zeitschrift für Psychologie (2017), 225(1), Ó 2017 Hogrefe Publishing

6 M. Lüftenegger et al., Classroom Goal Structure Questionnaire 69 and used criteria for semantic redundancy. These items were used as a preliminary version of the questionnaire in the present study. The selection of items for the final scales was based on several criteria: (1) having an efficient and balanced instrument; (2) representation of all proposed aspects of the respected TARGET dimension; (3) good psychometric properties of the scales (reliability and validity). This also included attending to both convergent and divergent scale validity. Items were selected according to convergent item validity (i.e., high factor loadings on the relevant TARGET scale) as well as divergent item validity (i.e., low factor loadings on other TARGET scales). We selected six items per dimension according to these three criteria. Analyses on the item level showed sufficient properties for the 36 items constituting the final Goal Structure Questionnaire (GSQ). Analyses on the scale level also revealed sufficient properties for all subscales (α =.69.85;CR =.69.85). Strategy of Analyses and Missing Data Statistical analyses were conducted using structural equation models (SEMs) with the complex design option in Mplus 7.4 (Muthén & Muthén, ) to control for the hierarchical nature of the data. The complex design option takes into account the nonindependence of the scores of students from the same class that is, the clustering effect of students nested within classes. As the constructs are measured as ordinal-level variables, we used robust weighted least squares estimation (WLSMV) for all analyses. Goodness-of-fit of the models was evaluated using several different indices, including the w 2 Test of Model Fit, the Tucker-Lewis Index (TLI), Comparative Fit Index (CFI), and Root Mean Square Error of Approximation (RMSEA). In addition, a 90% confidence interval around the point estimate enabled an assessment of the precision of the RMSEA estimate (for details about these indices, see Kline, 2011). We used traditional cutoff scores indicative of excellent and adequate fit to the data, respectively: CFI and TLI.95 and.90, andrmsea.06 and.08. We provide standardized coefficients. Standardized coefficients represent the amount of change in the outcome that can be expected from a one standard deviation unit change in the predictors. Following Cohen s (1988) guidelines, in the context of regression parameters, standardized values greater than 0.10, 0.30, and 0.50 generally reflect small, moderate, and large effect sizes. The influence of domain and sex was investigated in preliminary analyses. Main effects of the domain were found for autonomy and grouping. Main effects of sex were found only for mastery goals. All significant main effects were considered in the main analysis. The rate of individuals omitting items (nonresponse) in the present study was at a maximum of 1.3% forall considered items and, as such, very low. We used the full information maximum likelihood (FIML) approach implemented in Mplus 7.4 to deal with missing values. This approach takes all available information from the observed data into account when estimating parameter estimates and standard errors. Results Table 1 provides descriptive statistics, bivariate correlations, and reliabilities (Cronbach s α and composite reliability) of the investigated variables. The findings indicate that there was sufficient variation of scores on all scales and reliability for all scales ranged from moderate to excellent. Dimensionality of the Perceived Goal Structure In order to analyze the dimensionality of the perceived TARGET goal structure, confirmatory factor analyses (CFAs) were performed. In a first step, six CFAs were conducted to examine the construct validity of each of the six TARGET dimensions. All six models showed moderate to good model fit, CFI = , TLI = , RMSEA = Standardized item loadings were in the range of λ = with four exceptions: two items in the subscale task and two items for grouping only had a standardized loading of less than of λ =.27 and were excluded from all subsequent analyses and the final scales (see ESM 1). Without these items, the task and grouping model was characterized by good model fit indices and satisfactory standardized loadings (Figure 1). In a second step, we examined three models to test for the dimensionality of the perceived goal structure. Our hypothesized Model 1 included one factor for each of the six perceived TARGET dimensions with loadings of the respective items. This model fitted well with the data (see Table 2). We additionally tested Model 1 against an alternative model. Model 2 was a one-factor model reflecting a strictly unidimensional conceptualization of perceived goal structure in which all items load on one factor (Model 2). Model estimation and model comparison revealed significant advantages for the hypothesized Model 1 (see Table 2). To test our hypothesis that the six TARGET dimensions constitute a superordinate and uniform factor reflecting the overall goal structure in the classroom (Hypothesis 1), we specified another model (Model 3):Based on Model 1, we modeled one second-order factor with loadings of all six dimensions. This model also showed acceptable fit to the data (see Table 2), although slightly worse than that for Model 1. The model comparison revealed advantages for Model 1 over Model 2. Model3 provides estimates of the latent relationships, which were in the range of φ =.54.94, indicating the appropriateness of a mastery goal structure conception including six subdimensions. Ó 2017 Hogrefe Publishing Zeitschrift für Psychologie (2017), 225(1), 64 75

7 70 M. Lüftenegger et al., Classroom Goal Structure Questionnaire Table 1. Bivariate correlations, descriptive statistics, and reliabilities TARGET classroom structure 1. Task 2. Autonomy Recognition Grouping Evaluation Time Student characteristics 7. Mastery goal Performance goal AP-N Performance goal AP-A Performance goal AV-N Performance goal AV-A Implicit theories Number of items M SD Skewness Actual range α CR % CI CR ICC ICC Notes. AP-A = approach appearance; AP-N = approach normative; AV-A = avoidance appearance; AV-N = avoidance normative; CI = confidence interval; CR = composite reliability; ICC = intraclass correlation. r.07, p <.05. Zeitschrift für Psychologie (2017), 225(1), Ó 2017 Hogrefe Publishing

8 M. Lüftenegger et al., Classroom Goal Structure Questionnaire 71 Figure 1. Structural equation modeling showing external linkages of the superordinate and uniform TARGET goal structure factor. Standardized regression coefficients are reported and nonsignificant paths are shown as dashed lines. PGAP-A = performance goals approach appearance; PGAP-N = performance goals approach normative; PGAV-A = performance goals avoidance appearance; PGAV-N = performance goals avoidance normative. Although some of the relationships between conceptual similar constructs, such as recognition and evaluation, were high, they clearly indicate that all of the dimensions are separable, given that the latent coefficients were corrected for unreliability and represent the highest possible estimates for these relationships. These results indicate that it is justifiable to conceptualize perceived goal structure as hierarchically structured, consisting of distinguishable subdimensions that contribute to one superordinate uniform factor of mastery goal structure (see Hypothesis 1a). Thus, we decided in favor of the model with one superordinate factor and six dimensions. Classroom Differences As expected in Hypothesis 1b, we were able to find significant and moderate to large differences between classrooms in all six subdimensions and the superordinate uniform factor of mastery goal structure (ICC1 =.11.30; p <.001; seetable1). Within classrooms, students perceptions of the TARGET subdimensions seem to be rather Ó 2017 Hogrefe Publishing Zeitschrift für Psychologie (2017), 225(1), 64 75

9 72 M. Lüftenegger et al., Classroom Goal Structure Questionnaire Table 2. Results from CFA models of mastery classroom goal structure indicators w 2 or Δw 2 df or Δdf CFI TLI RMSEA [90% CI] Model fit Model 1: six dimensions * [ ] Model 2: one overall g factor * [ ] Model 3: six subfactors and one superordinate * [ ] uniform factor of mastery goal structure Model comparison Model 1 vs. Model * 15 Model 1 vs. Model * 9 Model 3 vs. Model * 6 Notes. n = 1,080. All items were treated as ordered categorical, utilizing the WLSMV estimator in Mplus (Muthén & Muthén, ). Model comparisons were conducted using the robust difference testing procedure for mean and variance adjusted test statistics. CFA = confirmatory factor analysis; CFI = comparative fit index; RMSEA = root mean square error of approximation; TLI: tucker-lewis index. *p <.001. homogeneous (ICC2 =.74.91). ICC2 can be interpreted as an indicator of the reliability of the measurement of a contextual characteristic via several individual perceptions. ICC2 is interpreted in line with other reliability measures (Lüdtke et al., 2009). Predicting Students Personal Achievement Goals In order to investigate whether students perception of the TARGET mastery goal structure can predict their personal achievement goals and implicit theories (Hypotheses 2a 2c), structural equation modeling was employed (see Figure 1 structural model). Overall fit indices showed a good model fit, CFI =.943, TLI=.940, RMSEA =.026 [.024,.028]. Estimation revealed that a perceived TARGET goal structure positively predicted personal mastery goals (b =.53, SE =.03, p <.001), performance approach goals with an appearance (b =.42, SE =.04, p <.001) and normative focus (b =.07, SE =.04, p =.048), and implicit theories (b =.17, SE =.03, p <.001). No associations were found between the TARGET goal structure and performance avoidance goals for both appearance (b =.02, SE =.04, p =.632) and normative avoidance focus (b =.04, SE =.03, p =.237). Discussion The purpose of the study was to develop and validate a measurement instrument which adequately assesses goal structures within the TARGET conceptualization. By building upon previous work (e.g., Lüftenegger et al., 2014), this study aimed to extend existing findings and refine previous attempts to measure goal structures in the TARGET framework. Our newly developed Goal Structure Questionnaire (GSQ) incorporates a broad range of theoretically-rooted aspects of the TARGET dimensions and can therefore be considered a highly comprehensive and differentiated measurement instrument. Results revealed that students perceptions comprise six dimensions of interrelated but distinguishable classroom structures (Task, Authority, Recognition, Grouping, Evaluation, Time). Instead of regarding these six dimensions as isolated aspects, this study s findings provide a more holistic view of goal structures. Taken together, the six dimensions form a learning environment in classrooms which we refer to as TARGET. The findings of this study support the existence of TARGET as a macroscopic construct representing students overall perceptions of their mastery goal structure (Hypothesis 1a). As students in different classrooms receive different goal-relevant messages from their teachers and are exposed to different classroom practices and activities, we assumed that the perception of TARGET varies between classrooms. As hypothesized, we found differences between perceived TARGET goal structures in different classrooms, indicating adequate variability of TARGET goal structures on the class level (Hypothesis 1b). All in all, these results substantiate the internal validity of the TARGET questionnaire. Moreover, the study shows that the TARGET questionnaire has predictive power in explaining students mastery goals (Hypothesis 2a) and implicit theories (Hypothesis 2b). As expected and in line with theoretical considerations as well as empirical evidence (Lau & Lee, 2008; Lüftenegger et al., 2014; Tapola & Niemivirta, 2008), TARGET goal structures turned out to be positively related to students personal mastery goals. This finding underscores the suitability of the developed scales for assessing a mastery goal structure. The positive relationship to implicit theories corroborates our research hypotheses and allows us to draw some initial conclusions about the interplay between TARGET goal structures and students adoption of an incremental theory of intelligence. Regarding the relationship between TARGET goal structures and the four types of students personal performance goals (Hypothesis 2c),it was shown that TARGETgoal structures are positively related to students performance approach goals with an appearance focus and, to a lesser extent, to students performance approach goals with a Zeitschrift für Psychologie (2017), 225(1), Ó 2017 Hogrefe Publishing

10 M. Lüftenegger et al., Classroom Goal Structure Questionnaire 73 normative focus. No significant relations between TARGET goal structures and either type of performance avoidance goals were found. All in all, the results for Hypotheses 2a 2c substantiate the external validity of the measurement instrument. Given the inconsistent findings on the relation between mastery goal structures and students personal performance goals (e. g. Lau & Lee, 2008; Urdan, 2004; Wolters, 2004) this study s findings reinforce the premise that a more comprehensive conceptualization of mastery goal structures as provided here may be especially appropriate for investigating and therefore enhancing our understanding of this relation. An important finding is that even though personal performance approach and avoidance goals are highly positively related (which is in line with broad empirical evidence; see Linnenbrink-Garcia et al., 2012), our results suggest that TARGET mastery-focused classrooms do not endorse both performance goals equally. Keeping in mind the negative consequences of pursuing performance avoidance goals, the finding that a TARGET mastery goal structure is not related to students adoption of performance avoidance goals has important implications for classroom practice. Furthermore, the positive relation of TARGET mastery goal structures with mastery goals and (both types of) performance approach goals can be interpreted in terms of a multiple goal perspective (Wormington & Linnenbrink-Garcia, 2016). Mastery goals are still considered the most beneficial goals, and their promotion should therefore represent a major aim in class (Wormington & Linnenbrink-Garcia, 2016). However, the simultaneous pursuit of a performance approach goal may also be adaptive for specific outcomes (Senko et al., 2011). The magnitude of the relations between the TARGET mastery goal structure and students personal goals found in this study (mastery > performance approach appearance > performance approach normative) reflects these assumptions and further emphasizes the suitability of the newly developed GSQ for research and classroom practice. Limitations and Implications for Future Research Some limitations of the present study should be noted. It has to be mentioned that our conclusions regarding the external validity of this study are limited by the correlational study design and do not allow us to interpret relations between TARGET goal structures, achievement goals, and implicit theories in causal ways. Additionally, due to the cross-sectional design it was not possible to examine the stability of the GSQ. Longitudinal studies could bring insights into the impact of TARGET goal structures on the development trajectories of achievement goals and incremental theories. Our reliance on students self-reports can be seen as another limitation. The use of multiple methods such as interviews (see e.g., Urdan, 2004)orclassroom observations (see e.g., Patrick, Anderman, Ryan, Edelin, & Midgley, 2001) is therefore highly recommended to expand our understanding of the TARGET goal structures. Moreover, additional research on TARGET should obtain information from multiple informants (teachers and students), with the concordance between students perceptions of classroom structures and teachers perceptions of their instructional behavior of particular interest (Urdan, Midgley, & Anderman, 1998). Furthermore, it is an open question whether our findings can be generalized to other populations (e.g., gifted students, students of different age groups like elementary school students). More research is also needed to investigate whether the results of this study can be replicated in different countries and cultures. Another interesting avenue for future research would involve the deployment of the TARGET questionnaire in a completely different context, for example, sport psychology. One example of a research question might be how various aspects of instructional structures (measured by the TARGET questionnaire) influence the motivational development of adolescents in competitive sports. Methodologically, the modest reliability of the dimension Task has to be acknowledged. Further studies could address this limitation by developing and psychometrically testing additional items for the Task dimension. The high loading of especially Recognition on the TARGET higher-order factor has two implications: First, Recognition is apparently the most salient indicator of the overall TARGET goal structure, in the sense that the higher-order factor is effectively identical to this specific lower-order indicator. Second, with regard to measurement, assessment of the Recognition factor provides at the same time a reliable proxy measure of the overall higher-order factor. This may be of use in future applied research. Additionally, the TARGET questionnaire should be further validated by fully considering the inherent multilevel structure of goal structure using doubly latent modeling (latent in relation to both measurement and sampling error). However, doubly latent models are very complex and require substantial ICCs and large sample sizes on both the individual and class level (Lüdtke et al., 2011). Future research should also focus on further broadening the nomological network of mastery goal structures by investigating the effects on other motivational constructs such as academic interest, self-efficacy or self-concept, and how this nomological network (goal structure? personal motivation) is connected to school performance. In conclusion, despite their limitations, the findings of the present study indicate that the newly developed scales are internally valid as demonstrated by confirmatory factor analysis, externally valid as demonstrated in terms of relationships with achievement goals, and reliable. The GSQ can therefore be considered the first comprehensive and psychometrically sound instrument assessing students perception of TARGET goal structures. Ó 2017 Hogrefe Publishing Zeitschrift für Psychologie (2017), 225(1), 64 75

11 74 M. Lüftenegger et al., Classroom Goal Structure Questionnaire Electronic Supplementary Material The electronic supplementary material is available with the online version of the article at /a ESM 1. Text (.docx). Items of the final questionnaire. References Ames, C. (1992). Classrooms: Goals, structures, and student motivation. Journal of Educational Psychology, 84, doi: / Bergsmann, E., Lüftenegger, M., Jöstl, G., Schober, B., & Spiel, C. (2013). The role of classroom structure in fostering students school functioning: A comprehensive and application-oriented approach. Learning and Individual Differences, 26, doi: /j.lindif Bong, M. (2009). Age-related differences in achievement goal differentiation. Journal of Educational Psychology, 101, doi: /a Church, M. A., Eliott, A. J., & Gable, S. L. (2001). Perceptions of classroom environment, achievement goals, and achievement outcomes. Journal of Educational Psychology, 93, doi: / Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum. Dewey, J. (1938). Experience and education. West Lafayette, IN: Kappa Delta Pi. Dweck, C. S., & Leggett, E. L. (1988). A social-cognitive approach to motivation and personality. Psychological Review, 95, Elliot, A. J. (2005). A conceptual history of the achievement goal construct. In A. J. Elliot & C. S. Dweck (Eds.), Handbook of competence and motivation (pp ). New York, NY: Guilford Press. Elliot, A. J., & Murayama, K. (2008). On the measurement of achievement goals: Critique, illustration, and application. Journal of Educational Psychology, 100, doi: / Epstein, J. L. (1988). Effective schools or effective students: Dealing with diversity. In R. Haskins & D. MacRae (Eds.), Policies for America s public schools: Teacher, equity and indicators (pp ). Norwood, NJ: Ablex. Federici, R. A., Skaalvik, E. M., & Tangen, T. N. (2015). Students perceptions of the goal structure in mathematics classrooms: Relations with goal orientations, mathematics anxiety, and help-seeking behavior. International Education Studies, 8, doi: /ies.v8n3p146 Greene, B. A., Miller, R. B., Crowson, M. H., Duke, B. L., & Akey, K. L. (2004). Predicting high school students cognitive engagement and achievement: Contributions of classroom perceptions and motivation. Contemporary Educational Psychology, 29, doi: /j.cedpsych Hackel, T. S., Jones, M. H., Carbonneau, K. J., & Mueller, C. E. (2016). Re-examining achievement goal instrumentation: Convergent validity of AGQ and PALS. Contemporary Educational Psychology, 46, doi: /j.cedpsych Hulleman, C. S., Schrager, S. M., Bodmann, S. M., & Harackiewicz, J. M. (2010). A meta-analytic review of achievement goal measures: Different labels for the same constructs or different constructs with similar labels? Psychological Bulletin, 136, doi: /a Kaplan, A., & Maehr, M. L. (1999). Achievement goals and student well-being. Contemporary Educational Psychology, 24, doi: /ceps Kaplan, A., & Maehr, M. L. (2007). The contributions and prospects of goal orientation theory. Educational Psychology Review, 19, doi: /s Kline, R. B. (2011). Principles and practice of structural equation modeling (3rd ed.). New York, NY: Guilford Press. Lau, K. L., & Lee, J. (2008). Examining Hong Kong students achievement goals and their relations with students perceived classroom environment and strategy use. Educational Psychology, 28, doi: / Lau, S., & Nie, Y. (2008). Interplay between personal goals and classroom goal structures in predicting student outcomes: A multilevel analysis of person-context interactions. Journal of Educational Psychology, 100, doi: / Lee, M., & Bong, M. (2016). In their own words: Reasons underlying the achievement striving of students in schools. Journal of Educational Psychology, 108, doi: / edu Linnenbrink-Garcia, L., Middleton, M. J., Ciani, K. D., Easter, M. A., O Keefe, P. A., & Zusho, A. (2012). The strength of the relation between performance approach and performance avoidance goal orientations: Theoretical, methodological, and instructional implications. Educational Psychologist, 47, doi: / Lüdtke, O., Marsh, H. W., Robitzsch, A., & Trautwein, U. (2011). A 2 2 taxonomy of multilevel latent contextual models: Accuracy-bias trade-offs in full and partial error correction models. Psychological Methods, 16, doi: /a Lüdtke, O., Robitzsch, A., Trautwein, U., & Kunter, M. (2009). Assessing the impact of learning environments: How to use student ratings of classroom or school characteristics in multilevel modeling. Contemporary Educational Psychology, 34, doi: /j.cedpsych Lüftenegger, M., van de Schoot, R., Schober, B., Finsterwald, M., & Spiel, C. (2014). Promotion of students mastery goal orientations: Does TARGET work? Educational Psychology, 34, doi: / Maehr, M. L. (1989). Thoughts about motivation. In C. Ames & R. Ames (Eds.), Research on motivation in education (Vol. 3, pp ). New York, NY: Academic Press. Maehr, M. L., & Midgley, C. (1996). Transforming school cultures. Boulder, CO: Westview Press. Meece, J. L., Anderman, E. M., & Anderman, L. H. (2006). Classroom goal structure, student motivation, and academic achievement. Annual Review of Psychology, 57, doi: /annurev.psych Midgley, C., Maehr, M. L., Hruda, L. Z., Anderman, E., Anderman, L., Freeman, K. E.,...Urdan, T. (2000). Manual for the Patterns of Adaptive Learning Scales (PALS). Ann Arbor, MI: University of Michigan. Midgley, C., & Urdan, T. (2001). Academic self-handicapping and achievement goals: A further examination. Contemporary Educational Psychology, 26, doi: /ceps Murayama, K., & Elliot, A. J. (2009). The joint influence of personal achievement goals and classroom goal structures on achievement-relevant outcomes. Journal of Educational Psychology, 101, doi: /a Muthén, B. O., & Muthén, L. K. ( ). Mplus (Version 7. 4). Los Angeles, CA: Muthén & Muthén. Nicholls, J. G. (1984). Achievement motivation: Conceptions of ability, subjective experience, task choice, and performance. Psychological Review, 91, doi: / X Zeitschrift für Psychologie (2017), 225(1), Ó 2017 Hogrefe Publishing