Influencing young adolescents motivation in the lowest level of secondary education

Transcription

1 See discussions, stats, and author profiles for this publication at: Influencing young adolescents motivation in the lowest level of secondary education Article in Educational Review January 2015 DOI: / CITATIONS 11 READS authors: Thea Peetsma University of Amsterdam 96 PUBLICATIONS 1,504 CITATIONS SEE PROFILE Ineke Van der Veen University of Amsterdam 74 PUBLICATIONS 826 CITATIONS SEE PROFILE Some of the authors of this publication are also working on these related projects: Motivation of university students in the Netherlands View project All content following this page was uploaded by Thea Peetsma on 17 December The user has requested enhancement of the downloaded file.

2 This article was downloaded by: [UVA Universiteitsbibliotheek SZ] On: 28 November 2014, At: 00:30 Publisher: Routledge Informa Ltd Registered in England and Wales Registered Number: Registered office: Mortimer House, Mortimer Street, London W1T 3JH, UK Educational Review Publication details, including instructions for authors and subscription information: Influencing young adolescents motivation in the lowest level of secondary education Thea Peetsma a & Ineke Van der Veen a a Faculty of Social and Behavioural Sciences, University of Amsterdam, Amsterdam, The Netherlands Published online: 24 Sep To cite this article: Thea Peetsma & Ineke Van der Veen (2015) Influencing young adolescents motivation in the lowest level of secondary education, Educational Review, 67:1, , DOI: / To link to this article: PLEASE SCROLL DOWN FOR ARTICLE Taylor & Francis makes every effort to ensure the accuracy of all the information (the Content ) contained in the publications on our platform. However, Taylor & Francis, our agents, and our licensors make no representations or warranties whatsoever as to the accuracy, completeness, or suitability for any purpose of the Content. Any opinions and views expressed in this publication are the opinions and views of the authors, and are not the views of or endorsed by Taylor & Francis. The accuracy of the Content should not be relied upon and should be independently verified with primary sources of information. Taylor and Francis shall not be liable for any losses, actions, claims, proceedings, demands, costs, expenses, damages, and other liabilities whatsoever or howsoever caused arising directly or indirectly in connection with, in relation to or arising out of the use of the Content. This article may be used for research, teaching, and private study purposes. Any substantial or systematic reproduction, redistribution, reselling, loan, sub-licensing, systematic supply, or distribution in any form to anyone is expressly forbidden. Terms & Conditions of access and use can be found at

3 Educational Review, 2015 Vol. 67, No. 1, , Influencing young adolescents motivation in the lowest level of secondary education Thea Peetsma* and Ineke Van der Veen Faculty of Social and Behavioural Sciences, University of Amsterdam, Amsterdam, The Netherlands After the transition to secondary school around age 12, a well-known decline in young adolescents motivation for learning causes particular concerns in the lowest level of secondary education, where the percentage of early school leavers is highest. This article focuses on a study of the effects of an intervention, designed to enhance motivation of students who are at the start of the lowest level of secondary school. An earlier intervention was further developed and tested with a random selection of students from classes in pre-vocational education by varying the components of the interventions in order to find possible effects of specific components or of the order of the two assignments of the intervention. Results showed positive effects of the intervention on students investment in schoolwork. Keywords: intervention; future time perspective; vocational education; motivation 1. Introduction The effort that young adolescents make at school decreases over their secondary school career in all types of school and many countries, as this has for instance been found in studies in the USA (Midgley, Feldlaufer, and Eccles 1989) and in the Czech Republic, Germany, The Netherlands and Switzerland (Peetsma et al. 2005) and again in The Netherlands (Van der Veen and Peetsma 2009). Educators have been concerned for years about this common phenomenon. Their focus was particularly on the transition from primary to secondary school, a period in which children have just become adolescents with all the uncertainties and developmental requirements belonging to this life phase. In The Netherlands, but also in other countries, the transition phase implies that children have to go to another school, usually much bigger and further from home. At the age of 12, after finishing primary school, in The Netherlands, children receive an advice for the level of secondary education they can attend. This advice is based on the judgement of the teacher(s) and usually also on the child s score on a national test. Students can attend three different types of secondary education of which the academic level is increasing from a four-year pre-vocational education preparing for intermediate vocational education, a five-year higher general secondary education preparing for higher professional education to a six-year pre-university education preparing for university. There is little transfer between the levels of secondary education which makes students school careers mainly decided at a young age. *Corresponding author. t.t.d.peetsma@uva.nl 2013 Educational Review

4 98 T. Peetsma and I. Van der Veen Dutch pre-vocational secondary schools have relatively more students with learning difficulties and behavioural problems and their students usually come from amongst the lowest achieving students at primary school, which is why their primary schools recommended them for the less academic type of secondary education. Loss of motivation in pre-vocational secondary education is more likely to lead to dropout, because there are no other options to transfer. Indeed, three times as many students in pre-vocational secondary education left school early without qualifications than students in the other types of secondary school (Dutch Inspectorate of Education 2005). The decrease in motivation has led, not only in The Netherlands, to attempts to improve students motivation in order to improve their chances of a successful school career. As the concept future time perspective proves to be a good predictor of students motivation (Husman and Lens 1999; Zimbardo and Boyd 1999), whereas students in pre-vocational secondary schools in particular appear to have less positive longer term perspectives on school and professional career (Peetsma 1992), time perspectives theory has been taken as the starting point for an intervention to enhance these students motivation (Peetsma and Van der Veen 2009). As this intervention produced promising results in the previous study, the current study was used to obtain more information about how the intervention works with a view to applying it more widely in schools. This article reports on the results of this research. 2. Theoretical background and research objectives 2.1. Motivation and self-regulation in learning Zimmerman (2000) underlined the cyclical nature of self-regulation in learning, assuming that students use feedback from prior learning experiences to make adjustments for subsequent efforts. One aspect of self-regulation is the use of metacognition (knowledge about knowledge), but it can also involve different aspects of effort to learn (initiative, intensity and perseverance at school) (Maehr and Braskamp 1986; Roede 1989) and students postponement opportunities to satisfy impulses immediately and to pursue long term academic goals, delay of gratification (Bembenutty and Karabenick 2004). These indicators of learning behaviour are influenced by motivational beliefs or motivators (Van der Veen and Peetsma 2009), which are quite varied and complex. Broadly speaking there are three categories of constructs that are relevant to students motivation in educational settings: values/ goals, expectations, and affective components (Peetsma et al. 2005). The values/ goals component concerns students goals and intentions when performing a task. It includes their aims and beliefs about the importance or usefulness of a learning task. Various distinctions are made with regard to this values/goals component. A wellknown distinction is between learning and performance approach goals (see Elliot and Church 1997; Schunk 1996). The latter is concerned with comparing one s performance with that of others and with how one s performance is judged by others, whereas the former is concerned with learning in order to improve on one s own earlier achievement. It was assumed that a performance approach goal leads to less good outcomes (see for instance Pintrich and Schunk 1996). However, when performance goals are linked with learning goals this can lead to equally good outcomes (Pintrich 2000) or even better outcomes than learning goals alone (Van der Veen and

5 Educational Review 99 Peetsma 2009). The expectations component includes students belief that they are able to complete a task (including their self-efficacy and control beliefs). Bandura (1993) claims that self-efficacy is the guiding force behind a person s thought and action. He found a positive association between self-efficacy and motivation and achievement (Bandura 1997). Based on a review of the literature on self-efficacy, Zimmerman (2000) concluded that it should be conceived as a distinct concept that predicts various aspects of performance. The affective component concerns students feelings or emotional reactions to a learning task or the learning environment. Students well-being in the class is an example of this. With regard to the values/goals component of motivational factors, a distinction is also made between intrinsic motivation and extrinsic motivation. Deci and Ryan s (1985) self-determination theory distinguishes these different types of motivation based on the different reasons or purposes that lead to action. Intrinsic motivation indicates that something is being done because of inherent interest or inherent pleasure in the action or performance of the task. Extrinsic motivation indicates that something is being done with a view to an outcome as distinct from the action itself, such as an expected reward for performing the task. Ryan and Deci (2000) introduced more distinctions into the category of extrinsic motivation. They distinguished four stages of self-regulation in extrinsic motivation, depending on the level of external regulation or self-determination. External regulation means that behaviour is controlled by factors or circumstances that are clearly outside the individual (such as reward or punishment). Introjected regulation means that behaviour is motivated by a pressure with which the individual cannot really identify. The behaviour is still externally regulated, therefore, and there is no question of self-determination. Identified regulation means that the individual does not behave in a certain way because he or she feels obliged to do so, but because he or she recognizes that it is a step or means toward a personal goal. The person identifies with the value(s) underpinning the goal and the action that should lead to the goal. In these circumstances the behaviour or action is purely instrumental and is therefore extrinsically motivated. The behaviour is instrumental for a purpose that the person has chosen for him or herself (self-determination) and is actually internally regulated therefore. Integrated regulation is the most autonomous and self-determined form of extrinsic motivation. In the case of integrated regulation, individuals no longer experience any conflict or disinclination as they regulate their conduct on the basis of rules, values and goals with which they have already identified at an earlier stage. As with intrinsic motivation, individuals determine their own behaviour. In the case of intrinsically motivated behaviour, the person is pursuing a goal for its own sake. The behaviour is instrumental in the case of integrated regulation. Both types of conduct are internally regulated and controlled by the individuals themselves. A piece of research aimed to add to the theory of Deci and Ryan showed that activities that generate extrinsic motivation can be highly motivational (Simons, Dewitte, and Lens 2000). This leads to the expectation that even when studying is extrinsically motivated, where this motivation is combined with integrated regulation the quality of the learning process can be just as good as with intrinsically motivated learning. Students who lack intrinsic motivation to perform a task, can still set to work seriously on performing the task because of integrated regulation of activities with a view to their future.

6 100 T. Peetsma and I. Van der Veen 2.2. Future time perspective As schooling contains utility value to attain future goals, it can by definition be considered future-oriented (Eccles and Wigfield 2002). Future time perspective is generally described as a representation of a particular life domain at a time in the future or the anticipation in the present of future events (Nuttin and Lens 1985). The concept is seen as a motivator for learning and can be regarded as belonging to the values/goals component. Future time perspective is characterized by extension and valence (see for instance Gjesme 1996; Husman and Lens 1998). Extension refers to the period in the future in which an event or life domain is represented in someone s perspective, for students both the time after leaving this school and the current school year seem to be meaningful terms in time. The valence of the time perspective indicates a person s appreciation of a certain life domain in the future. This is essential for the role of the concept as a motivational variable. Simons, Dewitte, and Lens (2000) describe the future time perspective in terms of instrumentality and distinguish three types of instrumentality. In the first type (E-E), present activities are extrinsically motivated with a view to the future task and present activities are externally regulated. An example is a student who puts a lot of effort into learning for a test, because he/she wants to earn a lot of money in the future. He/she does not enjoy learning but wants to avoid being poor later. In the second type (E-I), present activities are extrinsically motivated and the future goal regulates the present activities internally. An example is a student who puts a lot of effort into learning for a test, because he/she wants to make progress in his/her personal development. In this case it is not the learning itself that motivates, but the goal of personal development. In the third type (I-I), the future goal is an intrinsic motivator and it regulates the present activities internally. An example is a student who puts a lot of effort into learning for a test, because he/she wants to learn strategies that he/she will find useful later. Students of the first type (E-E) appear to have less motivation to perform tasks than students of the other two types. Simons, Dewitte, and Lens (2000) argue that students motivation can be enhanced if E-E is changed into E-I, which can be achieved by highlighting the personal relevance of the extrinsic motives. Peetsma (1992, 2000) included an affective component in the concept of time perspective. Time perspective is conceptualized here with three components (i.e., affect, cognition, and behavioural intention) directed at a life domain (such as school career or leisure time) over a period of time. By combining affect, cognition, and behavioural intentions, time perspective represents students internalization of and determination to reach valued goals in the present or in the future. In a recent study, combining future time perspective with the self-determination theory of Deci and Ryan (2000) De Bilde, Vansteenkiste, and Lens (2011) found indeed future time perspective to be positively associated with internal regulation. Time perspective in a school and professional career, conceptualized including an affective component, proved to be a better predictor of students school investment than perceived instrumentality operationalized as perceived usefulness and necessity of school for the future (Peetsma 1992). It is interesting to note that children only start to develop a future time perspective in their young adolescence from the age of 11 to 12 (Piaget 1955). Younger children s ideas about the future are more like fantasies. The future time perspective

7 Educational Review 101 develops into a permanent feature between the ages of 12 and 18, when fantasy is suppressed in favour of an awareness of reality (Klineberg 1967). This means that young adolescence is precisely the age when the future time perspective starts to have a significant influence on them. If it were possible to influence a decreasing student perspective on school, this could be expected to have a positive effect on motivation for school Intervention in previous and current study In a previous study (Peetsma and Van der Veen 2009), an intervention was developed aiming to make adolescents aware that what they do at school and the choices they make will have consequences for their opportunities in the future. Being important with regard to self-regulated learning, time perspectives, internally controlled motivation and academic delay of gratification, were included in the intervention with a view to improving adolescents motivation for school. We used, for brevity, the word motivation for both motivators and for motivated learning behaviour. The intervention was based on the future time perspective concept including affective, cognitive, and intentional components with regard to a life domain, such as school and professional career, in the short or long term. If students are not internally but externally controlled motivated, attention to the personal gains that students can get from learning and the correlation between behaviour now and opportunities later, changing E-E type students into E-I type students, is expected to lead to strengthening students motivation. The intervention also incorporated motivation for potentially competing life domains, such as maintaining friendships and how the adolescents spend their free time. The idea of acceptance of delay of gratification was used for this. In the design of the intervention possible selves were used, whereby students had to visualize possible futures for themselves. Possible selves can serve to guide and regulate behaviour, providing a roadmap connecting the present to the future (Oyserman et al. 2004). The steps that connect the present to the future can serve as a kind of guide for the student. In an intervention study (Oyserman, Terry, and Bybee 2002) four characteristics seemed to be essential for its success. Firstly, the students need to have realistic ideas about future possibilities (possible selves). Secondly, there has to be an emphasis not only on what they should do to reach a goal but also on what they should avoid doing. Thirdly, the path to the future goal should be broken down into small manageable steps. Last but not least the students should attach importance to the possible futures being held out before them. Finally, the intervention had to be designed so that it could be applied in the school setting and so an intervention was designed that could be used within the space of one school lesson. As well, was wanted to avoid the intervention having any negative effects as far as possible. That is why a largely in-direct intervention was chosen. One assignment of the intervention used an imagined future in sport and the other an imagined future in music. This was translated to the school career through a virtual classmate in the first instance. Only at the end, were the students asked what this could mean for them, the intervention was direct Development of the intervention: the previous study During the development of the intervention in a previous study (Peetsma and Van der Veen 2009) an attempt was made to improve the perspective on school career

8 102 T. Peetsma and I. Van der Veen and professional career, and therefore also the motivation, of 30 students by intervening twice during their first year at school. A sample of students who appeared to have poorer perspectives at the start of the school year was selected. To investigate the effect of the intervention, 25 other students were also interviewed on two occasions about their motivation for school and questionnaires on motivation were administered to a large group of students (765) on four occasions. It was expected that the intervention would enhance students learning behaviour. It was also expected that the intervention would boost motivators, like students values, expectations, affections, and time perspectives towards school. Differences in the expected directions were found between the students who had taken part in the interventions and the others who had either filled in questionnaires about motivation or been interviewed about their motivation for school. Especially, girls and students from an ethnic minority background benefited most from the interventions. The findings of the interventions were promising, as even after a period of four months had passed since the last intervention, effects were still being found on the effort students were making at school. Possibly, the effects of the intervention would have been less in case of a general school population instead of students with poorer time perspectives on a school and professional career Objectives of the further development of the intervention in the current study As the constant aim of the developers has been to design an intervention that could be used by school staff, more research was needed to justify its wider application. In this study the intervention was not used on students with less positive future perspectives but on a random sample of young students. Also, it was aimed to produce more in-depth information about the effect of the interventions. So it was performed on a larger group of students and the sequence and a number of components of the intervention were varied. We had a control group of students whose motivation was investigated and another group who were asked to focus on this in an interview only in order to test the effect of the intervention rigorously. The main focus of this second intervention study was on a potential positive effect on students learning behaviour, but effects of the intervention on different motivators, namely, values/ goals, expectations, affections, and time perspectives towards school, were also investigated. 3. Method 3.1. The research group A total of 224 students from two pre-vocational secondary schools in two cities close to the capital filled in a questionnaire on four occasions: immediately after the start of the first lesson in September 2007, in October 2007, November 2007 and in January Of the 224 students 79% (n=176) completed the questionnaire on all four occasions, 17% (n=38) did it three times, 4% (n=8) twice and 1% (n=2) completed the questionnaire once. All their students who were starting their first year at the schools participated. Forty students (about five per class) from both schools were randomly selected out of the 224 for the intervention. They took part in the intervention immediately before the second and third measurements using the questionnaires. Sixteen other

9 Educational Review 103 Table 1. Background characteristics of the three groups of students in the study. Column % Intervention Interview other Age at measurement Gender Boy Girl Ethnic background Netherlands Mixed Surinam Turkey Morocco Other Highest education of Primary education Parents Lower secondary vocational education Senior secondary vocational education Higher education Number students (about two per class), also from both schools, were interviewed about their motivation for school immediately before the second and third questionnaire measurements. These 16 students were the first control group. Comparing the experimental group (the group that took part in the intervention) with this group of students allowed us to investigate how far simply focusing attention on motivational aspects had a positive effect. If, for instance, we found that both the intervention and the interview had a positive effect on motivation, it would permit the conclusion that it was simply giving attention to students motivation and not the intervention itself that was successful. The second control group consisted of the other 168 (224 minus 40 minus 16) students. Table 1 shows background characteristics of the three groups of students. Our analysis found no significant differences in background characteristics between the three groups. Most of the students were aged 12 to 13 at the time of the first measurement. More than half of the parents of the students had been educated to intermediate vocational education or higher. There was a higher proportion of students with a migrant background than in the general population because they came from schools in the neighbourhood of a big city The instruments Three instruments were used: a questionnaire with closed questions, an interview guide with open questions and the intervention The questionnaire A questionnaire with pre-coded questions was administered to all the students. The questions consisted of statements to which the students could answer on a Likert

10 104 T. Peetsma and I. Van der Veen scale of 1 to 5. The questionnaire consisted of 14 well-known scales: four on self-regulated learning behaviour and 10 on self-efficacy, well-being, goal-orientation, and time perspective. Table 2 shows more information on the scales. For some of the scales, it was decided to focus the statements on a specific subject, namely, mathematics. This was done to allow for the known phenomenon that it does make a difference to students motivation which subject or field they are being asked Table 2. Scales, example items, number of items, and internal consistency for the scales in this study (per scale minimum score = 1, maximum score = 5). Scale Origin Example item Academic delay of gratification Bembenutty and Karabenick 1998 I do my homework before I meet with my friends Number of items α m1 α m2 α m3 α m Effort at school Roede 1989 I am bored at school Effort at maths Roede 1989 I put a lot of effort into maths Metacognitive strategies Self-efficacy Well-being with teachers Well-being with classmates Well-being at school Mastery approach Performance approach Short-term future time pers. on school and professional career Pintrich and De Groot 1990 Midgley et al Peetsma, Wagenaar, and De Kat 2001 Peetsma, Wagenaar, and De Kat 2001 Peetsma, Wagenaar, and De Kat 2001 Seegers, van Putten, and de Brabander 2002 Seegers, van Putten, and de Brabander 2002 Peetsma 1992 I ask myself questions to make sure I know the material I have been studying Even if the work is hard, I can learn it I can talk about personal problems with my teachers I get along well with my classmates At school I feel at home I feel satisfied when I have learned something in maths that makes sense to me I enjoy getting a better grade in maths than my classmates I am pleased that I will learn a lot of new things this year (Continued )

11 Educational Review 105 Table 2. (Continued ). Scale Origin Example item Long-term future time pers. on study and professional career Short-term future time pers. on leisure Long-term future time pers. on leisure Peetsma 1992 Peetsma 1992 Peetsma 1992 It does not matter too much to me what I study, or what kind of work I will be able to do after finishing school I enjoy my leisure time and holidays very much now In future, leisure will mean a lot to me Number of items about (Bong 2001). Apart from that, research has found that the relationships between school subjects are strong. We chose mathematics because it is an important subject that all students in the first year of pre-vocational secondary school have to take. A factor analysis and a reliability analysis was performed on each scale. We found that two items on the effort at school and well-being at school scales did not fit the scales well enough and there was no fit for one item on the academic delay of gratification and use of metacognitive strategies scales. These items were therefore taken out of the scales The interview The 16 students selected for interview were asked about each motivation aspect in the questionnaire on two occasions. They were asked, for example, about their wellbeing at school, but also about their expectations about work and school in the future. The second interview emphasized changes since the previous interview, but all of the aspects of motivation were dealt with in this interview too. The interviews took about minutes and took place at the same time as the interventions, and was, like the interventions, face-to-face The intervention Each intervention took a total of minutes, was applied to each student twice and consisted of three parts. Each intervention involved one interviewer and one student. The first in-direct part of the intervention consisted of a separate role play. In the in-direct second part, the student was asked to imagine a classmate with certain characteristics and aims and to give that person advice. In the in-direct third part, the student was asked how far he/she recognized him/herself in that classmate. Two role plays were developed: one on the subject of sport, specifically high jumping, and one on the subject of playing in a band. These subjects were chosen because students are asked to imagine other students aspiring to be good at sports or music, even if they themselves are not fond of sports or music. Descriptions of two types α m1 α m2 α m3 α m

12 106 T. Peetsma and I. Van der Veen of classmate were also used. The sequence of the assignments or role plays was varied systematically in the research. This means that there were four variants: (1) intervention 1: (a) high jumping assignment, (b) classmate type 1; intervention 2: (a) playing in a band assignment, (b) classmate type 2 (2) intervention 1: (a) high jumping assignment, (b) classmate type 2; intervention 2: (a) playing in a band assignment, (b) classmate type 1 (3) intervention 1: (a) playing in a band assignment, (b) classmate type 1; intervention 2: (a) high jumping assignment, (b) classmate type 2 (4) intervention 1: (a) playing in a band assignment, (b) classmate type 2; intervention 2: (a) high jumping assignment, (b) classmate type 1 We will now describe the intervention in more detail, using the high jumping assignment to illustrate part 1 of the intervention. This assignment was similar to the other assignment, which was about playing in a band and being selected to play at a music festival. The students were asked to read a short text on the high jumping assignment. Then they were asked to imagine taking part in this sport and that their goals were to come through the selection at the end of the year to take part in a competition in two years time and then to do well at that competition. Each student was asked to imagine: being good at high jumping, even though practice is needed to be able to jump higher (cognition); liking high jumping, dreaming about it a lot and getting a good feeling (affection); developing a training programme with the trainer including the steps towards the final athletic competition (behavioural intentions); not missing any training sessions, not only because this means missing necessary practice, but also because otherwise the trainer will assume that the student is not taking high jumping seriously and may not want to train him/her any more (behavioural intentions); dealing with distractions by giving high jumping priority (delay of gratification). The text was summarized for the student. Then the student was asked (and given time) to imagine him/herself in that situation with those goals. The student was encouraged to contribute ideas of his/her own that did not come out of the text. The student was interviewed and the interviewer made sure that the following aspects came up: affect (feelings about the goals and the activity); cognition (knowledge and opinions about the goals to be reached); behavioural intentions (including small and big steps toward the goals to be reached); and distraction in the areas of affect, cognition, and behaviour (what to do about distractions like bad weather, frustration during training, an unsympathetic training partner, among others). In part 2 of the intervention, the students were asked to imagine a classmate who really wants to progress to the next year and get the diploma. In this assignment two types of classmate were presented alternately. One classmate was a good enough learner to achieve this but sometimes felt unsure about his/her ability. The other classmate had the same aims and was also a good enough learner, but often

13 Educational Review 107 made no effort at school, assuming that success would come naturally without trying. This classmate often fails tests. The students were asked what they would advise their classmate to do to reach the goals (going up to the next year and getting the diploma), bearing in mind the things that had been discussed in the role play in the first part of the intervention. Once again the interviewer made sure that the four aspects (affect, cognition, behavioural intentions, distractions) came up in the interview. In part 3 of the intervention the students were asked whether they recognized themselves in the imaginary classmate and why or why not. The school and possible professional career of the students themselves were discussed. Again, the interviewer made sure that the four aspects came up in the conversation. The interviewer encouraged and exerted an influence on the student by, for instance, giving compliments and suggesting alternatives. For instance, when a student had problems with a classmate or with a teacher, we suggested he/she could go to the mentor or seek for other help instead of having these troubles continue. During the three parts the interviewer checked if all aspects were included in every part of the intervention with use of an interview guideline. This was done in order to secure that students dealt with all necessary aspects Design of the analysis The development in motivation of the three groups of students (the 40 students who took part in the intervention, the 16 students who were interviewed and the other 168 students) were compared using latent growth curve analysis (LGCA). The advantage of this analysis technique is that the pattern of development of each student can be shown in the form of a unique curve. It allows for the fact that students had different degrees of motivation at the start when the first measurement was taken and that individual students vary in the extent to which their motivation increases or decreases. The level of a variable at the first measurement (the intercept of the line) and the measure of growth in a variable (the slope of the line) are treated as latent variables. This type of analysis can be performed using programmes designed for analysing structural models; we used Mplus (Muthén and Muthén 2004). The advantage of modelling the intercept and the curve as latent variables is that these variables can be correlated with each other and with other latent or control variables. A negative correlation between the start level and the degree of growth means, for example, that a lower start measurement corresponds with a greater degree of growth and vice versa. Another example is the effect of gender on the latent variables. This indicates the amount of difference between boys and girls at the start and in the degree of growth. Listwise deletion is a technique that is often used in research to deal with missing data: students for whom one or more items of data are missing are deleted from the analysis. If, for instance, a student had not been able to fill in the questionnaire at one of the four measurements, he or she would have been taken out of the analysis. In this study, this approach would have resulted in the data from 21% (100 minus 79%) of the 224 students not being included in the analysis. Listwise deletion as a method has been strongly criticized (Little and Rubin 1989). In our research all the students were included in the analyses. Missing data were estimated using fullinformation maximum likelihood estimation (FIML). FIML is based on the assumption that missing values are missing at random, which means that missing

14 108 T. Peetsma and I. Van der Veen values can be predicted from the data that is available. The removal of students with missing values (listwise deletion) is based on the stricter assumption that the missing data are completely at random. To investigate the degree of effectiveness of the intervention, two analyses were performed. A multi-group latent growth model analysis was performed twice on each variable in the questionnaire (see Table 2). In the first multi-group analysis, a growth model was estimated for each variable using the two latent variables (score at measurement 1 and score for growth) and using gender and ethnic origin (ethnic minority yes/no) as control variables. These variables, gender and ethnic minority, were used as control variables because of differences found in the first intervention study. The ethnic minority variable was not broken down by country of origin, because the number of students in the different groups would then have been too small for the analyses. We investigated whether there were any differences in the start score and the degree of growth between the intervention group and the group of students who were neither interviewed nor took part in the intervention, and whether the degree of growth was different for ethnic minority and Dutch students and for boys and girls (interaction effects of degree of growth with gender and ethnic origin). As the number of students in the interview group was quite small, namely, 16, it was not possible to investigate interaction effects of degree of growth with gender and ethnic origin for this group. For the comparison with the interview group, therefore, a second multi-group latent growth model analysis was performed, which compared the intervention group with the interview group. Whether these two groups had different scores on the variable at measurement 1 and different degrees of growth was investigated. To find out whether the intervention was having an effect, the most important thing to find out was whether the degree of growth varied between the groups. If, for instance, the first multi-group analysis were to find that the amount of effort put into school work by students in the intervention group increased more than that put in by the students in the control group who did not take part in the intervention and were also not interviewed, then this would be an indication that the intervention was effective. If, however, the second analysis were to find that there was no difference between the intervention group and the interview group in the increase in effort, this would indicate that it was the personal conversation with the student about motivation that was the main factor in the positive effect and not the intervention itself. With regard to the score at measurement 1, we expected to find few differences, because the students had been selected for the groups at random, and the first measurement took place before the intervention. To evaluate the size of significant differences between groups from measurement 1 to 4, effect sizes for differences in intercepts and slopes were calculated by means of Cohen s d, with 0.2 being indicative of a small effect, 0.5 of a medium, and 0.8 a large effect size (Cohen 1988). The analyses allowed for the hierarchical structure of the data, that is that students belong to a class within a school. Allowance was also made for the time between the different measurements which varied. Time was expressed in weeks in the analyses.

15 Educational Review Results In this section we present the results of: (1) the comparison between the intervention group and the large control group (students who were neither interviewed nor took part in the intervention); (2) the comparison between the intervention group and the interview group Comparison between the intervention group and the control group of students who neither took part in the intervention nor were interviewed As explained, this comparison set out to investigate whether the degree of growth varied between the intervention group and the control group of students who neither took part in the intervention nor were interviewed. If a difference was to be found and the scores of the intervention group were better than those of the control group, this would indicate that the intervention was effective. In section 4.2 we compare the intervention group with the interview group, to investigate the possibility that it was the personal conversation with the students that had a positive effect rather than the intervention itself. Table 3 shows the results of the comparison between the intervention group and the students who neither took part in the intervention nor were interviewed. The column score m1 shows the average score at measurement 1. Apart from one variable, this was the same for the intervention and control group. The column growth shows whether the degree of increase or decrease varied between the two groups, and the columns growth*girl and growth*ethnic minority show how much this varied by gender and ethnic origin. These three columns ( growth, growth*girl and growth*ethnic minority ) give an indication of the effectiveness of the intervention. It is the results in these columns that we are mainly interested in. The last column shows the correlation between the score at measurement 1 and growth. This is included for the sake of completeness, although this figure is less relevant to our investigation of the effectiveness of the intervention. A negative score here shows that the higher the start score, the less growth (and vice versa). A difference between the intervention group and the control group in this column shows that the growth of the two groups did not run parallel and the extent of the divergence. For the significant differences between the intervention and control group, effect sizes were calculated (indicated with d in the table). The effect sizes were calculated for the duration of the study, namely 20 weeks. Indications that the intervention had a positive effect can be seen in the scores for effort at school, effort at maths and use of metacognitive strategies. Effort at school in general, and specifically at maths, decreased less in the children in the intervention group than in the children in the control group. Use of metacognitive strategies by students in the intervention group increased more than by the students in the control group. However, effect sizes were very small to small. A difference between boys and girls with regard to the effectiveness of the intervention was found for academic delay of gratification, effort at school in general and use of metacognitive strategies. A difference between ethnic minority students and native Dutch students with regard to the effectiveness of the intervention was found for long-term future time perspective on leisure. We will now look at these results in more depth with the aid of graphs.

16 110 T. Peetsma and I. Van der Veen Table 3. Results of the latent growth curve analysis of the comparison between the intervention group and the students who neither took part in the intervention nor were interviewed (unstandardized estimates and effect sizes (d) for the significant differences between both groups from measurement 1 to 4). score m1 d growth d growth*girl d growth*ethnic minority d score m1*growth d Academic delay of gratification Intervention no intervention/interview effort at school Intervention no intervention/interview effort at maths Intervention no intervention/interview metacognitive strategies Intervention no intervention/interview self-efficacy Intervention no intervention/interview well-being with teachers Intervention no intervention/interview well-being with fellow students Intervention no intervention/interview well-being at school Intervention no intervention/interview Mastery approach Intervention no intervention/interview performance approach (Continued )

17 Educational Review 111 Table 3. (Continued ). score m1 d growth d growth*girl d growth*ethnic minority d score m1*growth d Intervention no intervention/interview short-term future time perspective on school & career Intervention no intervention/interview long-term future time perspective on school & career Intervention no intervention/interview short-term future time perspective on leisure Intervention no intervention/interview long-term future time perspective on leisure Intervention no intervention/interview Note: Bold indicates that there is a significant difference (p < 0.05) between the two groups.

18 112 T. Peetsma and I. Van der Veen Table 4. Results of the latent growth curve analysis of the comparison between the intervention group and the students who were interviewed (unstandardized estimates and effect sizes (d) for the significant differences between both groups from measurement 1 to 4). score m1 d Growth d score m1*growth D Academic delay of gratification Intervention Interview effort at school Intervention Interview effort at maths Intervention Interview metacognitive strategies Intervention Interview self-efficacy Intervention Interview well-being with teachers Intervention Interview well-being with fellow students Intervention Interview well-being at school Intervention Interview Mastery approach Intervention Interview performance approach Intervention Interview short-term future time perspective on school & career Intervention Interview long-term future time perspective on school & career Intervention Interview short-term future time perspective on leisure Intervention Interview long-term future time perspective on leisure Intervention Interview Note: Bold indicates that there is a significant difference (p < 0.05) between the two groups Academic delay of gratification Figure 1 shows that academic delay of gratification for boys in the intervention group increased more than for boys in the control group (small effect size: d = 0.180). For girls there was a decrease and this was more marked in the intervention group than in the control group. This indicates that the intervention had a positive effect on boys but a negative effect on girls. On average, the intervention had no effect: the results for boys and girls cancelled each other out.

19 Educational Review Effort at school Effort at school decreased slightly less sharply for students in the intervention group than for students in the control group (see Figure 1). This would seem to indicate, therefore, that the intervention was successful. If, however, we look at Figure 1, we can see that the decrease was only less marked for the girls in the intervention group. For the boys it was the other way round. Figure 1. Significant differences between the intervention group and the control group.