Assessing implicit motives with an online version of the picture story exercise

Transcription

1 Motiv Emot (2011) 35: DOI /s ORIGINAL PAPER Assessing implicit motives with an online version of the picture story exercise Katharina Bernecker Veronika Job Published online: 13 August 2010 Ó Springer Science+Business Media, LLC 2010 Abstract An online version of a Picture Story Exercise (PSE) was administered to 180 participants to assess implicit motives for achievement, affiliation, and power. First, data were analyzed with respect to different aspects of participant behavior (dropout, writing time, story length, setting). Second, online data were compared with data collected in the laboratory (n = 93). Results regarding motive density, internal consistency, picture profiles, and position effects were comparable among samples and consistent with previous research (Pang and Schultheiss in J Person Assess 85: , 2005; Schultheiss and Brunstein in J Person Assess 77: 71 86, 2001). In addition, online participants completed self-report measures assessing explicit motives and the Big Five personality factors. Explicit motives showed no substantial overlap with PSE motive scores, replicating previous research. Personality factors revealed gender differences in correlates: men with a high implicit power motive reported less agreeableness and conscientiousness. Results suggest that administering the PSE online yields reliable and valid data. Keywords Implicit motives Picture story exercise Internet Assessment Introduction In the past few years, the Internet became an important tool for research in psychology, as indicated by a rapid expansion of web-based research. 1 The Internet offers researchers the opportunity to gather information from a larger number of individuals, without expensive lab assessment sessions that require laboratory space, equipment, and personnel hours from the researchers, and time and travel from participants (Birnbaum 2004a; Buchanan 2000; Reips 2002a). Particularly psychologists involved in the assessment of personality attributes have profited from Internet administration of questionnaires. It is easy to post items from personality inventories on web-pages to be completed anonymously and in a variety of settings convenient to participants all around the world (Buchanan et al. 2005; Buchanan 2001; Gosling et al. 2004). An open question remains whether the Internet can be a valuable platform to measure implicit motives, aspects of personality, which cannot be assessed using self-report measures. Instead implicit motives are usually measured by Picture Story Exercise (PSE; as the Thematic Apperception Test, TAT; Murray 1943) a projective measure which asks participants to write imaginative stories as open-ended responses to a set of picture cues. Implicit motives K. Bernecker (&) Department of Psychology, University of Erfurt, Postfach , Erfurt, Germany katharina.bernecker@stud.uni-erfurt.de K. Bernecker V. Job Department of Psychology, Stanford University, Stanford, CA, USA Implicit motives are defined as enduring preferences for specific classes of affective incentives (McClelland 1985; Schultheiss 2008). They provide the motivational resource 1 For a recent list of studies conducted online see the American Psychological Society (APS) experiment list maintained by John Krantz:

2 252 Motiv Emot (2011) 35: to strive for motive-congruent goal states as they orient, select, and energize behavior directed to a motive specific end state and a corresponding affective experience. Three motives came to be seen as primary and were called by McClelland (1985) the big three : the achievement motive, the affiliation motive, and the power motive. The achievement motive represents the need (n) for the experience of accomplishing something difficult, attaining a high standard, or excelling (nachievement). The affiliation motive is the need for the establishment and maintenance of positive relationships with others (naffiliation). The power motive in the need for the experience of having an impact on other people by influencing and directing their behavior (npower). Implicit motives develop very early in the preverbal phase of life via affective, not verbally processed experiences (McClelland and Pilon 1983), which is why they are not accessible to introspection. Implicit motives prompt spontaneous, self-initiated behavior and, therefore, predict long-term behavioral patterns (Brunstein and Maier 2005; McClelland et al. 1989; Spangler 1992). Engagement in behavior that is congruent with implicit motives leads to positive affective experiences which, in turn, may accumulate to a higher level of emotional well-being. Accordingly, people who progress towards goals congruent with their implicit motives report higher levels of emotional well-being than people whose goal striving is not in line with their implicit motives (Brunstein et al. 1998). Chronic frustration of implicit motives has been shown to constitute a stressor which is associated with negative affect and which has negative effects on physical health (e.g., McClelland and Jemmott 1980). In the last two decades, researchers became increasingly interested in the relation of implicit motives and explicit motives (exachievement, exaffiliation and expower) which are assessed via self-report measures (McClelland et al. 1989; Spangler 1992). There is abundant evidence that implicit motives are largely independent from explicit motives (e.g., decharms et al. 1955; King 1995; Schultheiss et al. 2009; Spangler 1992; but see Emmons and McAdams 1991; Thrash et al. 2007, for correlation depending on the assessment procedure). Empirical findings suggest that explicit motives, as opposed to implicit motives, shape short-term respondent behavior that is performed self-consciously in response to the extrinsic demands of a given situation. The low correlation between implicit and explicit motives and the different ways they shape behavior led to the development of a two-systems model of motivation which defines implicit and explicit motives as two distinct constructs with different predictive validity (McClelland et al. 1989). Discrepancies between implicit and explicit motives (e.g., someone high on npower but low on expower) opened a new field of research. Implicit/explicit motive discrepancies were found to be related to negative outcomes in terms of health and wellbeing (Baumann et al. 2005), self-regulation (Kehr 2004), and health behavior (Job et al. in press). Research on implicit motives, and hence the use of PSE as a diagnostic tool, has experienced a renaissance during the last 10 years (Schultheiss and Brunstein 2010). The role of implicit motives has been investigated in fields such as affective neuroscience and hormone release (Schultheiss et al. 2008b; Stanton and Schultheiss 2009), memory (Woike et al. 2001; Woike et al. 2003), health and health behavior (Baumann et al. 2005; Job et al. in press), and personal goal striving (Brunstein et al. 1998). Assessment of implicit motives with the PSE The PSE is a projective measure in which participants write imaginative stories in response to a set of usually 4 8 picture cues. The pictures show people in different situations, for instance a ship captain talking to a passenger or an architect working in an office. Each picture is presented separately and participants are instructed to make up a short story about each of them within a certain amount of time. The PSE was traditionally administered individually or in small groups using a paper pencil format. During the session an experimenter had to be present to show the pictures and control writing time. Today computer-based administration is more common with no experimenter being present during the session. In this case, stories are typed directly into the computer via keyboard. 2 Stories are later coded for specific content which indicate a person s implicit motive disposition. In the past, there has been a great deal of debate regarding the psychometric properties of the PSE. Past research found the instructions to be a crucial factor to increase the PSE s test retest reliability (Lundy 1981, 1985; Winter and Stewart 1977). In a recent meta-analysis, the average test retest reliability of the PSE was satisfactory and the rate of decrease with increasing retest intervals was similar to that observed for questionnaire trait measures (Schultheiss and Pang 2007). Picture motive profiles, which is the amount of achievement, affiliation and power motive imagery for an individual picture cue, has also been demonstrated to be stable between two testing sessions spaced 2 weeks apart (Schultheiss et al. 2008a). 2 Typed stories were found to be by a third longer than handwritten ones, but when corrected for word count the two methods of administration produce comparable motive scores (Blankenship and Zoota 1998; Schultheiss et al. 2008a).

3 Motiv Emot (2011) 35: The PSE on the Internet Researchers on implicit motives generally face a variety of costs and hurdles accompanying the application of PSE. Although computer-based administration in the laboratory decreases time for otherwise necessary transcription and word counting tasks, the administration of the PSE is nevertheless time-consuming for researchers and participants. This aspect might be particularly disadvantageous when non-student populations are desired, such as managers or middle-aged adults, leading researchers to choose more economical procedures than the PSE (e.g., Job et al. in press; Kehr 2004). Although administration of the PSE on the Internet does not eliminate all costs involved in PSE assessment (i.e., participants still have to write stories, which still have to be manually coded) the Internet can substantially reduce some costs and offer new benefits, particularly with respect to limit the need for laboratory sessions, as well as to provide access to larger and non-student populations. For instance, researchers could administer the PSE online before participants are invited to the lab, which leaves more time for manipulation and measurements and allows for the potential to invite only participants who score high on certain motive dispositions. However, Internet assessment of implicit motives with the PSE also raises a variety of challenges. The most serious problem involves the lack of experimental control over the testing situation. Since implicit motives are aroused by and therefore sensitive to incentives in the surrounding environment, a standardized, neutral testing situation was often seen to be a precondition for the valid assessment of implicit motives (Klinger 1968; Schultheiss and Pang 2007; Smith 1992). Further, distractions or motivational issues (Buchanan and Smith 1999) can negatively affect the flow of imagery and participants involvement in story writing. Finally, since the PSE is time consuming and requires high amounts of cognitive effort and involvement from the participant, participants who do not want to put forth cognitive effort may self-select out of the study. This could occur in two ways. First, people could decline to participate in such a long study. Second, participants could decide to drop out over the course of the procedure (Birnbaum 2004b). Therefore, participants behavior in online PSE assessment and properties of the resulting PSE data have to be examined before implicit motives can be reliably assessed on the Internet. Overview of this study The aim of the present research was to create a web-version of the PSE and compare data collected via the Internet with data collected in the laboratory. To make the two settings comparable lab participants completed the PSE on the computer in separated cubicles and with no experimenter present. Comparing the two data sets we focused on the following questions: participant behavior and setting (e.g., drop-out rate), motive density (amount of motive imagery in relation to story length), internal consistency (stability of motive imagery across picture cues), picture motive profiles (amount of nachievement, naffiliation and npower imagery for an individual picture cue), and picture position effects (effects due to position of the picture within the set). The online sample additionally completed self-report measures assessing explicit motives and the Big Five personality traits to compare correlations with the PSE motive scores with those found in previous research (Pang and Schultheiss 2005; Schultheiss and Brunstein 2001). Participant behavior and setting. In contrast to participants in the lab, online participants can drop out from a study more easily, simply by closing the browser window. This makes online research more sensitive to dropout and sampling bias (Birnbaum 2004b; Frick et al. 2001; Reips 2000; Reips and Bosnjak 2001). Web experiments that take longer than a few minutes might cause psychological reactance and data quality may suffer from high dropout rates (Reips 2002a). Hence, we were interested in how sensitive the rather time consuming PSE would be to dropouts. Another question was at which point people would stop participating and whether specific picture cues lead to more dropouts than others. As opposed to usual PSE administration in the lab online participants were given the opportunity to proceed to the next picture before response time for story writing was over, hence, it was further possible to look at writing times for each picture in the online study and to compare story lengths with the lab sample. Since participants in the online sample chose the setting themselves, we wanted to investigate whether specific characteristics of the surroundings, for instance the presence of other people, would affect PSE motive scores. Motive density of written stories. The uncontrolled environment might distract participants of the online study in a number of ways. It is further possible that participants online might look at pictures and instructions less closely which might lead them to write stories that are not comparable to lab-written stories regarding codable motive content. One indicator to check for comparability is motive density, which means the amount of motive imagery produced per word count (Winter 1994). If stories written in the lab and online were comparable the type of study (online vs. lab) should not predict overall motive imagery after controlling for word count. Previous research with PSE consistently showed that there is a positive relationship between story length and motive imagery. The longer stories participants write the more motive content they usually produce (Schultheiss

4 254 Motiv Emot (2011) 35: et al. 2008a; Smith et al. 1992). It is not self-evident that this relationship will hold for stories written online. There might be more variation in online stories regarding motive density which would be reflected in lower correlations between word count and motive imagery. Therefore, we compared whether the type of study moderates the word count motive imagery relationship. Internal consistency. Previous research reported low scores for internal consistency between picture cues estimated by Cronbach s a (-0.02 to 0.43) (Schultheiss and Pang 2007; Schultheiss et al. 2008a). Atkinson (1981) argued that measures of reliability are not as applicable for the assessment of implicit motives, because they are dynamic need states that depend on environmental incentives. People might have learned which situations allocate motivational incentives; hence behavior may differ between these situations. Similarly, people s stories vary in motive imagery between different picture cues. Although we did not expect internal consistency to be high in either sample, we explored whether Cronbach s a would be comparable between the lab and online assessment. Picture motive profiles. Previous research has repeatedly found that different picture cues elicit different amounts of nachievement, naffiliation, and npower in participants stories (Langan-Fox and Grant 2006; Pang and Schultheiss 2005; Schultheiss and Brunstein, 2001). For instance, women in laboratory constantly elicited high amounts of nachievement and npower imagery but almost no naffiliation imagery. The different environments of online participants might not only contain possible distractions but a variety of random incentives which might interact with picture cues as well as peoples motive dispositions, and so affect motive imagery. The comparison of picture motive profiles between the two samples can give insight into the presence of such random factors. If the picture motive profiles from the online sample are less pronounced than the motive profiles obtained in the lab this could indicate the impact of additional incentives. Therefore, the present research tested whether picture profiles from the lab and from previous research can be replicated in the online sample. Picture position effects. Pang and Schultheiss (2005) first explored effects of picture position on story length, motive imagery, and motive profiles of individual picture cues. By randomizing the order of picture cues for each participant, they found that participants increased the length of their stories in the first few pictures, and maintained this length to the last picture. Achievement motive imagery remained relatively constant, whereas affiliation imagery dropped in the middle of the picture sequence and power imagery showed the reverse pattern. However, the dynamic effects of both motives were modest and as far as we know have not been replicated yet. Individual picture cues did not differ in motive profiles with regard to their position in the picture sequence. Therefore, our study aimed to compare online and lab study data in position effects on story length, motive imagery, and on motive profiles of individual picture cues. Correlations between PSE motive scores and self-report measures. Past research consistently found little overlap between implicit motives and explicit motives (e.g. King 1995; Thrash et al. 2007; Schultheiss et al. 2009). Others studied the relationship of implicit motives and traits measured with self-report scales such as the Big Five personality factors and did not find substantial overlap either (Schultheiss and Brunstein 2001; Winter et al. 1998). As previous research has already addressed these questions with lab administered PSEs, we were interested how implicit motive scores were related to self-report scales in our online study. Methods Designing an online PSE measure We programmed the online PSE measure in WEXTOR, a web-based tool for Internet experiments created by Reips and Neuhaus (2002), which includes features that meet general methodological requirements of web research (e.g., generation of a session ID to detect multiple submissions by the same participant). As described below, we used several techniques to address methodological challenges of Internet research (Reips 2002a, b) and specifically for an online PSE assessment. To reduce dropout we used two techniques that were used in previous web experiments (Musch and Klauer 2002; Reips 2000, 2001; Reips and Morger 2001): warm-up and seriousness check. The warm-up technique was developed on the basis of the observation that most participants drop out in the beginning of a study. Therefore, Reips (2002a) recommends placing the main measurement several pages into the survey. Before participants see the first PSE picture cue on the eighth page, they have already answered questions on demographic information and filled in a seriousness check, which is a single question asking, How do you intend to browse the Web pages of this study? with two response choices: I would like to seriously participate now. or I would like to look at the pages only. Data is recorded in both cases, but only data from persons who expressed a serious motivation to participate were analyzed. First participants read the standard instructions adapted from Smith (1992): On the upcoming pages we will show you different pictures, each for about 20 s and ask you to make up

5 Motiv Emot (2011) 35: a story about each of them. Try to imagine what is happening. Then write down what is going on now, what happened before, what the people are thinking and feeling, and what they are going to do. Write a story that is complete a story that has a beginning, a middle, and an end. There are no right or wrong stories, so write whatever story you think of when you look at the picture. Spelling, punctuation, and grammar are not at all important. What is important is to write whatever story you think of as you imagine what is going on in each picture. The four picture cues (ship captain, architect at desk, women in laboratory, and trapeze artists) were presented in randomized order, in the middle of the screen, and formatted to fit screens with 17 diagonal or larger. Each picture was headed by a short instruction saying: Look at the picture The program will automatically turn to the next page within 20 s. On the next page a text field was presented with six reminders about what to include in the story (e.g. What is happening?, Who are the people? ). We limited the text field to 3000 words to prevent participants from pasting long texts into the survey. The text field was presented for 5 min before the program automatically turned to the next page. As the program recorded response times (in milliseconds) for all pages we were able to control for the time participants spent on story writing, and therefore allowed participants to move onto the next picture whenever they were ready. Participants The sample consisted of 180 participants for the online study (97 women, 56 men, 6 not indicated, 21 dropped out before indicating their gender) and 93 participants (58 women, 26 men, 9 not indicated) for the lab study. The two samples did not differ in terms of gender composition and the number of non-indications, v 2 (2, N = 252) = 4.41, ns. Participants in the online study were significantly older (M = 37.01, SD = 9.34) than participants in the lab study (M = 29.75, SD = 11.83), t(210) =-4.97, p \ (two-tailed), d = As previous research found that older participants express significantly less achievement motive imagery (Pang and Schultheiss 2005), we controlled for age in all further analysis to rule out age differences as an explanation for the results. Procedures Online study. Participants were recruited from a nationwide online pool. New pool members are recruited via advertisements on a website that provides local sales and job forums throughout various cities of the US. Registered members receive an once a week informing them of studies available to them that week. They can then log in and follow the link to participate in those studies. In the present study, participants were invited to take part in a 60-min online study on Relationships, Leisure, and Work that would ask participants to write short stories about people in different situations (i.e., referencing the PSE) and to answer questions about different areas of your everyday life (i.e., referencing the other personality measures administered after the PSE). 3 The description further informed potential participants that they would need to provide their address to receive a $12 gift card for compensation. When following the link to the survey, participants first answered the seriousness check. Next they were informed about the purpose and duration of the study, before they provided consent and verified their age. The main study began with the PSE assessment of implicit motives. In the second part of the study participants were asked control questions about the kind of place (i.e. private, semi-public, public) where they had completed the PSE, if they were surrounded by other people and if they had communicated with them as they completed the measures. Then participants completed a standard measures of explicit motives and the Big Five personality factors as well as additional measures of interest for a larger research project. After the study participants were automatically guided to a separate online survey, where they were asked to provide their address in order to receive a gift card for compensation. This approach allowed us to save addresses separately from the rest of participants data securing anonymity. We also provided a link to a website offering easy creation of compostable addresses (ones that expire after a given time of use) in case participants did not want to provide a private address. Lab study. Most participants were recruited by flyers, ads on a website that provides local sales and job forums, and from the participant pool of the psychology department at Stanford University. Participants took part individually in lab sessions in a university building. They were seated in a separate cubicle in front of a computer. To make the settings comparable lab participants also completed the PSE on the computer and without any experimenter being present during the session. Hence, the 60-min session consisted of two parts with the PSE measure administered first. Instructions and pictures of the PSE measure were identical to the online measure; except participants were not allowed to move onto the next picture until 5 min had elapsed and the program would lead them there. 3 The PSE assessment was part of a questionnaire for a larger research project on long-term relationships.

6 256 Motiv Emot (2011) 35: Coding of PSE stories Stories were coded for motive-specific content using Winter s (1991) Manual for Scoring Motive Imagery in Running Text assessing nachievement, naffiliation, and npower. According to the manual, nachievement was coded for story contents expressing concern for a standard of excellence, e.g. indicated by adjectives that positively evaluate performances or positive evaluated goals and performances. Story content coded for naffiliation was that indicating the establishment, maintenance, or restoration of friendly relations among persons including a warm and friendly quality to the relationship, e.g. indicated by expressions of positive or intimate feelings toward others. Finally, npower was scored for content indicating impacts or influence on other persons or groups, e.g. by strong, forceful actions or eliciting strong emotions in others. Each story was coded by two coders who reached an overall agreement of Cohen s Kappa of Additional measures in online study Explicit motives. To assess explicit motives, we used the three subscales for exaffiliation, exachievement, and expower (ex stands for explicit) of McClelland s Personal Values Questionnaire (PVQ, McClelland 1991). Each scale consists of ten items representing personal values, e.g. Maintaining close relationships with the people I really care about (exaffiliation, M = 4.56, SD = 0.75, a = 0.85), Continuously new, exciting, and challenging goals and projects (exachievement, M = 4.61, SD = 0.80, a = 0.90), and Important positions and projects that can give me recognition (expower, M = 3.72, SD = 1.12, a = 0.92). Participants were asked to rate the personal importance of each item on a fully labelled scale from 1 (not important to me) to7(extremely important to me). Personality scales. We assessed the Big Five personality factors, neuroticism, extraversion, openness to experience, conscientiousness, and agreeableness, using a 10-item short version of the Big Five Inventory (Rammstedt and John 2007). Participants rated their agreement on each item on a fully labelled scale from 1 (strongly disagree) to 5 (strongly agree). Sample items included, for neuroticism I see myself as someone who gets nervous easily (M = 2.77, SD = 0.97, a = 0.64), for extraversion, e.g. I see myself as someone who is outgoing, sociable (M = 3.20, SD = 0.86, a = 0.43), for openness to experience, e.g. I see myself as someone who has few artistic interests (reverse, M = 3.42, SD = 0.82, a = 0.34), for conscientiousness, e.g. I see myself as someone who does a thorough job (M = 3.84, SD = 0.92, a = 0.69) and for agreeableness, e.g. I see myself as someone who tends to find fault with others (reverse, M = 3.58, SD = 0.84, a = 0.57). As internal consistency for openness to experience was low (a = 0.34), we excluded it from further analysis. Results Participants behavior in the online assessment Dropout. Out of 180 participants who followed the link to our online survey, 137 (76.1%) completed the PSE measure. We identified a group of participants (11.7%) for whom the program did not collect response times for the very first page and concluded that those were not able to start the survey probably due to technical problems like interaction of certain web browser versions and incompatible elements on web pages (Schwarz and Reips 2001). These participants do not count as dropouts since they did not choose to not continue the survey (Reips 2002a). Of the 159 participants who actually started the survey 13.8% dropped out, which is about the same rate of 13.2% found by Frick et al. (2001), who offered financial incentives at the beginning of the survey and asked for personal information at the end of it, similar to our procedure. All participants who dropped out did so during the first nine pages; none did later. Most drop outs quit after the first text field (n = 9), presented on the ninth page of the survey. There was no effect of the picture cue presented on page eight on drop outs, v 2 (3, n = 147) = 3.32, ns. Figure 1 shows the dropout and remaining sample size over the course of the survey. Of participants who finished the PSE, 8.0% did not write stories but instead wrote short descriptions of the pictures or wrote about something that was not related to the picture. Because we doubted whether these participants understood the instructions, we excluded their data from Absolute number of dropout Instr 2 Consent Instr 1 Demogr Start Picture 2 Text field 1 Picture 1 Instr 4 Instr 3 Web page Dropout Sample size Fig. 1 Dropout and sample size for online study tracked over pages of the PSE measure Thank Percentage of remaining participants

7 Motiv Emot (2011) 35: the coding procedure, resulting in a sample of 125 for the final analysis. Writing time and story length. We were interested in how long participants in the online study actually spent writing stories, as they could proceed to the next picture at any time. Therefore, we averaged writing times for the four stories (recorded response times for the four pages including the text field) and found an average time of M = (SD = ) seconds (about 11 min). This was about half the time participants were allowed to write the four stories. Individual picture cues did not differ in writing time, F(3, 134) = 0.80, ns. Next, we tested for a trend in writing time and conducted a repeated measures ANOVA with writing time at each position (one to four) as within-subjects variable and found a significant linear trend, F(3, ) = 6.67, p \ 0.01, n 2 = Writing time decreased over the four stories. Next we examined the relation of writing time and motive imagery and found a significant correlation between writing time and raw motive scores for all three motives, r(125) [ 0.32, p \ The longer participants wrote the more motive imagery was expressed. As participants in the lab were not allowed to proceed to the next picture until 5 min had elapsed for each picture, we could not compare the writing time in the two studies. Instead we looked at differences in story length counting words for all four pictures. Stories written in the lab were significantly longer (M = words; SD = ) than stories written online (M = words; SD = ), F(1, 216) = 81.44, p \ 0.01, partial n 2 = 0.27, which is not surprising since participants in the online study were allowed to proceed to the next picture. Since we aimed to compare motive scores that are corrected for story length, we examined whether story length fully accounted for differences in writing time in the online study. Hence, we ran a linear regression to see whether writing time still predicted raw motive scores when controlling for story length. For all three motives the analysis revealed that writing time was no longer a significant predictor for motive scores when story length was entered into the regression equation, t(121) \ 0.30, ns. Therefore, controlling for story length in further analyses is sufficiently accounting for the differences in writing time between the two studies. Setting. In the online study 90.7% of participants indicated that they had completed the PSE measure in a private place, 6.2% in a semi-public place, and 3.1% in a public place. We combined the latter two and compared them with the private group regarding writing time, story length, and motive imagery using two-tailed t tests. The private group spent marginally more time writing, t(120) = 1.70, p = 0.09, d = 0.54, but did not produce longer stories, t(120) = 0.88, ns. The stories of the private group contained significantly more nachievement motive imagery observing raw scores, t(120) = 2.34, p \ 0.05, d = 0.74, but did not differ in naffiliation or npower imagery, \ t(120) \ 0.92, ns. 4 Second, we tested if being among other people (reported by 29.5% of participants) related to writing time, story length, or motive imagery. We conducted two-tailed t tests to compare this group with participants who indicated they were alone when they completed the measures. There was no difference in writing time, t(120) \ 1.11, ns, story length, t(120) \ 0.80, ns, or raw motive scores, \ t(120) \ 1.36, ns. Motive density One way to check for comparability in motive density is to analyze whether the study type (online vs. lab) predicts overall motive imagery after controlling for word count. Hence, we ran a hierarchical regression analysis predicting overall motive imagery by word count in a first step adding study type (online = 0; lab = 1) in a second step. This analysis revealed both a significant main effect of word count, R 2 = 0.48, b = 0.70, t(1, 216) = 14.25, p \ 0.001, and of study type, b =-0.14, t(1, 216) =-2.41, p \ 0.05, DR 2 = 0.01, p \ There was a tendency for stories written online to be higher in motive density than stories written in the lab. We ran the same regression analysis for each motive separately. In a hierarchical regression we predicted the raw motive scores by word count and study type (online = 0; lab = 1). For naffiliation we found a significant main effect of setting, b = -0.23, t(2, 215) =-3.34, p \ However, there was no effect of study type predicting raw scores for nachievement, b =-0.09, t(2, 215) =-1.25, ns, or for npower, b = 0.01, t(2, 215) = 0.12, ns. Online stories were denser specifically for naffiliation. In a last step we wanted to see whether the online study differed from the lab study in the variation in motive density. Therefore, we compared whether the type of study moderates the word count motive imagery relationship. Hierarchical regression analyses tested whether the study type interacted with word count to predict motive scores revealed no interaction for any of the three motives, t(3,214) \ -1.32, ns. Hence, we found no evidence for a difference between online and lab written studies in the relationship between word count and motive imagery. To control for the effects of story length, raw motive scores are often corrected using a formula introduced by Winter (1994): (motive score/word count) 9 1, Six participants did not provide stories to all four picture cues and were therefore not included in the analysis for overall word count.

8 258 Motiv Emot (2011) 35: Table 1 shows raw motive scores and word corrected motive scores for each picture cue separately for both studies. Two-tailed t tests showed that after word correction stories in the online sample included more motive imagery in naffiliation, t(216) =-3.56, p \ 0.001, d = -0.49, and in nachievement, t(216) =-2.00, p \ 0.05, d = For npower the trend was in the same direction but the effect was not significant, t(216) =-1.14, ns. Internal consistency We estimated Cronbach s a for nachievement, naffiliation, and npower using the raw motive scores for all four pictures and found somewhat higher reliability scores in the lab study (a Ach = 0.55, a Aff = 0.50, a Pow = 0.53) than in the online study (a Ach = 0.27, a Aff = 0.48, a Pow = 0.45). For word corrected motive scores internal consistency was still somewhat higher in the lab study (a Ach = 0.20, a Aff = 0.32, a Pow = 0.52) than online study (a Ach = 0.15, a Aff = 0.29, a Pow = 0.002), meaning that although pictures in the online study were on average denser in motive imagery and individuals stories differed more in motive imagery across picture cues. Picture profiles We tested the difference in motive profiles between individual picture cues and between studies in a repeated measures ANOVA using picture (one to four) and motive (nachievement, naffiliation, and npower) as within-subjects variables and study type (online = 0, lab = 1) as a between-subjects variable predicting word corrected motive scores. Replicating the results of previous research the picture cues differed in their motive profiles, as the Picture 9 Motive interaction was highly significant, F(2, 215) = 51.64, p \ 0.001, n 2 = The studies slightly differed in picture profiles as the three-way interaction for Picture 9 Motive 9 Study Type was significant, F(6, 211) = 4.98, p \ 0.001, n 2 = 0.02, although the effect was small. 5 Figure 2 shows motive profiles for individual picture cues for both studies for raw motive scores and word corrected motive scores. Motive profiles in the online study showed a greater range compared to the lab profiles. For instance, architect at desk showed the highest pull for naffiliation in both studies but the difference between naffiliation and nachievement was larger in the online study. Similarly, women in laboratory showed the highest 5 When we used raw motive scores we found the same effects: both the Picture 9 Motive interaction, F(2, 215) = 70.28, p \ 0.001, n 2 = 0.25, and the three-way interaction were significant, F(6, 211) = 2.34, p \ n 2 = pull for nachievement in both studies, but the difference with npower was again larger in the online study. Motive profiles for ship captain and trapeze artists were similar in pull and range though for the two studies. To test these differences for statistical significance we conducted repeated measures ANOVAs separately for picture cues and entered motive as a within-subjects variable and study type as a between-subjects variable. For architect at desk the interaction for Motive 9 Study Type was significant, F(2, 216) = 11.31, p \ 0.001, n 2 = Post hoc t-tests confirmed that architect at desk had a significantly higher pull for affiliation imagery online than in the lab, t(218) = 3.73, p \ 0.01, d = 0.51, while there was no difference between online and lab study for nachievement and npower. For women in laboratory the Motive 9 Study Type interaction was smaller but significant, F(2, 216) = 3.34, p \ 0.05, n 2 = Post hoc t-tests confirmed the higher pull for achievement imagery for women in laboratory in the online study, t(218) = 2.45, p \ 0.05, d = There was no difference between online and lab study for naffiliation and npower. For ship captain and trapeze artists the Motive 9 Study Type interaction was not significant, F(2, 216) = 1.55, ns, and F(2, 215) = 0.98, ns, respectively. Table 1 illustrates similarities and differences in absolute pull of pictures for motive imagery: High pull represents when at least 50% responded with at least one instance of codable motive imagery to the picture (Schultheiss and Brunstein 2001). For both studies, architect at desk pulled high for naffiliation, women in laboratory high for nachievement and trapeze artists high for both nachievement and npower, which is consistent with findings from previous studies using this indicator of picture profiles (Pang and Schultheiss 2005; Schultheiss and Brunstein 2001). But only for the lab study the pictures ship captain and women in laboratory pulled high for npower, and replicated previous findings. Picture position effects As we randomized the order of picture cues in both studies we were able to look at the effects of picture position on story length and motive imagery. Pang and Schultheiss (2005) found word count to increase with later position for PSEs administered in the lab and using six picture cues. To compare position effects in word count for the two studies we entered study type (online = 0, lab = 1) as betweensubjects variable in a repeated measures ANOVA predicting word count for the four positions. There was a marginally significant Position 9 Study Type interaction, F(3, 208) = 2.23, p = 0.08, n 2 = 0.01, meaning there were different trends for word count in the two studies. Therefore, we conducted the repeated measures ANOVAs

9 Motiv Emot (2011) 35: Table 1 Means and standard deviations of raw and word corrected scores across motives and picture cues for online and lab study Picture nachievement naffiliation npower Words M SD M SD M SD M SD Raw scores Ship captain Online-study Lab-study Difference -0.36** ** -1.19** Architect at desk Online-study Lab-study Difference -0.39** ** -1.08** Women in laboratory Online-study Lab-study Difference ** -1.14** Trapeze artist Online-study Lab-study Difference -0.32* ** Total Online-study Lab-study Difference -0.41** ** -1.24** Word corrected scores Ship captain Online-study Lab-study Difference ? Architect at desk Online-study Lab-study Difference ** Women in laboratory Online-study Lab-study Difference 0.36* Trapeze artist Online-study Lab-study Difference * Total Online-study Lab-study Difference 0.27* 0.49** Note: Difference = (Cohen s d) differences derived from t tests (two-tailed) between online and lab study within picture and coding category. Underlined motive scores indicate that more than 50% of participants in the total sample or subsample (online or lab study) have responded with at least one instance of codable motive imagery to the picture cue Online: n = 125; Lab study: n = 93? p \ 0.10; * p \ 0.05; ** p \ 0.01

10 260 Motiv Emot (2011) 35: Raw motive scores Online Study nachievement naffiliation npower Lab Study nachievement naffiliation npower 0 Ship captain Architect at desk Women in laboratory Trapeze artist 0 Ship captain Architect at desk Women in laboratory Trapeze artist Word corrected motive scores Ship captain Architect at desk Women in laboratory nachievement naffiliation npower Trapeze artist Ship captain Architect at desk Women in laboratory nachievement naffiliation npower Trapeze artist Fig. 2 Picture profiles for online study and lab study for raw motive scores (above) and word-corrected motive scores (below) separately for the studies and found word count consistently increased in the lab study, F(1, 91) = 9.91, p \ 0.01, n 2 = 0.10, replicating the findings by Pang and Schultheiss (2005). However, in the online study we did not find any linear, F(1,119) = 0.01, ns, or quadratic trend, F(1, 119) = 0.44, ns. Next, we tested whether studies differed in the effects of position on motive imagery. Therefore, we created twelve variables each representing word corrected scores for nachievement, naffiliation, and npower at each of the four positions. We entered those variables in a repeated measures ANOVA with motive (nachievement, naffiliation, and npower) and position (one to four) as within-subjects variable and study type (online = 0, lab = 1) as betweensubjects variable. We found that motive imagery did not change over time for any of the motives, as the main effect for position, F(3, 208) = 0.25, ns, and the Position 9 Motive interaction were both not significant, F(6, 205) = 0.44, ns. The studies did not differ in this regard as the three-way interaction for Position 9 Motive 9 Study Type was also not significant, F(6, 205) = 1.28, ns. Last, we tested position effects for individual picture profiles and conducted MANOVAs predicting picturespecific motive scores in nachievement, naffiliation, and npower with picture position as quantitative predictor and study type (online = 0, lab = 1) as categorical predictor variable. For none of the picture cues position affected any of the motive imagery, as all Position 9 Study Type interactions were not significant, F(1, 214) \ 2.34, ns. Correlations for PSE motive scores and self-report measures Correlations among PSE motive scores. In the online study we found significant positive correlations for raw scores between all three motives, r(125) [ 0.25, p \ However, for word corrected motive scores the correlations were not significant, r(125) B 0.11, ns. In the lab study only raw scores for nachievement and npower were significantly correlated, r(92) = 0.38, p \ 0.001, which is consistent with findings from Schultheiss and Brunstein (2001). However, for word corrected motive scores the correlations were again not significant, r(92) B 0.16, ns. Explicit motives. Analyzing word corrected PSE scores we did not find any significant correlation between implicit and explicit motives, r(121) B 0.13, ns, replicating results from previous research (e.g. King 1995; Schultheiss et al. 2009). However, when dividing the sample by gender for men there was a marginally significant negative correlation between npower and exaffiliation (r =-0.29, p = 0.07)

11 Motiv Emot (2011) 35: Table 2 Correlations between word corrected implicit motives, explicit motives and the big five personality factors for men and women Gender exach exaff expow N E A C Men n Achievement ? n Affiliation n Power ? ** -0.26? Women n Achievement * n Affiliation n Power Note: ExAch, explicit Achievement motive; exaff, explicit Affiliation motive; expow, explicit Power motive; N, Neuroticism; E, Extraversion; A, Agreeableness; C, Conscientiousness Men: n = 83; Women: n = 43? p \ 0.10; * p \ 0.05; ** p \ 0.01 and a marginally significant positive correlation between nachievement and expower (r = 0.27, p = 0.08). Table 2 shows correlation coefficients between word corrected PSE scores and explicit motives separately for men and for women. Personality variables. In the online sample we found word corrected PSE scores for nachievement to be significantly correlated with self-reported conscientiousness, r(125) = 0.21, p \ and npower scores to be significantly negatively correlated with agreeableness, r(125) = -0.20, p \ Interestingly, there were gender differences in the relation of implicit motives and the Big Five personality factors. Table 2 shows correlation coefficients between word corrected PSE scores and the Big Five personality factors separately for men and women. Among men npower was negatively correlated to agreeableness and conscientiousness, which was highly significant and marginal significant, respectively. For women both correlations were near zero. However, among women npower was negatively correlated to neuroticism, whereas for men this correlation was positive, although not significant. Among both sexes nachievement was positively correlated to conscientiousness, but the relation was only significant among women. To see whether gender moderates the relationships between personality variables and motive scores we conducted hierarchical regression analyses. We predicted word corrected npower scores by gender (male = 0; female = 1) and agreeableness in a first step and entered their interaction term in a second step. The Gender 9 Agreeableness interaction got significant, DR 2 = 0.09, b =-0.53, t(121) = 3.49, p \ Similar hierarchical regression models predicting npower revealed a significant Gender 9 Conscientiousness interaction, DR 2 = 0.04, b = 0.40, t(121) = 2.13, p \ 0.05, and a marginal significant Gender 9 Neuroticism interaction, DR 2 = 0.03, b =-0.36, t(121) =-1.86, p = Last, we regressed word corrected nachievement on gender, conscientiousness, and their interaction term, but the interaction with gender got not significant, DR 2 = 0.001, b = -0.07, t(121) =-0.36, p [ Discussion The aim of the present research was to investigate whether it is possible to measure implicit motives with an Internet version of the PSE, which asks participants to write short stories as response to a set of picture cues. First, we documented how participants processed the online PSE as reflected by dropouts, story writing time, story length, and whether there were effects regarding the surrounding setting on motive imagery. Second, we compared stories written online with stories written in the lab regarding motive density, internal consistency, picture profiles, and position effects. Our basic goal was to document whether the data collected online were comparable with data collected in the lab and whether both were consistent with previous research (Pang and Schultheiss 2005; Schultheiss and Brunstein 2001). Participant behavior in the online study Dropout. The dropout rate in the online study was similar to those found in previous online studies using comparable procedures (Frick et al. 2001). The standard techniques incorporated in the online PSE measure successfully encouraged early dropouts and led to few dropouts later in the questionnaire. The fact that no participant dropped out after completion of the first story suggests that the PSE can be assessed online despite its time consuming and rather demanding procedure.