Assessing Anxiety with Extrinsic Simon Tasks

Transcription

1 Assessing Anxiety with Extrinsic Simon Tasks Stefan C. Schmukle 1 and Boris Egloff 2 1 Johannes Gutenberg-University Mainz, Germany 2 University of Leipzig, Germany Abstract. This article introduces two new indirect measures of anxiety that are based on the Extrinsic Affective Simon Task (EAST; De Houwer, 2003). The EAST differs from the more established Implicit Association Test (IAT; Greenwald, McGhee, & Schwartz, 1998) in that participants responses to different trials within one block of trials are compared rather than performance between two different blocks of trials. Two studies led to the following results: (a) Both extrinsic Simon tasks for assessing anxiety showed only moderate internal consistencies, (b) one of the two tasks showed at least some convergent validity with an IAT for assessing anxiety, and (c) both tasks were dissociated from self-reported anxiety. Implications for future adaptations of the EAST for the indirect assessment of personality dimensions are discussed. Keywords: Extrinsic Affective Simon Task, Implicit Association Test, indirect measures, anxiety In the last few years, several reaction time tasks were developed that aim at indirectly assessing psychological constructs, such as attitudes, stereotypes, self-esteem, and personality (Fazio & Olson, 2003). These so-called implicit measures hide their purpose of measurement (i.e., when responding to these measures, participants are unaware of the way in which conclusions are drawn from indicators of overt behavior). These tasks aim at minimizing response biases such as self-presentation and demand characteristics. Furthermore, drawing on a distinction between a reflective system (that processes information more consciously and in a controlled way) and an impulsive system (that processes information more unconsciously and automatically), implicit measures are assumed to capture information about the latter system that cannot be assessed by self-reports (see, e.g., Strack & Deutsch, 2004). The Implicit Association Test (IAT) Probably the most well-known implicit measurement procedure is the Implicit Association Test (IAT; Greenwald, McGhee, & Schwartz, 1998). The IAT measures strengths of associations between concepts by comparing response times in two combined discrimination tasks. Participants are required to sort stimuli representing four concepts using just two responses, each assigned to two of the four concepts. The basic assumption of the IAT is that if two concepts are highly associated, the sorting task will be easier (i.e., faster) when the two associated concepts share the same rather than different response keys. We adapted the IAT (IAT-Anxiety) to provide an implicit measure of the self-concept of anxiety (Egloff & Schmukle, 2002). Toward this aim, the categorization of items into self and other categories was combined with the classification of items into anxiety and calmness categories. An IAT effect indicating implicit anxiety was computed as the difference between self anxiety and self calmness. Thus, by measuring the relative ease of categorizing self items with anxiety items as compared to self items with calmness items, the IAT-Anxiety effect is an indicator of the implicit self-concept of anxiety. The IAT-Anxiety has been shown to be a reliable measure with good predictive validity (Egloff & Schmukle, 2002). However, there is a growing amount of literature that points to the problem that the IAT might measure more than only implicit associations as, for example, salience asymmetries (Rothermund & Wentura, 2004) or taskswitching abilities (Mierke & Klauer, 2003). Specifically, the IAT has limitations due to the fact that the IAT score is based on a comparison of responses between two different blocks of trials. These limitations include the following: 1. The order in which the two different blocks are presented affects the IAT effect (Greenwald et al., 1998). 2. The IAT aims to measure the relative strengths of associations between two bipolar concepts (e.g., the bipolar concepts of self vs. other and anxiety vs. calmness in the IAT-Anxiety). That is, for every personality trait, IAT assesses the self concept of a trait relative to the other concept of a trait. However, for personality assessment, one would be more interested in the pure association between the self concept and a particular personality dimension (cf. Karpinski, 2004; Pinter & Greenwald, 2005). 3. The IAT is limited to the assessment of only one dimension per test. Thus, several separate IATs are needed to measure multiple personality dimensions. Unfortunately, there is evidence that the order in which different IATs are presented might affect the results (e.g., the correlations with self-report measures; Schnabel, Banse, & Asendorpf, in press) Hogrefe & Huber Publishers Experimental Psychology 2006; Vol. 53(2): DOI /

2 150 Schmukle & Egloff: Extrinsic Simon Tasks The Extrinsic Affective Simon Task (EAST) Recently, De Houwer (2003) introduced a new task for an indirect assessment, the Extrinsic Affective Simon Task (EAST). The EAST differs from the IAT in that participants responses to different trials within one block of trials are compared, rather than performance between two different blocks of trials. Consequently, the EAST avoids the three limitations of the IAT described above. First, effects due to block order cannot occur in the EAST. Second, the EAST permits the assessment of an association for a single concept (such as the self concept). Third, the EAST allows the assessment of multiple dimensions within one task. The EAST is a modification of the Simon task (Craft & Simon, 1970) in which participants are asked to make a spatial response based on a nonspatial stimulus feature: For instance, participants are asked to press a left key when a red light is presented and a right key when a green light is presented. The red and green lights are presented either on the left or right side of a visual display. This latter feature is irrelevant and has to be ignored by the participants. Nevertheless, the responses to the red light are faster when it is presented on the left side than on the right side. Similarly, the responses to the green light are faster when it is presented on the right side as compared to the left side. Based on the original Simon task that focuses on a spatial relation between the irrelevant feature and the response, an affective variant of the Simon task was developed by implementing an affective relation between the irrelevant feature and the response (De Houwer, Crombez, Baeyens, & Hermans, 2001; De Houwer & Eelen, 1998). For instance, depending on the grammatical category of the presented word (i.e., noun or adjective), participants respond by saying either positive or negative. Similar to the original Simon effect, participants were faster in congruent trials (i.e., the connotation of the presented word matches the required response) than in incongruent trials. The EAST is a further modification and differs from the affective variant of the Simon task in that the response is intrinsically unrelated to affective valence (namely, pressing a left or a right key). However, this neutral response is extrinsically related to affective valence (De Houwer, 2003). In this task, the required classification depends on the color of the word. When the word is presented in blue or green, participants have to classify by colors (i.e., press the left key when the word is blue, and the right key when the word is green). In contrast, when the word is presented in white, participants have to classify by stimulus valence (i.e., respond with the left key when the affective valence of the word is negative, and with the right key when the word is positive). This affective classification of white words leads to an extrinsic relation between the response and affective valence (i.e., the left key becomes the bad key, and the right key becomes the good key ). Consequently, participants respond faster to a blue word (for which the required response is the right key, or goodkey ) when it is of positive valence than when it is of negative valence. And participants respond faster to a Experimental Psychology 2006; Vol. 53(2): green word (for which the required response is the left key, or bad key ) when it is negative rather than positive. The EAST can easily be adapted to measure self-esteem by administering words related to self and other in blue and green, in addition to positive and negative words in white (De Houwer, 2003, Experiment 2). In the self-esteem EAST, the faster a participant responds to a self-related word (e.g., the participant s first name) in blue (for which the required response is the right key, or good key ) compared to one in green (for which the required response is the left key, or bad key ), the higher their estimated selfesteem. Furthermore, the EAST was adapted to measure multiple implicit self-concept dimensions of personality (Teige, Schnabel, Banse, & Asendorpf, 2004). In this adaptation, self and other stimuli are presented in white and have to be discriminated according to their categorical meaning by pressing a left key when a stimulus of the category other is presented, and a right key when a stimulus of the category self is presented. That is, the left key becomes the other key, and the right key becomes the self key. Additionally, personality attributes of three different dimensions (shyness, anxiousness, and angriness) are presented in blue or green colors and have to be discriminated according to their color. An EAST score is calculated for each of the three personality dimensions by comparing the averaged reaction times between different kinds of trials (e.g., a score of implicit shyness is computed by subtracting the mean reaction time in trials in which participants have to press the self key in response to shy attributes or the other key in response to nonshy attributes from the mean reaction time in trials in which participants have to press the self key in response to nonshy attributes or the other key in response to shy attributes). Unfortunately, the results of Teige et al. (2004) are rather disappointing. The reliability of the EAST scores was very low (Cronbach s between.19 and.24), as was the convergent validity with IATs assessing the same personality dimensions (correlations around zero). In particular, the reliability was much lower than that of an EAST for assessing self-esteem (corrected split-half correlation of.65; De Houwer, 2003). However, possible reasons for these findings cannot be unequivocally identified, because the EAST for assessing multiple personality dimensions differs in three major factors from the EAST for assessing self-esteem: (a) Personality adjectives are used as stimuli, (b) the extrinsic relation is switched so that the response is extrinsically related to the categories self and other, and (c) multiple personality dimensions are assessed within one task. Thus, each of these factors, or any factor combination, might cause the low reliability. Extrinsic Simon Tasks for Assessing Anxiety In the present research, we adapted the EAST for measuring the implicit self-concept of anxiety. To be able to identify possible factors that influence reliability and validity 2006 Hogrefe & Huber Publishers

3 Schmukle & Egloff: Extrinsic Simon Tasks 151 of EAST adaptations, we developed two tasks that differ in their degree of divergence from the original EAST. The first one structurally differs from the self-esteem EAST (De Houwer, 2003) only in that we used personality adjectives as stimuli. Thus, the response was extrinsically related to anxiety versus calmness rather than to affective valence (i.e., the left key becomes the anxiety key, and the right key becomes the calmness key ). Because of this change, we removed the term affective from the task s name and called this task Extrinsic Simon Task-Anxiety (EST- Anxiety). In a second version of the EST-Anxiety, we additionally switched the extrinsic relation. Now the response was as in Teige et al. s (2004) task extrinsically related to the categories self and other rather than to anxiety and calmness (i.e., the left key becomes the self key, and the right key becomes the other key ). This response switch is a necessary condition for using the task as a measure of multiple dimensions. However, a semantic processing of the anxiety and calmness stimuli cannot be ensured with this response assignment, because these stimuli have to be classified by color rather than meaning. This limited depth of processing might be a reason for low reliability, especially if different dimensions are assessed simultaneously. Therefore, in our second task, participants did not have to classify anxiety and calmness stimuli by color but by grammatical category (adjective vs. substantive) a discrimination that was already successfully used in an affective Simon task (De Houwer & Eelen, 1998). Taken together, the present studies analyze the suitability of two EST-Anxiety versions based on either an extrinsic anxiety relation (EST-A) or an extrinsic self relation (EST-B) as indirect measures of anxiety. For this purpose, we (a) examine the reliability of these tasks by estimating their internal consistencies, (b) analyze the convergent validity with the IAT-Anxiety, and (c) explore the associations with self-reported anxiety. Study 1 Method Participants In this study, 106 students (82 women and 24 men) from Johannes Gutenberg-University Mainz participated in exchange for research participation credit. The average age of participants was 23.4 years (SD 5.5). Self-Report Measure The participants explicitly rated five anxiety items (nervous, afraid, fearful, anxious, uncertain) and five calmness items (relaxed, balanced, at ease, calm, restful) that were identical to those used in the IAT and the ESTs (i.e., Please indicate on a scale from 0 [not at all] to 5 [very high] the extent to which the following attributes apply to you ). An explicit anxiety score was computed as the mean of the five anxiety and the five reverse-scored calmness ratings. IAT-Anxiety All reaction time tasks were administered on a personal computer with the program Inquisit (Draine, 2001). In the IAT, stimuli from self (me, my, own, I, self) and other (they, your, them, you, others) categories as well as items from anxiety (nervous, afraid, fearful, anxious, uncertain) and calmness (relaxed, balanced, at ease, calm, restful) categories were presented. The IAT procedure comprises five blocks. Participants practiced the discrimination of self and other items (target discrimination) in the first block, which comprised 20 trials (each item was presented twice), by pressing a left key or a right key, respectively. The same was done for the attribute discrimination by sorting items into anxiety and calmness categories in Block 2 and for practicing the switched key assignment in Block 4 (20 items each). The critical Blocks 3 and 5 consisted of 20 practice trials and 60 test trials. In these trials, participants categorized items into two combined categories, each including the attribute and the target concept that were assigned to the same key. Attribute and target stimuli were presented alternately in a randomized order for each participant. The order of critical blocks was counterbalanced between subjects. Participants were told they would be making a series of category judgments. In each trial, a stimulus word was displayed in the center of a computer screen. Category labels were displayed on the left and right sides of the window. Participants used the letter A on the left side of the keyboard and the number 5 on the right-side numeric keypad for their responses. They were told, Please try to be as accurate as possible, while also going as quickly as possible. If your selection is incorrect, you will see a red X. To continue with the next judgment, you must make the correct selection. Participants were told to keep their index fingers on the A and 5 keys throughout the experiment to facilitate responding quickly. An intertrial interval of 150 ms was used. The computer recorded elapsed time between the start of each stimulus presentation and the correct response. Mean latencies and error rates were displayed after each block. IAT data were treated following the conventional procedure (the so-called log measure) outlined by Greenwald et al. (1998) as well as by using an improved scoring algorithm (the so-called D measure) described by Greenwald, Nosek, and Banaji (2003). For the log measure (a) only test trials were used, and the first two of those trials were dropped; (b) error trials were included in the analysis by using the latency until the correct response was given; (c) trials with latencies less than 300 ms or greater than 3,000 ms were recoded to 300 ms or 3,000 ms, respectively; (d) the reaction times were log transformed prior to subtracting the mean latency in test trials of the self anxiety block from the self calmness block. For the D measure (a) trials with latencies greater than 10,000 ms were eliminated; (b) error trials were included in the analysis by using the latency until the correct response was given; (c) the mean latency in critical trials of the self anxiety block was subtracted from the self calmness block, separately for test and practice trials leading to two differences; (d) these differences were divided by the 2006 Hogrefe & Huber Publishers Experimental Psychology 2006; Vol. 53(2):

4 152 Schmukle & Egloff: Extrinsic Simon Tasks individual-respondent standard deviations of reaction times in test or practice trials, respectively, leading to two scores; (e) the IAT effect for anxiety was computed as the average of these two scores (weighted for the number of test and practice trials). Thus, for both measures, the higher the IAT effect is, the more anxious the estimated implicit selfconcept of a participant is. EST-A (EST-Anxiety With Extrinsic Anxiety Relation) In the EST-Anxiety, the same stimuli as in the IAT-Anxiety were used. These stimuli were successively presented in the center of the computer screen with order being randomized for each participant. Items from the categories anxiety and calmness were presented in white font, and items from the categories self and other in colored font (Table 1 gives an overview of this task). Participants were instructed to classify the items depending on the meaning or color of the word presented. When the word was white, the meaning of the word was crucial. The participants had to respond with the left key (A) when it was an anxiety stimulus, and with the right key (5), when it was a calmness stimulus. When the word was colored, the participants had to press the key on the basis of the color of the word. Half of the participants were instructed to press the left key when the word was blue and the right key when the word was green. The other participants received the reverse of these instructions. To maximize similarity with the EAST (De Houwer, 2003), we used an identical form of trial presentation and error handling. Thus, the EST-A differed in some presentational details from the IAT-Anxiety. In particular, in Table 1. Critical blocks of the two Extrinsic Simon Task- Anxiety (EST-Anxiety) versions Response Key Assignment EST-A (extrinsic anxiety relation) EST-B (extrinsic self relation) Left Key Right Key Left Key Right Key anxiety calmness self other blue a green a adjective a substantive a Examples of Trials and Correct Responses EST-A (extrinsic anxiety relation) EST-B (extrinsic self relation) anxious [white] b me calm [white] they me[green] ANXIETY self[blue] ANXIOUS they[blue] CALM your[green] CALMNESS Note. The examples in the lower four rows are critical trials. For a complete list of stimuli, see the Method section of Study 1. a The key assignment of colors and grammatical categories was counterbalanced between subjects. For reasons of simplicity, Table 1 displays only one condition. b The color in brackets indicates the color in which the word is presented. Experimental Psychology 2006; Vol. 53(2): each trial, a white fixation cross appeared for 500 ms before the stimulus word was displayed, and category labels were not displayed on the screen during the trial. If participants selection was incorrect, a red X appeared for 400 ms. However, the participant was not expected to make the correct response to continue with the task. An intertrial interval of 1,200 ms was used. The computer recorded elapsed time between the start of each stimulus presentation and the first response (regardless of whether it was correct or not). The EST procedure comprised six blocks. The first two blocks were practice blocks in which either only white or colored words were presented. The last four blocks were test blocks, in which both white and colored words were presented. In each test block, 30 trials were presented in random order. The 30 trials consisted of the 10 anxiety and calmness stimuli in white font, the 10 self and other stimuli in green font, and the 10 self and other stimuli in blue font. Within test blocks, trials consisting of colored words (i.e., words of the categories self and other) were the critical trials. We estimated the implicit self-concept of anxiety by comparing reactions in the following four different kinds of critical trials: (a) self/anxiety trials, in which self stimuli were assigned to the left key (i.e., the anxiety key ); (b) other/calmness trials, in which other stimuli were assigned to the right key (i.e., the calmness key ); (c) self/calmness trials, in which self stimuli were assigned to the right key (i.e., the calmness key ); (d) other/anxiety trials, in which other stimuli were assigned to the left key (i.e., the anxiety key ). The difference between averaged reactions in self/calmness and other/anxiety trials and those in self/anxiety and other/calmness trials is an indicator of the implicit selfconcept of anxiety (e.g., if a participant has a more anxious self-concept, then it is easier to respond to the last two kinds of trials than to the first two kinds of trials, leading to a positive value of the index). Following De Houwer (2003), we additionally estimated an index of the implicit self-concept of anxiety based only on trials with self stimuli. This index was computed as the difference between mean reaction times in self/calmness trials and those in self/anxiety trials. This scoring aims at assessing the pure association between the self-concept and anxiety rather than the association of the self-concept relative to the association of the concept of others. For each of the two kinds of indices described above, three different EST scores were computed. The first score was based on log-transformed response latencies (log score): (a) Trials with incorrect responses were excluded; (b) trials with latencies less than 300 ms or greater than 3,000 ms were recoded to 300 ms or 3,000 ms, respectively; (c) resulting values were log transformed; and (d) the mean latency in self/anxiety and other/calmness trials was subtracted from self/calmness and other/anxiety trials. The second score was based on error frequencies (error score): The mean error rate in self/anxiety and other/calmness trials was subtracted from self/calmness and other/ anxiety trials. These two scores were identical to those reported by De Houwer (2003) Hogrefe & Huber Publishers

5 Schmukle & Egloff: Extrinsic Simon Tasks 153 Additionally, we computed an EST score by using the improved scoring algorithm of the IAT (D measure; Greenwald et al., 2003): (a) Trials with latencies greater than 10,000 ms were eliminated; (b) error trials were included in the analysis by using the mean latency in correct trials plus a 600 ms error penalty (c) the mean latency in self/ anxiety and other/calmness trials was subtracted from self/ calmness and other/anxiety trials; (d) these differences were divided by the individual-respondent standard deviations of reaction times in critical trials. EST-B (EST-Anxiety With Extrinsic Self Relation) This EST version is displayed in the right half of Table 1. In this version, the discrimination during critical trials was based on the grammatical category of the stimulus (substantive vs. adjective) rather than on the color. For this purpose, we used substantives from the categories anxiety (nervousness, discomposure, fear, anxiety, uncertainty) and calmness (recreation, composure, ease, calmness, relaxation) as additional stimuli. During the task, items from the categories self and other were presented in lowercase letters, whereas adjectives and substantives from the categories anxiety and calmness were presented in uppercase letters. Stimuli order was randomized for each participant. The task was to classify the word based on meaning or grammar. When the word was presented in lowercase letters, the meaning of the word was crucial. The participants had to respond with the left key (A) when the word was a stimulus of the category self, and with the right key (5) when it was a stimulus of the category other. When the word was presented in uppercase letters, the participants had to press the key on the basis of the grammatical category of the word. Half of the participants were instructed to press the left key when the word was an adjective and the right key when the word was a substantive. The other participants received the reverse of these instructions. As in the EST-A, there were six blocks altogether, two practice blocks and four test blocks that consisted of the same number of trials compared to the EST-A. However, in contrast to the EST-A, the presentational form of the EST-B followed the IAT-Anxiety (see the IAT-Anxiety subsection above). We used this form because pretests showed that the EST-B was by far more difficult than the EST-A. Some participants even had problems executing the task properly when it was in the presentational form of the EST-A. Using the presentational form of the IAT made the task much easier because the category labels were then presented on the left and right sides of the screen. We estimated the implicit self-concept of anxiety by comparing reactions in the following four different kinds of critical trials: (a) calmness/self trials, in which calmness stimuli were assigned to the left key (i.e., the self key ); (b) anxiety/other trials, in which anxiety stimuli were assigned to the right key (i.e., the other key ); (c) anxiety/ self trials, in which anxiety stimuli were assigned to the left key (i.e., the self key ); (d) calmness/other trials, in which calmness stimuli were assigned to the right key (i.e., the other key ). The difference between averaged reactions in calmness/ self and anxiety/other trials and those in anxiety/self and calmness/other trials is an indicator of the implicit selfconcept of anxiety (e.g., if a participant has a more anxious self-concept, then it is easier to respond to the last two kinds of trials than to the first two kinds of trials). Additionally, we computed a second index based only on critical self trials by computing the difference between mean reactions in calmness/self trials and those in anxiety/self trials. As in the EST-A, three different scores (the log score, the error score, and the D score) were computed for each of the two indices. As in the IAT-Anxiety, for the calculation of the D score the reaction times of correct responses were used in error trials. Procedure The participants first responded to the self-report measure of anxiety, then to the IAT-Anxiety, followed by the EST- B and finally the EST-A. Results Descriptive Statistics and Reliabilities For the EST-A (EST-Anxiety version with extrinsic anxiety relation), the incorrect response rate for the critical trials was M 3.28% (SD 3.23). The mean reaction times were M 680 ms (SD 145) for self/anxiety trials, M 669 ms (SD 140) for other/calmness trials, M 699 ms (SD 146) for other/anxiety trials, and M 696 ms (SD 171) for self/calmness trials. The corresponding mean error rates were 2.36%, 2.26%, 4.53%, and 3.96%, respectively. This reaction pattern led to EST scores that were somewhat above zero (see Table 2), indicating that on average, participants responded faster and more correctly in self/anxiety and other/calmness trials than in other/anxiety and self/calmness trials. Next, we estimated the split-half reliability of the EST- A based on two different methods of sorting trials into test halves. First, in line with De Houwer (2003) and Teige et al. (2004), we sorted the trials block-wise and calculated separate EST scores for the first two blocks and for the last two blocks. Afterward, we computed Cronbach s of the two test-half scores. Second, we sorted the trials by using an odd-even method. Trials were consecutively numbered in chronological order separately for each kind of critical trial. Then, odd trials were sorted into one test half, and even trials were sorted into the other test half. Again, Cronbach s of the two test-half scores was computed. An advantage of the odd-even method is that the trials of the two test halves are equally distributed over the task. Thus, the odd-even method is less affected than the block-wise method by response changes during the task. As Table 2 shows, the reliability estimates varied within a range of.17 to.68. The highest internal consistencies of above.60 were obtained for the log and D scores. For these scores, the reliability estimates were consistently higher when they were based on the odd-even method rather than the blockwise method. For the EST-B (EST-Anxiety version with extrinsic self relation), we observed an error rate of M 3.67% (SD 4.10). The mean reaction times were M 937 ms (SD 2006 Hogrefe & Huber Publishers Experimental Psychology 2006; Vol. 53(2):

6 154 Schmukle & Egloff: Extrinsic Simon Tasks Table 2. Descriptive statistics and reliabilities of the different measures (Study 1) Experimental Psychology 2006; Vol. 53(2): Cronbach s M SD block-wise odd-even EST-A (extrinsic anxiety relation) Log score Error score D score Log score Error score D score EST-B (extrinsic self relation) Log score Error score D score Log score Error score D score IAT-Anxiety Log score D score Self-reported anxiety a Note. N 106. EST Extrinsic Simon Task; IAT Implicit Association Test. The log score is based on logarithmic reaction times, the error score is based on error frequencies, and the D score is given by the improved scoring algorithm. a Cronbach s based on items. 219) for anxiety/self trials, M 968 ms (SD 238) for calmness/other trials, M 937 ms (SD 196) for anxiety/other trials, and M 930 ms (SD 230) for calmness/ self trials. The corresponding error rates were 3.63%, 4.43%, 2.83%, and 3.77%, respectively. Thus, reaction times were much higher than in the EST-A, reflecting that categorizing by grammatical category was more time-consuming than categorizing by color. Although Table 2 indicates that most of the mean values were negative, only the D score based on all critical trials was significantly below zero. The reliability estimates were somewhat lower compared to the EST-A and ranged from.12 to.55. The estimates did not differ as strongly as for the EST-A between the block-wise method and the odd-even method. For the IAT-Anxiety, the mean reaction times were M 941 ms (SD 225) for the self anxiety block and M 828 ms (SD 183) for the self calmness block. Thus, as shown in Table 2, the mean IAT effects were negative, which resembles results of previous studies with the IAT-Anxiety (e.g., Egloff & Schmukle, 2002, 2003). As expected, the IAT effect differed between whether the self anxiety block was presented first, M 0.23 (SD 0.26), or whether the self calmness block was given first, M 0.31 (SD 0.26; D score). However, the difference did not reach the conventional level of significance: t(104) 1.59, p.12. The incorrect response rate for the critical trials was M 3.76% (SD 3.51). Next, we estimated the reliability of the IAT-Anxiety by applying the scoring algorithms separately to two mutually exclusive subsets of the IAT s combined-task trials. We divided the 160 trials into the two subsets by using the odd-even method, ensuring that the same number of target and attribute stimuli were assigned to each test half (i.e., the trials 1,4,5,8,9,...,156, 157, 160 were assigned to the first subset, and the trials 2, 3, 6, 7, 10, 11,...,155, 158, 159 to the second subset). The internal consistency of these two test halves was satisfactory (Cronbach s.76 and.77, respectively, for the two scoring algorithms). These results resemble those found in other studies (e.g., Egloff & Schmukle, 2002). Self-reported anxiety showed a very good internal consistency of.87. Intercorrelations Correlations between the different measures are shown in Table 3. Note that the correlations of the D scores with the log scores and with the error scores are a logical consequence of the calculation of the D scores. We observed surprisingly low correlations between the two EST-Anxiety versions. Only 5 of the 36 correlations reached marginal significance. Likewise, only small correlations occurred between the EST-Anxiety and the IAT-Anxiety. The highest associations were those between the log scores and D scores of the EST-A based on critical self trials and the IAT-Anxiety (r.18 to.20). Implicit and explicit measures of anxiety did not correlate with each other at all. This mirrors previous studies that found no or only very low associations between the IAT-Anxiety and selfreported anxiety (e.g., Egloff & Schmukle, 2002, 2003). Discussion The reliability estimates of the EST-Anxiety strongly varied between the different versions, scores, and splitting methods. Overall, most of the internal consistency coefficients were not sufficient for using the EST-Anxiety as an indirect test aiming at the assessment of individual differences. With the exception of reaction time based EST-A scores, all internal consistencies were below.60. Additionally, reliability estimates of above.60 were only observed when the trials were sorted into test halves by means of an odd-even method rather than a block-wise method. Differences between these two methods might be a consequence of response changes during the task, which might have also attenuated previously reported block-wise reliability estimates of the EAST (De Houwer, 2003; Teige et al., 2004). However, before drawing far-reaching conclusions, this result should first be replicated in a second study. Furthermore, scores of the two EST-Anxiety versions were not associated. Nevertheless, it was encouraging that there were at least marginally significant associations between the EST-A and the IAT-Anxiety. A weakness of Study 1 was that the tasks were presented in a constant order for each participant (i.e., first the IAT, then the EST-B, and lastly the EST-A). However, when several implicit tests are performed within one session, it 2006 Hogrefe & Huber Publishers

7 Schmukle & Egloff: Extrinsic Simon Tasks 155 Table 3. Correlations between the different measures (Study 1) EST-A (extrinsic anxiety relation) 1 Log score 2 Error score.10 3 D score.85***.51*** 4 Log score.76***.07.66*** 5 Error score.20*.73***.46***.25** 6 D score.68***.29**.74***.90***.48*** EST-B (extrinsic self relation) 7 Log score Error score D score ***.37*** 10 Log score *.85***.18.75*** 11 Error score ***.33*** D score ***.42***.82***.87***.45*** IAT-Anxiety 13 Log score * D score *** 15 Self-reported anxiety Note. N 106. EST Extrinsic Simon Task; IAT Implicit Association Test. The log score is based on logarithmic reaction times, the error score is based on error frequencies, and the D score is given by the improved scoring algorithm. p.10. * p.05. ** p.01. *** p.001 (two-tailed) Hogrefe & Huber Publishers Experimental Psychology 2006; Vol. 53(2):

8 156 Schmukle & Egloff: Extrinsic Simon Tasks may be that the measurement error is higher for the later ones due to a more superficial processing of stimuli. Consequently, Study 1 cannot rule out that the reliability of the ESTs was lower compared to that of the IAT just because the ESTs were presented at the end of the session. For this reason, in Study 2, the two EST versions were presented at the beginning of the session, before the IAT. Study 2 Method Participants In this study, 81 students (56 women and 25 men) from Johannes Gutenberg-University Mainz participated in exchange for research participation credit or for monetary compensation of 5. The average age of participants was 24.4 years (SD 5.7). Measures The implicit and explicit measures were identical to those in Study 1. In addition to these measures, we used the trait form of the State-Trait Anxiety Inventory (STAI; Spielberger, Gorsuch, & Lushene, 1970; German version: Laux, Glanzmann, Schaffner, & Spielberger, 1981). This widely used questionnaire contains 20 items that assess enduring symptoms of anxiety on a 4-point Likert scale from 1 (almost never) to 4 (almost always). Procedure The participants first responded to the two different EST- Anxiety versions, then to the IAT-Anxiety, and lastly to the self-report measures of anxiety. The order of the two EST-Anxiety versions was counterbalanced, and participants were randomly assigned to one of the two order conditions. Results Descriptive Statistics and Reliabilities For the EST-A, the incorrect response rate for the critical trials was M 4.01% (SD 4.08%). The mean reaction times were M 701 ms (SD 152) for self/anxiety trials, M 686 ms (SD 145) for other/calmness trials, M 710 ms (SD 168) for other/anxiety trials, and M 713 ms (SD 167) for self/calmness trials. The corresponding error rates were 3.40%, 3.27%, 4.75%, and 4.63%, respectively. This reaction pattern was similar to that in Study 1 and led to EST scores that were somewhat above zero (see Table 4). The reliability estimates varied within a range of.19 and.64. The highest estimates of.60 or above were obtained for the error scores and D scores when they were based on critical self trials only. In contrast to Study 1, this time the reliability estimates tended to be higher for the block-wise method compared to the odd-even method. Furthermore, the internal consistency estimates of the error Experimental Psychology 2006; Vol. 53(2): scores were higher than in Study 1, which might be due to the somewhat higher incorrect response rate in Study 2. For the EST-B, we observed an error rate of M 4.65% (SD 4.69). The mean reaction times were again similar to Study 1: M 963 ms (SD 254) for anxiety/self trials, M 965 ms (SD 252) for calmness/other trials, M 928 ms (SD 220) for anxiety/other trials, and M 938 ms (SD 255) for calmness/self trials. The corresponding error rates were 3.89%, 5.56%, 4.07%, and 5.06%, respectively. Table 4 shows that with the exception of the D score based on all critical trials, all average scores did not differ significantly from zero. The reliability estimates were quite similar to Study 1 and ranged from.19 to.55. For the IAT-Anxiety, we observed mean reaction times of M 924 ms (SD 206) for the self anxiety block and M 816 ms (SD 169) for the self calmness block. As in Study 1, the mean IAT effects were again about one standard deviation below zero (see Table 4). The order of critical blocks in the IAT did not influence the IAT effect (D score), t(79) 0.32, p.76. The mean IAT effect amounted to M 0.26 (SD 0.32) when the self anxiety block was presented first, and to M Table 4. Descriptive statistics and reliabilities of the different measures (Study 2) Cronbach s M SD block-wise odd-even EST-A (extrinsic anxiety relation) Log score Error score D score Log score Error score D score EST-B (extrinsic self relation) Log score Error score D score Log score Error score D score IAT-Anxiety Log score D score Self-reported anxiety Rating of stimuli a STAI-Trait a Note. N 81. EST Extrinsic Simon Task; IAT Implicit Association Test; STAI State-Trait Anxiety Inventory. The log score is based on logarithmic reaction times, the error score is based on error frequencies, and the D score is given by the improved scoring algorithm. a Cronbach s based on items Hogrefe & Huber Publishers

9 Schmukle & Egloff: Extrinsic Simon Tasks (SD 0.36) for the opposite block order, with a mean incorrect response rate of M 4.63% (SD 4.69). The internal consistencies of the IAT effects were above.80 and thus slightly better than in Study 1. Self-reported anxiety assessed both by the rating of the stimuli and by the STAI-Trait showed very good internal consistencies of above.90. Intercorrelations Table 5 displays the correlations between the different measures of Study 2, which resemble those of Study 1. While the scores between the two EST versions were unrelated, the EST-A was correlated with the IAT-Anxiety in the range of between.20 and.30. These associations were not affected by statistically partialing out the explicit ratings. In contrast to the EST-A, the EST-B was unrelated to the IAT. A significant association between the implicit measures and self-reported anxiety was only observed for the IAT. Discussion Study 2 was conducted to test the hypothesis that the lower reliability of the EST-Anxiety compared to the IAT-Anxiety in Study 1 was a consequence of the presentational order. The results showed that when the EST-Anxiety was given before the IAT rather than after the IAT, the reliability did not increase for either of the two EST versions. Additionally, the speculation raised in Study 1 that blockwise sorting of trials attenuates the reliability estimates of the EST could also not be confirmed. In Study 2, the blockwise method even tended to result in higher reliability estimates compared to the odd-even method. That is, the block-wise method of estimating reliability does not seem to be a reason for the low to moderate reliability estimates of the EAST and its adaptations in previous studies (De Houwer, 2003; Teige et al., 2004). Furthermore, both Study 1 and Study 2 revealed a similar correlational pattern of the measures. General Discussion The two studies we conducted explored the suitability of the EAST (De Houwer, 2003) for measuring the implicit self-concept of anxiety. For this purpose, we developed two different adaptations of the EAST for the assessment of anxiety. The first task, the EST-Anxiety with extrinsic anxiety relation (EST-A), was very similar to an EAST for assessing self-esteem (De Houwer, 2003). It differed only in that the response was extrinsically related to the categories anxiety and calmness rather than to affective valence. However, for assessing multiple personality dimensions within one task, the original EAST had to be changed more fundamentally. In particular, the extrinsic relation had to be switched to the self-other dimension (Teige et al., 2004). For this reason, we developed a second version of the EST-Anxiety (EST-B). In this task, (a) the response was extrinsically related to the categories self and other rather than to anxiety or affective valence, and (b) the discrimination was based on the grammatical category rather than on the color of the word to ensure a semantic processing. The internal consistencies ranged from.12 to.68 across different studies, EST-versions, scoring algorithms, and reliability estimation methods. A closer examination of the results suggests the following: 1. The reliability of the two EST-Anxiety versions was similar in Study 1 and Study 2 (mean internal consistency estimates of.44 vs..42). Thus, on average, the sequence of tasks did not influence the reliability. 2. The method of reliability estimation (block-wise vs. odd-even) did not influence the estimates (means of.43 for both methods). 3. The EST-A showed on average a somewhat higher reliability than the EST-B (means of.46 vs..40). Furthermore, reliability estimates of above.60 were exclusively observed for the EST-A. 4. The reliability of the D scores tended to be higher than for the error and log scores (mean estimates of.47,.43, and.39, respectively). Additionally, the reliability of scores that were based on all critical trials was slightly higher than the reliability of those based on critical self trials only (means of.46 vs..40). However, the latter scores were based on only 40 trials, whereas the former were based on 80 trials. For comparison purposes, we corrected the reliability of the scores based on critical self trials with the Spearman-Brown formula assuming 80 critical self trials. This led to an average reliability estimate of.57. Thus, after correcting for test length, scores based on critical self trials are more reliable than those based on all critical trials. This indicates that the critical self trials might constitute a separate scale and should not be put together with the other critical trials into one score. The fact that the critical self trials are more homogeneous might reflect the theoretical advantage of indices based solely on critical self trials in that they measure the pure association between the self concept and anxiety independent of the other concept. Nevertheless, it has to be considered that indices based only on critical self trials have the methodological limitation that the contrast self/ anxiety versus self/calmness either is confounded by response time differences in pressing the anxiety key versus the calmness key (EST-A) or is confounded by response time differences in categorizing substantives versus adjectives (EST-B). Overall, compared to the Spearman-Brown corrected split-half correlation of.65 for a self-esteem EAST (De Houwer, 2003), the average internal consistencies in our studies were lower. This difference becomes even more meaningful when considering that the EAST score was based on 24 critical trials, whereas the EST-Anxiety scores were based on 80 critical trials (and accordingly, 40 critical trials when the scores were based solely on self trials). However, the internal consistencies of both EST-Anxiety versions outperformed those reported for a multiple personality traits EAST (mean internal consistency of.21, with 40 critical trials per dimension; Teige et al., 2004) and a pictorial spider EAST (internal consistencies of.57 for scoring based on reaction times and Hogrefe & Huber Publishers Experimental Psychology 2006; Vol. 53(2):

10 158 Schmukle & Egloff: Extrinsic Simon Tasks Table 5. Correlations between the different measures (Study 2) EST-A (extrinsic anxiety relation) 1 Log score 2 Error score.37** 3 D score.86***.69*** 4 Log score.71***.34**.66*** 5 Error score.20.74***.50***.25* 6 D score.63***.55***.77***.86***.63*** EST-B (extrinsic self relation) 7 Log score Error score *.18.26* 9 D score ***.52*** 10 Log score ***.15.55*** 11 Error score ***.33** D score ***.42***.69***.83***.42*** IAT-Anxiety 13 Log score *.22*.17.23* D score.20.26*.29**.23*.21.27* *** Self-reported anxiety 15 Rating of stimuli **.32** 16 STAI-Trait *.28*.85*** Note. N 81. EST Extrinsic Simon Task; IAT Implicit Association Test. STAI State-Trait Anxiety Inventory. The log score is based on logarithmic reaction times, the error score is based on error rates, and the D score is given by the improved scoring algorithm. p.10. * p.05. ** p.01. *** p.001 (two-tailed). Experimental Psychology 2006; Vol. 53(2): Hogrefe & Huber Publishers

11 Schmukle & Egloff: Extrinsic Simon Tasks 159 for scoring based on error rates; Huijding & de Jong, in press). In addition to exploring the reliability of the tasks, we also analyzed the convergent validity of the EST-Anxiety with the IAT-Anxiety and looked at the associations with self-reported anxiety. With respect to convergent validity, the EST-A was moderately correlated with the IAT-Anxiety, whereas neither of the EST versions among themselves nor the EST-B and the IAT-Anxiety were associated. Similarly, both EST-Anxiety versions were dissociated from explicit anxiety ratings. Taken together, our studies suggest three different conclusions: 1. The higher internal consistency of the self-esteem EAST (De Houwer, 2003) as compared to the EST-A suggests that an extrinsic relation to affective valence leads to more reliable results than an extrinsic relation to a personality dimension such as anxiety. 2. The higher internal consistency of the EST-A compared to the EST-B and to the multiple personality traits EAST (Teige et al., 2004) in combination with an at least moderate convergent validity with the IAT-Anxiety suggests that an extrinsic relation to affective valence or to a personality trait leads to more reliable and more valid measures than an extrinsic relation to self and other. 3. The particularly low reliabilities of the multiple personality traits EAST (Teige et al., 2004) suggest that the simultaneous assessment of different traits further lowers the reliability and the validity. However, the overall reliability of both EST-Anxiety versions was not sufficient for using the EST-Anxiety as an indirect test aimed at the assessment of individual differences. In particular, the internal consistency coefficients were lower than those observed for the IAT-Anxiety (the internal consistency of the IAT, however, might be overestimated due to method-specific variance; see Back, Schmukle, & Egloff, 2005; Mierke & Klauer, 2003). On the other hand, although the internal consistencies of extrinsic Simon tasks are only moderate, they were higher than those observed for many other indirect measures (see Banse, 2001, for affective priming; Bosson, Swann, & Pennebaker, 2000, for several implicit self-esteem tasks; Nosek & Banaji, 2001, for the go/no-go association test; Schmukle, in press, for the dot-probe paradigm). Thus, it seems worthwhile modifying the extrinsic Simon paradigm in order to enhance its psychometric properties. First, the reliability might be increased by lengthening the task. For example, as long as there are no changes in participants responses to later trials, doubling the number of critical trials would lead to reliabilities of about.75 for the D score of the EST-A. Our second suggestion aims at experimentally increasing the error rate. This might be a successful strategy, because in Study 2 in which the error rates were comparatively high, quite high reliability estimates also emerged for error scores. Thus, one improvement could be to use a moving response window, which increases the error rate by forcing participants to respond faster than they ordinarily would. For subliminal semantic priming, this technique indeed increased the reliability of scores based on error rates (Draine & Greenwald, 1998). A third possible improvement consists of increasing the number of primary task trials in the critical blocks (i.e., to decrease the proportion of critical trials). 1 Thus, the primary task becomes something like the default, which might strengthen the extrinsic relation and increase the sensitivity of the EST measures. Results of the present studies further suggest that in order to develop a more reliable and valid Simon task for assessing personality traits, implementing an extrinsic relation to a personality trait rather than to the categories self and other seems to be more promising (although the unexpected positive effect scores are in need of explanation). Unfortunately, this would imply that one of the most prominent features of the extrinsic Simon paradigm the possibility of simultaneously assessing multiple personality dimensions within one task could not be realized. Acknowledgments This research was supported by Deutsche Forschungsgemeinschaft (German Research Foundation) Grant EG 143/ 2-1. We thank Mitja Back, Jennifer Dickes, and Brigitte Köhler for their help with data collection; and Mitja Back and Judith Kappesser for helpful comments on an earlier version of this article. References Back, M. D., Schmukle, S. C., & Egloff, B. (2005). Measuring task-switching ability in the Implicit Association Test. Experimental Psychology, 52, Banse, R. (2001). Affective priming with liked and disliked persons: Prime visibility determines congruency and incongruency effects. Cognition & Emotion, 15, Bosson, J. K., Swann, W. B., & Pennebaker, J. W. (2000). Stalking the perfect measure of implicit self-esteem: The blind men and the elephant revisited? Journal of Personality & Social Psychology, 79, Craft, J. L. & Simon, J. R. (1970). Processing symbolic information from a visual display: Interference from an irrelevant directional cue. Journal of Experimental Psychology, 83, De Houwer, J. (2003). The Extrinsic Affective Simon Task. Experimental Psychology, 50, De Houwer, J., Crombez, G., Baeyens, F., & Hermans, D. (2001). On the generality of the affective Simon effect. Cognition & Emotion, 15, De Houwer, J. & Eelen, P. (1998). An affective variant of the Simon paradigm. Cognition & Emotion, 12, Draine, S. C. (2001). Inquisit (Version 1.32) [Computer software]. Seattle, WA: Millisecond Software. Draine, S. C. & Greenwald, A. G. (1998). Replicable unconscious semantic priming. Journal of Experimental Psychology: General, 127, Egloff, B. & Schmukle, S. C. (2002). Predictive validity of an 1 We thank Klaus Rothermund for raising this issue Hogrefe & Huber Publishers Experimental Psychology 2006; Vol. 53(2):