No prior entry for threat-related faces: Evidence from temporal order judgments

Transcription

1 No prior entry for threat-related faces 1 1 Running head: NO PRIOR ENTRY FOR THREAT-RELATED FACES No prior entry for threat-related faces: Evidence from temporal order judgments 5 6 Antonio Schettino 1,2, Tom Loeys 3, & Gilles Pourtois Department of Experimental-Clinical and Health Psychology, Ghent University, Ghent, Belgium 2 Institute of Psychology I, University of Leipzig, Leipzig, Germany 3 Department of Data Analysis, Ghent University, Ghent, Belgium Corresponding author: Antonio Schettino Institute of Psychology I University of Leipzig Seeburgstraße Leipzig Germany Phone: +49 (0) antonio.schettino@uni-leipzig.de

2 No prior entry for threat-related faces Abstract Previous research showed that threat-related faces, due to their intrinsic motivational relevance, capture attention more readily than neutral faces. Here we used a standard temporal order judgment (TOJ) task to assess whether negative (either angry or fearful) emotional faces, when competing with neutral faces for attention selection, may lead to a prior entry effect and hence be perceived as appearing first, especially when uncertainty is high regarding the order of the two onsets. We did not find evidence for this conjecture across five different experiments, despite the fact that participants were invariably influenced by asynchronies in the respective onsets of the two competing faces in the pair, and could reliably identify the emotion in the faces. Importantly, by systematically varying task demands across experiments, we could rule out confounds related to suboptimal stimulus presentation or inappropriate task demands. These findings challenge the notion of an early automatic capture of attention by (negative) emotion. Future studies are needed to investigate whether the lack of systematic bias of attention by emotion is imputed to the primacy of a non-emotional cue to resolve the TOJ task, which in turn prevents negative emotion to exert an early bottom-up influence on the guidance of spatial and temporal attention Keywords: temporal order judgment; visual prior entry; threat-related faces; emotion; attention

3 No prior entry for threat-related faces No prior entry for threat-related faces: Evidence from temporal order judgments Introduction Results obtained from a variety of experimental paradigms suggest that, under specific circumstances, negative emotional stimuli may receive prioritized access to awareness by biasing perceptual and attentional processes [1,2,3,4,5]. In variants of the Stroop task, for instance, naming the color of a word is slower when the stimulus carries a negative emotional meaning, even though this semantic feature is task-irrelevant [6,7,8]. Similarly, in visual search tasks participants are usually faster at detecting negative emotional targets embedded in an array of neutral distracters [3,9,10]. Furthermore, the well-known deficit in perceiving the second of two targets presented rapidly one after another among a stream of distracter items (attentional blink; see Refs ) is reduced if the second target carries a negative emotional meaning [13,14,15], or prolonged if the first target is (highly) arousing [16,17,18,19]. Finally, studies using spatial cueing tasks have shown that emotion-laden stimuli facilitate the processing of (non-emotional) targets subsequently presented at the same location, consistent with the assumption of a rapid orienting of attention towards these (task-irrelevant) stimuli, as opposed to neutral ones [20,21,22,23,24,25,26,27]. Taken together, these findings suggest that motivationally relevant stimuli (including negative facial expressions) can exert a strong modulatory influence on attentional control processes. However, still little is known about how these stimuli are initially prioritized by dedicated attentional control systems, mainly because the initial attentional orienting was not directly measured in these earlier studies. Visual search, spatial cueing, or attentional blink tasks, in fact, require participants to quickly engage, disengage, and reallocate attention towards different competing stimuli. Therefore, these paradigms are not suited to titrate changes in the initial

4 No prior entry for threat-related faces allocation of attention towards emotional vs. neutral stimuli [28]. By contrast, temporal order judgment (TOJ) tasks provide a more direct, sensitive, and accurate measure of attentional capture [29,30,31,32]. In a typical TOJ task, attention is oriented either to the left or the right side of fixation, and participants have to judge which of two competing stimuli, displayed on the left and right at various stimulus onset asynchronies (SOAs), was presented first. Because attention accelerates sensory processing [33,34], the stimulus appearing on the attended location is processed faster and, as a consequence, its onset is perceived as occurring first (visual prior entry; see Refs. 35,36,37). To date, two studies already used TOJ tasks to assess whether emotional faces could lead to a prior entry effect when competing with neutral faces. In their study, Fecica & Stolz [38] presented schematic neutral, happy, or angry faces -- separated by SOAs of 0, 17, 34, or 100 ms - - on the left and right side of fixation, and asked participants to judge the location of the stimulus that appeared first. Results showed that, in conditions of high uncertainty (i.e., at short as opposed to long SOAs), happy and angry faces were consistently perceived as appearing first compared to neutral faces. Moreover, a stronger prior entry effect was observed for happy relative to angry faces. This latter result is at variance with the well-known negativity bias for threatening stimuli [9,23,39,40] and might ultimately be explained, at least in part, by the use of a small number of stimuli (i.e., three schematic faces consistently repeated across trials) which may have introduced systematic attentional biases based on the fast processing of specific lowlevel perceptual features [41, 42; but see 43]. West, Anderson, & Pratt [44] conducted several experiments using the TOJ task to investigate whether motivationally significant stimuli could be prioritized over neutral ones. First, they reported a prior entry effect for schematic upright (neutral) faces when competing with inverted

5 No prior entry for threat-related faces schematic faces, providing evidence for a bias in the early allocation of attention towards these biologically relevant stimuli. Moreover, they found that this initial attentional deployment was influenced by the emotional content of the faces (i.e., schematic angry faces were prioritized over neutral faces), and was further enhanced by the use of realistic photographs of angry faces. However, in this study alike, a limited number of face stimuli was used (i.e., four angry and four neutral identities). Therefore, based on these earlier studies, it remains unclear whether the negative emotional facial expression per se, or rather uncontrolled perceptual factors, led to a differential early allocation of attention towards these emotion-laden stimuli. In the present study we used a large set of realistic photographs of faces (extensively validated in the literature) and assessed whether negative emotional facial expressions could lead to a prior entry effect when competing with neutral faces. Importantly, to overcome any low-level perceptual confound, on each and every trial we presented participants with a pair of faces (with a variable SOA between their respective onsets) that were always visually dissimilar, both in terms of identity and facial expression (i.e., either neutral or emotional). The rationale of this manipulation is that, across trials, visual dissimilarity between the two competing faces is always present and variable -- and thus uninformative -- and, accordingly, it cannot implicitly be used by participants as a distinctive visual cue to decide which of the two faces appeared first [27,45,46]. In these conditions, presumably, only the differential emotional content of the face would influence perceptual judgments. Furthermore, to verify that the emotional facial expressions were recognized as such, at the end of the experiment we asked participants to rate the emotion intensity of each and every face stimulus used during the main TOJ task. The main goal of our study was to evaluate whether negative (threat-related) emotional faces were processed faster

6 No prior entry for threat-related faces than neutral faces (see Ref. 4), thereby showing prior entry consistent with the assumption of early attentional capture. Experiment 1 Ethics statement All the experiments were approved by the ethics committee of the Faculty of Psychological and Educational Sciences, Ghent University. All participants were required to give written informed consent prior to their participation. Participants Thirty-seven undergraduate psychology students of Ghent University participated in the study in exchange of course credits. All volunteers were native Dutch speaking, right-handed, had normal or corrected-to-normal vision, with no history of neurological or psychiatric disorders. The data of five participants were excluded from subsequent analyses due to abnormal psychometric functions in at least one experimental condition [37,47,48], indicating that their performance was not influenced by the main SOA manipulation (see below). Thus, the final sample consisted of 32 participants (27 women, mean age 19 years, range 18-22). Stimuli We used pairs of grayscale photographs of ten different individuals (four women) selected from the standardized Ekman database [49]. In order to remove most of the external facial features (e.g., hair and ears) and to standardize the spatial layout occupied by each face, each stimulus was enclosed in an oval frame encompassing 8.86 x 7.63 of visual angle (Figure 1; for a similar procedure, see also Ref. 27). Means and standard deviations of pixel luminance were extracted using ImageJ (v1.44; and apparent contrast, defined as the standard deviation divided by the mean, was calculated for each and every face stimulus.

7 No prior entry for threat-related faces Independent samples t-tests revealed that neutral and fearful faces did not differ with regard to apparent contrast [t(18) = -0.65, p =.523]. Procedure The experiment was conducted in a small, dimly lit room on a PC connected to a 19" CRT monitor (refresh rate: 100 Hz) running E-Prime 2.0 ( Viewing distance was held constant at 60 cm throughout the experimental session, with head motions restrained by a chinrest. After filling out the informed consent, participants were presented with four blocks (90 trials each) of the experimental task, preceded by verbal instructions and a practice block containing 10 trials with happy and neutral faces. Trials were structured as follows (Figure 1). A central black cross (0.96 x 0.96 ) was displayed for 1000 ms on a white background. Participants were instructed to maintain fixation on this cross. Afterwards, the first face (8.86 x 7.63 ) appeared in one of two placeholders located on the left or right side of fixation. After a variable time interval (SOAs: 100, 30, or 10 ms), the second face appeared on the opposite side. Both stimuli were equidistant from fixation (distance between the center of the cross and the center of the face: ). Both faces remained on the screen for 100 ms before being replaced in synchrony by a uniform mask until response. The task was to indicate, as fast and accurately as possible, the location (either left or right) of the stimulus that was perceived as appearing first (i.e., two-alternative forced-choice task), using numbers 2 or 8 of the numeric pad of a standard AZERTY keyboard. In order to avoid any stimulus-response compatibility effects [35,37], we opted for the use of response buttons whose (vertical) alignment was orthogonal to the stimuli appearing on the screen along the horizontal axis. Response buttons were counterbalanced across participants. Importantly, each face pair always consisted of two different identities, resulting in a total number of 90 face pairs per

8 No prior entry for threat-related faces condition. In 50% of the trials, one face conveyed a fearful expression, while the other one displayed a neutral expression. Each emotion expression appeared equally often to the left or right of the central fixation cross. As control conditions, either two neutral faces (25% of the trials) or two fearful faces (25% of the trials) were presented on screen. Thus, three stimulus pair conditions were presented in random order: fearful face-neutral face (FearNeut), fearful facefearful face (FearFear), neutral face-neutral face (NeutNeut). To verify that the emotional content of the faces selected in our study was perceived in line with the normative ratings, at the end of the experiment we asked participants to rate the amount of fear conveyed by each neutral and fearful face. A standard 9-point Likert scale was used for this purpose, with anchor 1 corresponding to not afraid and anchor 9 to extremely afraid. Questionnaires At the end of the experiment (also valid for Experiments 2-5), participants were asked to fill out two questionnaires, in order to assess whether specific affective or personality traits might be related to task performance. Levels of trait anxiety were measured using the Dutch version of the State-Trait Anxiety Inventory, trait characteristics [50]. Participants also completed the Need For Affect Scale [51], which provides an estimate of participants general motivation to either approach or avoid emotion-inducing situations. The results confirmed normal scores of trait anxiety and Need for Affect (Table 1). In particular, no significant differences were found between the STAI-T scores of our participants and the average normative STAI-T scores obtained in the Dutch student population (M = 36.90, SD = 8.40) [52]. More importantly, no significant correlation was found between these scores and the behavioral results obtained across the five experiments described below. Therefore, the potential modulatory role of these

9 No prior entry for threat-related faces personality factors on the prioritized allocation of attention towards negative emotional stimuli will not be discussed further. Data analysis Accuracy was expressed as the proportion of right first responses. Positive SOAs refer to cases when the first stimulus was presented on the right hemifield, whereas negative SOAs indicate that the first stimulus was presented on the left side (see Figure 2A). The effect of prior entry was assessed by calculating each participant s point of subjective simultaneity (PSS). This parameter indicates the time interval needed by each participant to perceive the two stimuli as arriving simultaneously or, in other words, an estimate of the SOA at which participants would be likely to make each response equally often [36,37,47,53,54]. To compute the PSS, transformed z-scores of the proportion of right first responses were first obtained by applying the inverse of the standard normal distribution function to the raw proportion scores (probit analysis; see Ref. 55). This transformation enabled us to perform a linear regression on the transformed data to derive the PSS, calculated from the slope and intercept of the best-fitted line of the z-scores (PSS = -slope/intercept). To account for the correlation of measurements within the same subject, we used a mixed probit regression model, where each participant had his/her own intercept and slope with estimated random effects from a bivariate zero-mean normal distribution. If a PSS value was falling outside the SOA range (i.e., > +100 or < -100 ms), the data of this participant were excluded from further analyses (for a similar procedure, see Refs. 37, 48). Based on previous research [44], we hypothesized a prior entry effect (i.e., PSS significantly different from zero, as assessed by two-tailed, one-sample t-tests) for fearful compared to neutral faces in the FearNeut condition, whereas no difference ought to be observed in the two control conditions (i.e., FearFear and NeutNeut).

10 No prior entry for threat-related faces Interestingly, several studies point either to a possible advantage of the right hemisphere in attention selection mechanisms [56,57,58], or a disadvantage of the left hemisphere in these processes [59,60]. Moreover, earlier work suggested that the right hemisphere could preferentially be engaged in the processing of emotion-laden stimuli [61,62,63,64]. Accordingly, in all the experiments reported here, we also assessed whether any enhanced prior entry effect could be observed when the first (emotional or neutral) face in the pair was presented in the left vs. right hemifield relative to fixation. However, we did not find any effect of the side of presentation during the TOJ task. These results are also consistent with the study by Fecica & Stolz [38]. We also computed and analyzed the so-called just noticeable difference (JND; see Table 3). JND corresponds to the slope of the best-fitted line of the z-scores (0.675/slope). This metric reflects the smallest temporal interval between two stimuli needed for an observer to correctly judge which stimulus had been presented first on 75% of the trials, since ±0.675 represents the 75% and 25% point on the cumulative normal distribution [36,47,54,65]. However, from a theoretical standpoint, the effects of spatial attention on JND in a TOJ task are still unclear [54,66]. In addition, our analyses performed on the JND obtained for each of the five experiments did not reveal any valuable (compared to the PSS) information regarding differential prior entry effects for emotional relative to neutral faces. Reaction times (RTs) were analyzed by means of repeated measures ANOVAs. However, it should be noted that previous studies (see, for instance, Ref. 53) have been equivocal with regards to the reliability of this dependent variable in assessing genuine prior entry effects, particularly because TOJ tasks are usually performed under unspeeded time constraints [67,68]. Therefore, RT data were analyzed with the sole purpose to provide additional evidence that our

11 No prior entry for threat-related faces main experimental manipulation (i.e., SOA) was successful: at short SOAs (i.e., ±30 and ±10 ms), where uncertainty was high, participants would be slower than at long SOAs (i.e., ±100 ms). The results unambiguously confirmed this prediction. However, this analysis did not reveal any significant result that would be compatible with a prior entry effect for threat-related faces. Hence, we will not report the outcome of this analysis, either for Experiment 1 or the subsequent Experiments (2-5). The alpha level for all statistical analyses was set at p < Results Trials whose RTs were slower than three standard deviations from the mean (calculated for each condition and SOA separately across participants) were removed from the analysis (M = 1.12%, SD = 0.73). Figure 2A shows the proportion of right first responses for each condition (FearNeut, FearFear, NeutNeut). A clear inverted S-shaped psychometric function was obtained for each condition, providing evidence that the main experimental manipulation (i.e., SOA) was successful. Thus, participants perceived the onsets of the two stimuli in accordance with their respective occurrences. More specifically, participants TOJs were more uncertain (i.e., the proportion of right first responses was close to chance) at short (i.e., ±30 and ±10 ms) compared to long (i.e., ±100 ms) SOAs. The PSS values for each condition are reported in Table 2. For none of the three conditions did the one-sample t-test reach significance [FearNeut: t(31) = 1.15, p =.260; FearFear: t(31) = 1.37, p =.180; NeutNeut: t(31) = 1.82, p =.079], indicating no reliable prior entry effect for fearful compared to neutral faces (Figure 2B). Importantly, results of the post-experiment ratings unequivocally confirmed that fearful faces were perceived as more fearful compared to neutral faces [t(31) = 28.10, p <.001] (Figure 2C).

12 No prior entry for threat-related faces Discussion In Experiment 1, participants were presented with pairs of fearful and neutral faces, and were instructed to report whether the first stimulus appeared on the left or right visual hemifield. We hypothesized that fearful faces, because of their enhanced intrinsic motivational salience, could rapidly capture exogenous attention and, accordingly, bias TOJs (as reflected by PSS values being significantly different from zero in the FearNeut condition). However, we did not observe such pattern of results. Importantly, these non-significant findings could not easily be accounted for by mere task difficulty, abnormal temporal perception, or attentional allocation spread throughout the visual field, since most of the participants could correctly identify the first onset in the pair (as evidenced by the presence of a clear inverted S-shaped psychometric function observed for each experimental condition; see Figure 2A). Moreover, post-experiment ratings confirmed that fearful faces were clearly recognized as such compared to neutral faces (Figure 2C), ruling out the possibility that the fearful faces selected in this experiment displayed weak or undifferentiated negative emotional expressions. Although fearful faces were previously shown to influence early attention selection processes (see Ref. 4, for a recent review), the lack of a reliable prior entry effect for fearful faces might be explained by the fact that the threat displayed in these faces is indirect in essence, thereby affecting the motivational significance to a lower extent than angry faces, which convey a more direct threat [69,70]. Moreover, earlier studies using TOJ tasks already reported prior entry effects for (schematic and realistic) angry faces [38,44]. Therefore, in Experiment 2, we concurrently used angry and fearful faces in order to assess whether any prior entry effect for negative emotional facial expressions might be specific to angry faces or not. Furthermore, we substantially reduced the size of the face stimuli compared to Experiment 1, as well as their

13 No prior entry for threat-related faces eccentricity relative to fixation. We reasoned that the use of large face stimuli (i.e., subtending 8.86 x 7.63 of visual angle) shown in the far periphery (i.e., from fixation) may have favored the use of low-level features to perform the TOJ task in Experiment 1. By comparison, West and colleagues [44] presented schematic or human faces in squared boxes subtending 3.80 x 3.60 of visual angle at a lower horizontal eccentricity (3.15 from fixation). Accordingly, in Experiment 2, our stimulus parameters were more closely matched to those used previously by West, et al. [44]. Experiment 2 Participants Forty healthy psychology students participated in the study in exchange of course credits. None of them had participated in Experiment 1. All volunteers gave informed written consent prior to their participation. The data of two participants were excluded from further analyses due to an abnormal inverted S-shaped psychometric function in at least one experimental condition (similarly to Experiment 1). Thus, the final sample consisted of 38 participants (32 women, mean age 18 years, range 17-22). Stimuli Fearful and neutral faces were identical to the ones used in Experiment 1. However, they were now enclosed in a smaller oval frame, spanning 4.77 x 3.06 of visual angle. In addition, 10 faces displaying an angry expression were selected from the same standardized Ekman series [49]. Apparent contrast was also calculated for angry faces, and independent samples t-tests revealed no significant difference between neutral and angry faces [t(18) = -0.99, p =.334], as well as between fearful and angry faces [t(18) = -0.16, p =.877]. Procedure

14 No prior entry for threat-related faces The procedure and design of the task were similar to Experiment 1. However, here the facial stimuli were presented closer to fixation (distance between the center of the fixation cross and the center of the face: 6.68 ) compared to Experiment 1. The stimulus pair conditions were angry face-neutral face (AngerNeut) and fearful face-neutral face (FearNeut). No additional condition (i.e., AngerAnger, FearFear, or NeutNeut) was included, in order to avoid an excessively high number of trials and long testing session likely causing drops or lapses in attention. Note that the use of the AngerNeut and FearNeut conditions alone is sufficient to establish whether any reliable prior entry (for either angry or fearful faces) was present or not [44]. Ratings of perceived anger and fear conveyed by each face stimulus were collected at the end of the main TOJ task by means of 9-point Likert scales, ranging from 1 ( not afraid/angry ) to 9 ( extremely afraid/angry ). Additionally, participants were asked to provide ratings of perceived brightness for each emotional and neutral face (from 1, very dark, to 9, very bright ), to further corroborate the lack of clear difference in this low-level visual property across the three emotion categories (i.e., neutral, angry, and fearful). Results Following standard practice, trials whose RTs were slower than three standard deviations from the mean were discarded (M = 0.98%, SD = 0.66). Behavioral results showed that the distribution of the proportion of right first responses was consistent with the results obtained in Experiment 1: responses were close to chance level at short compared to long SOAs. Table 2 shows the PSS values for each condition. None of the one-sample t-tests were significantly different from zero [AngerNeut: t(37) = 0.99, p =.327; FearNeut: t(37) = 0.74, p =.466]. Thus, no prior entry for negative emotional facial expressions (either fear or anger) was evidenced.

15 No prior entry for threat-related faces Post-experiment ratings confirmed that fearful faces were perceived as more fearful compared to neutral [t(37) = 34.02, p <.001] and angry faces [t(37) = 29.60, p <.001]. Similarly, angry faces were rated higher along the anger intensity dimension compared to neutral [t(37) = 33.15, p <.001] and fearful faces [t(37) = 25.97, p <.001]. Thus, participants correctly perceived the respective emotion content displayed by the selected face stimuli. Results further revealed higher perceived brightness for emotional relative to neutral faces [anger vs. neutral: t(37) = 4.73, p <.001; fear vs. neutral: t(37) = 2.97, p <.001], an effect that could be explained by an emotionenhanced perceptual vividness [71]. Note that, despite these subjective differences in brightness, no prior entry effect for either angry or fearful faces was found. Discussion Results of Experiment 2 failed to show any significant prior entry effect for either fearful faces (replicating the results of Experiment 1) or angry faces when compared to neutral faces, despite a clear effect of SOA on TOJs (i.e., inverted S-shaped psychometric function). Unlike previous studies mainly using schematic angry faces [38,44], here we did not find evidence for the preferential (exogenous) orienting towards photographs of realistic fearful or angry faces when they compete with neutral faces for attention selection. Because our experimental setup was similar to West and colleagues [44], these results are unlikely to be explained by suboptimal stimulus parameters or task demands. Moreover, since participants of Experiment 2 unambiguously identified the emotion conveyed by fearful and angry faces during a postexperiment rating phase, these results cannot be accounted for by the use of face stimuli providing weak or undifferentiated emotional expressions relative to neutral faces. An intriguing possibility to account for these non-significant findings (Experiments 1-2) may be related to the specific task set adopted by the participants throughout the experimental

16 No prior entry for threat-related faces session. Given that participants had to focus on spatial and temporal properties to carry out the two-alternative forced-choice task (i.e., is it the left or right stimulus appearing first?), the emotion content of the faces could somehow be filtered out in these two experiments. Moreover, previous research showed that early and automatic affective stimulus processing could substantially be reduced when concurrent non-affective (spatial) stimulus dimensions became task-relevant [72,73,74,75], consistent with the idea that the (exogenous) capture of attention by emotion is not magic, but subject to (state) fluctuations depending on the availability of attentional resources, as well as the specific task set [4]. In light of this evidence, we surmised that participants of Experiments 1-2 may have adopted an efficient strategy and primarily allocated attentional resources to the processing of the spatial and temporal properties of the two face stimuli, while actively ignoring their emotional content because poorly informative to resolve the task. We have to acknowledge, however, that this account already posits that negative emotional facial expressions do not automatically capture attention, because this effect (at least in the case of a TOJ task) may actually depend upon the specific task demands [76]. Accordingly, no prior entry for angry or fearful faces was evidenced in these two first experiments, probably because participants could easily ignore the emotional content of the two competing faces and focus on a specific non-affective stimulus feature whose processing was sufficient to perform the task. To address this issue, in Experiment 3 we modified the task instructions and asked participants to judge whether the emotional or the neutral face appeared first (emotion TOJ), making the differential emotional content of the two faces in the pair directly task-relevant. Hence, in Experiment 3 a two-alternative forced-choice task was still required, but it concerned the content rather than the spatial position of the face stimuli. Experiment 3

17 No prior entry for threat-related faces Participants Thirty-seven psychology students, who did not participate in Experiments 1 or 2, took part in Experiment 3. Using the same exclusion criteria as above (see Experiments 1 and 2), the data of 16 participants had to be removed from the subsequent statistical analyses. The data of 21 participants (19 women, mean age 18 years, range 18-21) were thus included in the final sample. Stimuli and procedure The stimuli were identical to Experiment 2. However, unlike Experiments 1-2, participants were asked to perform a two-alternative forced-choice task based on the emotional content of the face stimuli in the pair. More precisely, they were instructed to judge whether the stimulus that appeared first had either a neutral or an emotional expression, thereby making the emotional content of the face stimuli directly task-relevant. Another notable difference between Experiment 3 and Experiments 1-2 was the use of a block design. In order to facilitate participants discrimination between emotional and neutral faces, AngerNeut and FearNeut trials were no longer presented in random order throughout the experimental session, but in two separate blocks (counterbalanced across participants). Finally, ratings for the perceived anger, fear, and brightness of the individual face stimuli were collected post-experiment, similarly to Experiments 1-2. Results A total of 0.55% (SD = 0.40) of trials were discarded because their RTs were slower than three standard deviations from the mean. As expected, the proportion of emotion first responses was close to chance level at short compared to long SOAs, as evidenced by a clear inverted S-shaped psychometric function.

18 No prior entry for threat-related faces However, PSS values for each condition (see Table 2) revealed no significant prior entry effect [AngerNeut: t(20) = 0.18, p =.858; FearNeut: t(20) = -1.27, p =.218]. Post-experiment ratings confirmed that fearful faces were perceived as more fearful compared to neutral faces [t(20) = 15.84, p <.001] and angry faces [t(20) = 16.36, p <.001]. In addition, angry faces were perceived as carrying more anger intensity than neutral [t(20) = 17.00, p <.001] and fearful faces [t(20) = 16.72, p <.001]. Finally, participants rated emotional and neutral stimuli as equally bright (ps >.05). Discussion Despite the use of an emotion TOJ task (as opposed to a TOJ task based on the location of the face appearing first; see Experiments 1-2), we did not find evidence for a differential prior entry effect for either fearful or angry faces relative to neutral faces. Noteworthy, these non-significant results were obtained despite a clear emotion differentiation of the three emotion categories (as confirmed by post-experiment ratings), as well as the presence of clear inverted S-shaped psychometric functions in 21 participants (unambiguously revealing a clear sensitivity to the main SOA manipulation). The lack of prior entry effect for angry faces is puzzling to some extent, since participants were asked to process the emotional content of the faces in the pair in order to perform the task. Previous research showed that in these conditions (i.e., when emotion is directly task-relevant), rapid and automatic effects of (negative) emotion on feature-specific attention allocation could be observed in healthy adult participants [72,75,77]. Furthermore, these findings are also at odds with earlier results showing a reliable prior entry effect for angry faces [44], because similar stimulus parameters were used in these two studies. Using a stringent and standard exclusion criterion [37,47,48], we found out that the data of sixteen participants had to be removed from the analysis because they did not show a normal

19 No prior entry for threat-related faces change in TOJ (at least in one experimental condition) as a function of the SOA. This exclusion rate was substantially larger than what we found in Experiments 1-2 (where participants were instructed to focus exclusively on spatial and temporal properties of the two face stimuli in the pair), suggesting that the discrimination of the emotional content of the faces was more demanding than processing the temporal and spatial features of the first face appearing on screen. Noteworthy, none of the two previous studies looking at prior entry for angry faces used a similar exclusion criterion [38,44], suggesting that the results reported in these earlier studies might include the data of poor-performers who may encounter difficulties to process the (finegrained) changes in the respective onsets of the two faces. In Experiment 4, we aimed at addressing this question and, accordingly, we devised a new modification of the TOJ task enabling to briefly train temporal perceptual abilities with low-level geometrical stimuli, before the putative effect of the emotional content of the face was systematically explored. We hypothesized that this initial task familiarization with geometrical figures might later reduce the drop rate for the emotion TOJ. Hence, at the beginning of Experiment 4, we included two training blocks during which participants had to perform the TOJ task based on the orientation of line gratings (being either horizontal or vertical). Then, participants performed the emotion TOJ, as described in Experiment 3. Experiment 4 Participants Forty psychology students, who did not participate in any of the previous experiments, took part in Experiment 4 for course credits. Using the same exclusion criteria as above (see Experiments 1-3), the data of 23 participants had to be excluded from the subsequent statistical

20 No prior entry for threat-related faces analyses. Hence, the final sample consisted of 17 participants (13 women, mean age 20 years, range 18-30). Stimuli and procedure Face stimuli and procedure were identical to Experiment 3. In addition, before the emotion TOJ task, participants carried out a non-emotion TOJ task aimed at familiarizing them to detect asynchronies in the different onset times. Two blocks were included (each containing 90 trials), in which gratings consisting of either horizontal or vertical black lines on a white background (matched in size with the face stimuli; see Figure 1) were presented equally often on the left and right hemifield, separated by the SOAs described above (i.e., 100, 30, or 10 ms). Participants had to judge whether the horizontal or vertical line gratings appeared first. Ratings of the individual faces regarding the intensity of anger, fear, and brightness were collected at the end of the experiment. Results Trials whose RTs were slower than three standard deviations from the mean were discarded (M = 0.73%, SD = 0.51). Figure 3A shows the proportion of horizontal first responses for the non-emotion TOJ task during the two familiarization blocks, as well as the proportion of emotion first responses for the subsequent emotion TOJ task. Performance for the non-emotion TOJ task was remarkably accurate, as evidenced by a clear inverted S-shaped psychometric function (HorizVert condition in Figure 3A). By contrast, accuracy was substantially reduced for the emotion TOJ task, as shown by flatter inverted S-shaped psychometric functions for the AngerNeut and FearNeut conditions. Please note that the results reported here are for good performers only (i.e., participants whose PSS fell within the -100/+100 ms interval for all conditions). Table 2 shows

21 No prior entry for threat-related faces the PSS values for each condition separately. No significant prior entry effect was found in the HorizVert condition [t(16) = 0.58, p =.568], serving as control condition or low-level baseline. However, PSS values were also not significant in the AngerNeut [t(16) = -0.51, p =.616] and FearNeut [t(16) = -1.24, p =.232] conditions (Figure 3B). Post-experiment ratings confirmed that the face stimuli were perceived in line with the a priori emotion categories: fearful faces were perceived as more fearful compared to neutral [t(16) = 23.66, p <.001] and angry faces [t(16) = , p <.001]. Likewise, angry faces were perceived as more angry relative to neutral [t(16) = 16.99, p <.001] and fearful faces [t(16) = 14.54, p <.001], with no significant difference in perceived brightness across these three conditions (ps >.05) (Figure 3C). Discussion Results of Experiment 4 did not show any prior entry effect for either fearful or angry faces, when these threat-related face stimuli compete with neutral faces for early attention selection. As was already the case for Experiments 1-3, this result could not be imputed to a lack of perceived emotion differences between the three stimulus categories, since post-experiment ratings showed clear and predictable differences. We reasoned that the use of familiarization blocks with horizontal and vertical line gratings (i.e., non-emotional features) might have eased performance during the subsequent emotion TOJ task. However, this turned out to be a wrong prediction. Despite the introduction of these two familiarization blocks, in fact, the drop rate was still substantial (23 out of 40 participants, 58%). Hence, 23 participants had PSS values (at least in one condition) exceeding the maximum SOA range (± 100 ms). Unexpectedly, this drop rate was even higher compared to the one found in Experiment 3 (43%), where no familiarization with the vertical and horizontal gratings was introduced. However, if we only used the data of the TOJ

22 No prior entry for threat-related faces task performed on the line gratings, this drop rate would be remarkably lower (10%), suggesting that participants encountered specific difficulties only when asked to decide whether the emotional face in the pair was shown first or not, but not when asked to decide whether horizontal or vertical line gratings appeared first. This conclusion was also reinforced by the direct comparison of the two tasks for the 17 participants included in the analyses (see Figure 3A). We reasoned that task difficulty during the emotion TOJ might perhaps decrease if we would give more precise instructions to participants. Specifically, while in Experiments 3-4 instructions emphasized the discrimination between emotional and neutral faces, the use of distinct response labels (angry vs. neutral or fearful vs. neutral) could presumably improve performance. A refined task set biasing feature-specific attention allocation towards specific emotional features [72,74,77,78], in fact, could facilitate TOJs based on these emotional features. Accordingly, in Experiment 5, we used the same stimuli and setup as in Experiment 4, but asked participants to indicate whether the first stimulus was an angry/fearful (depending on the block) or a neutral face. Experiment 5 Participants Thirty-six psychology students, who participated in none of the previous experiments, took part in Experiment 5 in exchange of course credits. Using the same exclusion criterion as above, the data of twenty volunteers were removed from the subsequent statistical analyses, leaving a final sample of 16 participants (9 women, mean age 22 years, range 18-30). Stimuli and procedure

23 No prior entry for threat-related faces Stimuli were identical to Experiment 4. Similarly, two familiarization blocks with horizontal and vertical line gratings were used at the beginning of the experiment, to allow participants to familiarize with the TOJ task and the different SOAs. Unlike Experiment 4, however, for the subsequent emotion TOJ task participants were specifically asked to decide whether the face that appeared first in the pair was neutral, angry, or fearful (two blocks each, counterbalanced across participants). Ten practice trials with either angry-neutral or fearful-neutral stimulus pairs preceded the two experimental blocks. Ratings for the individual faces regarding the amount of anger, fear, or brightness were collected at the end of the experiment. Results Trials whose RTs were slower than three standard deviations from the mean were discarded (M = 0.66%, SD = 0.50). Overall, participants performed better in the familiarization task compared to the emotion TOJ task, as evidenced by flatter inverted S-Shaped psychometric functions for the AngerNeut and FearNeut conditions relative to the HorizVert condition. None of the PSS values (reported in Table 2) was significantly different from zero [HorizVert; t(15) = -0.65, p =.524; AngerNeut: t(15) = p =.184; FearNeut; t(15) = -0.68, p =.508]. Post-experiment ratings confirmed that emotional faces were perceived as such by participants. Fearful faces were perceived as more fearful compared to neutral [t(15) = 19.08, p <.001] and angry faces [t(15) = , p <.001]. Similarly, angry faces were perceived as more angry than neutral [t(15) = 15.21, p <.001] and fearful faces [t(15) = 9.77, p <.001]. Higher perceived brightness for emotional relative to neutral faces was also reported [anger vs. neutral: t(15) = 5.54, p <.001; fear vs. neutral: t(15) = 3.56, p =.003], consistent with an emotion-

24 No prior entry for threat-related faces enhanced perceptual vividness [71]. However, these subjective differences in brightness did not lead to prior entry effect for either angry or fearful faces relative to neutral faces. Discussion Using more specific task instructions than in Experiment 4 (i.e., by explicitly mentioning either anger or fear as target emotion), we still failed to observe a reliable prior entry effect for threat-related faces. Moreover, as was already the case for Experiment 4, the data of a high number of participants had to be discarded (drop rate of 56%) due to PSS values in the AngerNeut and FearNeut conditions that were falling outside the ±100 ms SOA range. Therefore, the use of specific emotion labels during the emotion TOJ (Experiment 5) did not lead to any gain in accuracy compared to more general task instructions primarily emphasizing the discrimination of emotional vs. neutral faces (Experiments 3-4). Again, these results could not be explained by difficulties to identify or recognize the different emotional facial expressions (see results of the post-experiment ratings), or the use of suboptimal SOAs and/or stimulus parameters (see results for the two familiarization blocks with the line gratings). Additional analyses Power analysis The estimated average effect size of West et al. [44] s experiments was remarkably high (Cohen s d = 0.75), with an estimated power of 71%. An a priori power analysis using G*Power 3 [79] indicated that a total sample of 16 participants would be needed to detect the same effect with 80% power using two-tailed, one-sample t-tests with α = The number of participants in all our experiments was therefore adequate to detect a potential visual prior entry effect for threat-related vs. neutral faces of similar size. More specifically, we had a 98% power to detect an effect with α = 0.05 and d = 0.75 in Experiment 1, 99% in Experiment 2, 90% in Experiment

25 No prior entry for threat-related faces , 83% in Experiment 4, and 80% in Experiment 5. Thus, our five experiments appeared sensitive enough to detect an effect size equal to West et al. [44] s. Importantly, our experiments were able to detect, with 80% power and α = 0.05, an effect size of 0.51 in Experiment 1, 0.47 in Experiment 2, 0.64 in Experiment 3, 0.72 in Experiment 4, and 0.75 in Experiment 5. Assessing basic problems with the elected experimental design Presumably, the lack of prior entry for threat-related faces could be imputed to uncontrolled experimental factors in our design that would somehow prevent this attention effect to occur. A way to rule out this possibility is to show that, using the exact same task demands and stimulus parameters, we could nevertheless reveal a significant prior entry effect when attention is reflexively oriented towards one of the two sides using a standard exogenous cue. To address this issue, we ran an additional control experiment. Twenty-five participants (18 women, mean age 27 years, range 24-32) were presented with five blocks (72 trials each) of the line orientation TOJ task used in Experiments 4 and 5. However, in two-thirds of the trials, the thickness of either the left or right placeholder was increased (from 4 to 14 pixels) for 45 ms, prior to the actual onsets of the two gratings (vertical and horizontal) within the two placeholders. The time interval between this exogenous cue and the first stimulus in the pair was constant and set to 60 ms (for a similar procedure, see Ref. 53). Participants were explicitly instructed to ignore this cue throughout the whole experimental session because non-informative (see also Ref. 80), and only judged whether the horizontal or vertical lines appeared first. After converting the cued location into a cued orientation [53], the proportion of horizontal first responses was calculated. When no cue was presented (one third of the trials), a reliable psychometric curve was observed in a vast majority of participants (N = 22). The average PSS was ms (SD = 8.41) and was not statistically significant from zero [t(21) = -0.81, p =.426],

26 No prior entry for threat-related faces replicating the findings of Experiments 4 and 5. By contrast, when the unilateral exogenous cue was used (two thirds of the trials), the stimulus (either horizontal or vertical lines) presented in the same (valid) location was systematically perceived as appearing first, replicating earlier findings [53]. Of note, for 14 participants, the attention capture effect of this cue was so strong that prior entry effects were observed for the cued stimulus regardless of the duration of the SOA. As a result, reliable PSS values could not be computed for these participants. However and most importantly, for the remaining 8 participants where PSS values could be computed for all experimental conditions (i.e., cue and no cue), the average PSS value was ms (SD = 24.17) when the horizontal lines were cued, and ms (SD = 30.46) when the vertical lines were cued. These values were significantly different from zero [t(7) = -9.27, p <.001 and t(7) = 4.75, p =.002, respectively]. These results suggest that attention was reliably biased towards the location of the exogenous cue, such that the vertical or horizontal lines appearing later at the same (valid) location were systematically perceived as appearing first. Accordingly, the lack of systematic bottom-up effect of threat-related vs. neutral faces on the guidance of attention reported in Experiments 1-5 cannot simply be ascribed to uncontrolled methodological problems with the experimental design. Good vs. poor performers When the emotional content became task-relevant (Experiments 3-5), as opposed to the mere appearance of the two faces in the pair (Experiments 1-2), many participants showed PSS values outside the SOA range (±100 ms). These poor-performers, therefore, had to be excluded from subsequent statistical analyses (see Table 4). This suggests that poor performers could not accurately carry out the emotion TOJ task even though, in Experiments 4-5, the majority of them could correctly discriminate which line gratings appeared first, ruling out the possibility of a