The obligatory nature of holistic processing of faces in social judgments

Perception, 2010, volume 39, pages 514 ^ 532 doi:10.1068/p6501 The obligatory nature of holistic processing of faces in social judgments Alexander Todorov, Valerie Loehr, Nikolaas N Oosterhof Department of Psychology, Princeton University, Princeton, NJ 08540, USA; e-mail: atodorov@princeton.edu Received 27 June 2009, in revised form 13 January 2010; published online 25 March 2010 Abstract. Using a composite-face paradigm, we show that social judgments from faces rely on holistic processing. Participants judged facial halves more positively when aligned with trustworthy than with untrustworthy halves, despite instructions to ignore the aligned parts (experiment 1). This effect was substantially reduced when the faces were inverted (experiments 2 and 3) and when the halves were misaligned (experiment 3). In all three experiments, judgments were affected to a larger extent by the to-be-attended than the to-be-ignored halves, suggesting that there is partial control of holistic processing. However, after rapid exposures to faces (33 to 100 ms), judgments of trustworthy and untrustworthy halves aligned with incongruent halves were indistinguishable (experiment 4a). Differences emerged with exposures longer than 100 ms. In contrast, when participants were not instructed to attend to specific facial parts, these differences did not emerge (experiment 4b). These findings suggest that the initial pass of information is holistic and that additional time allows participants to partially ignore the task-irrelevant context. 1 Introduction Multiple experiments in psychology have shown that face perception relies on configural rather than feature-based processing (Farah et al 1998; Maurer et al 2002). For example, in the composite-face effect, when the top half of one face is aligned with the bottom half of another face, the two halves create a new configuration that interferes with the recognition of the identity of the original faces (Young et al 1987). Similar effects have been demonstrated for perception of gender (Baudouin and Humphreys 2006), race (Michel et al 2007), attractiveness (Abbas and Duchaine 2008), and emotional expressions (Calder et al 2000). The composite-face effect is considered the signature of holistic processing of faces (Maurer et al 2002), namely the idea that faces are processed as gestalts so that the perception of individual features changes when integrated with other features. Although the evidence is clear that people are unable to ignore task-irrelevant facial features (eg facial halves are perceived as less attractive when aligned with unattractive than with attractive halves, Abbas and Duchaine 2008), it is also consistent with the idea that people have partial control over the processing of these features. That is, although task-irrelevant features do interfere with face processing, people are able to focus on the halves that they are instructed to attend. For example, in identity tasks, the identification of facial halves aligned with halves from other faces is accurate although slower (Young et al 1987). However, in most of these studies, the faces were either presented for unlimited time or for relatively long exposures (for an exception, see Hole 1994). Given that faces can be processed after minimal time exposure (Ballew and Todorov 2007; Bar et al 2006; Grill-Spector and Kanwisher 2005; Olson and Marshuetz 2005; Todorov et al, in press; Willis and Todorov 2006), it is possible that long exposures to faces allow for potential top ^ down control of face processing and, hence, partial suppression of task-irrelevant facial features. The main objective of the current research was to test whether holistic processing of faces emerges after extremely brief presentation of faces and whether differences

Obligatory holistic processing 515 consistent with top ^ down control of face processing emerge after longer exposures. To test these hypotheses, we used computer-generated faces that were manipulated on trustworthiness (Oosterhof and Todorov 2008), and asked participants to make judgments of trustworthiness. Because the faces were manipulated in a validated computer model, which provides precise `values' on trustworthiness, we could test how judgments change as a function of the task-irrelevant context. To the extent that social judgments rely on holistic processing, facial halves aligned with untrustworthy halves should be judged more negatively than halves aligned with trustworthy halves. More importantly, we can also measure the extent to which the effect of the to-be-judged halves on judgments is stronger than the effect of the to-be-ignored halves as a measure of control of holistic processing. In composite-face paradigms, in which there is a well-defined correct response (eg identity and emotion recognition tasks), often the main measure of interest is the response times for the judgments. Participants are generally slower to make a judgment when a facial half is aligned with an incongruent half (eg from a different facial identity or a different emotion category), although they are fairly accurate (eg Young et al 1987). In social-judgments paradigms, the more important measure is the actual judgments (Abbas and Duchaine 2008). Importantly, as noted above, when the faces are parametrically manipulated on the social dimension (Oosterhof and Todorov 2008), we can quantify the effect of the to-be-ignored context on judgments. The first three experiments were designed to demonstrate a composite-face effect for perceptions of trustworthiness. Previous studies have shown that social judgments from faces are affected by photographic negation (reversing the polarity of the image) and inversion, suggesting that these judgments rely on holistic processing (Santos and Young 2008). However, to the best of our knowledge, there have not been demonstrations of the composite-face effect for social judgments except for attractiveness (Abbas and Duchaine 2008). Experiment 1 demonstrated this effect for judgments of trustworthiness. The next two experiments replicated the effect and showed that the effect is substantially reduced when the faces are inverted (experiments 2 and 3) and when the to-be-judged and to-be-ignored facial halves are misaligned (experiment 3). Experiments 4a and 4b tested the main hypothesis of the paper of how early holistic processing of faces emerges by systematically manipulating the time exposure to faces. We capitalized on findings that social judgments from faces are made after extremely brief face presentations (Ballew and Todorov 2007; Bar et al 2006; Todorov et al 2009; Willis and Todorov 2006). For example, Todorov et al (2009) showed that judgments of trustworthiness made after as little as 33 ms, but not after 17 ms, masked exposure to faces agreed with judgments made in the absence of time constraints. With the increase in exposure time from 33 to 100 ms, there was a large improvement in this agreement and little improvement with longer exposures. In experiment 4a, participants were instructed to judge the upper halves of faces, which were presented with exposures ranging from 33 to 500 ms. We were specifically interested in whether differences between judgments of upper trustworthy halves aligned with untrustworthy halves and judgments of upper untrustworthy halves aligned with trustworthy halves would emerge after extremely brief or after longer exposures. To rule out the possibility that such differences are due to heavier reliance on the upper than the lower halves of the faces, participants in experiment 4b were not given instructions to focus on any facial half. 2 Experiment 1 To the extent that judgments of trustworthiness from faces rely on holistic information, it should be possible to demonstrate a composite-face effect for these judgments (Maurer et al 2002; Young et al 1987). To test this hypothesis, we created composite

516 A Todorov, V Loehr, N N Oosterhof faces that consisted of trustworthy and untrustworthy halves (see figure 1). Participants were instructed either to attend to the upper half of the face and ignore the lower half or to attend to the lower half of the face and ignore the upper half. If trustworthiness judgments rely on holistic processing, judgments of facial halves should be more positive when these halves are aligned with trustworthy halves than when they are aligned with untrustworthy halves. To generate trustworthy and untrustworthy faces, we used the computer model of face trustworthiness developed by Oosterhof and Todorov (2008). This model can generate an unlimited number of faces and precisely manipulate their perceived trustworthiness. We randomly generated faces and then produced trustworthy and untrustworthy versions for each facial identity. From these versions, we created trustworthy and untrustworthy (a) (b) (c) (d) (e) (f) (g) (h) (i) (j) (k) (l) (m) (n) (o) (p) Figure 1. Examples of stimuli used in the experiments. The first row shows (a) untrustworthy version of a face; (b) trustworthy version of a face; (c) composite version with upper trustworthy half and lower untrustworthy half; and (d) composite version with upper untrustworthy half and lower trustworthy half. The second row shows (e) untrustworthy upper half; (f) trustworthy upper half; (g) untrustworthy lower half; and (h) untrustworthy lower half. The third row shows the corresponding misaligned versions of the faces from the first row. The fourth row shows the corresponding inverted versions of the faces from the first row. The whole faces were used in all experiments. The facial halves were used in experiments 1 and 2. The misaligned faces were used in experiment 3. The inverted faces were used in experiments 2 and 3.

Obligatory holistic processing 517 halves that were aligned either with trustworthy (positive context) or untrustworthy (negative context) halves. In addition to whole faces, we included facial halves (see figure 1) to provide baseline trials relative to which to estimate the effect of positive and negative contexts on trustworthiness judgments, where the context is defined by the presence of a facial half that the participant is instructed to ignore. There is good evidence that negative information is weighted more heavily than positive information in social judgments (eg Fiske 1980; Pratto and John 1991; Rozin and Royzman 2001). These findings suggest that untrustworthy facial halves may have a greater effect on judgments than trustworthy facial halves. That is, relative to judging a facial half only, judging a facial half in the context of an untrustworthy half should result in a greater change in judgments (more negative judgments) than judging the same half in the context of a trustworthy half (more positive judgments). 2.1 Method 2.1.1 Participants. Twenty-six undergraduate students from Princeton University participated in the experiment for partial course credit. 2.1.2 Face stimuli. We used the Facegen Modeller program, version 3.1 (Singular Inversions 2006) to generate faces and the trustworthiness face model developed by Oosterhof and Todorov (2008). We randomly generated 60 emotionally neutral faces with the constraints that the faces were Caucasian males to avoid judgments affected by ethnic stereotypes and because male faces look more natural without scalp hair. For each of the 60 face identities, we created untrustworthy ( 3 SD on the normally distributed trustworthiness dimension) and trustworthy (3 SD) versions. Trustworthiness was manipulated with a custom-created and validated Facegen control that changes facial features that are specific for trustworthiness (Oosterhof and Todorov 2008). The images were saved as 4006400 pixels bitmaps. Copies of the 60 high and 60 low trustworthy faces were manipulated to create faces where half of the face (upper or lower) was high on trustworthiness and the other half was low on trustworthiness, resulting in 120 composite faces. Thus, each of the 60 original faces corresponded to four different types of faces: high and low trustworthy versions, and high/low and low/high composite versions (figure 1). In terms of the experimental design (see below), these faces map onto 2 (trustworthiness of to-be-judged half)62 (trustworthiness of to-be-ignored half) conditions. To create the composite faces, the high and low trustworthiness faces were separated into bottom and top halves. The top part of the faces was cut 0.5 inches (36 pixels) below the eye to realistically blur the two halves together, keeping the dark circles underneath the eye intact. The nose and the area underneath the eyes were merged together with the Blur tool of Adobe Photoshop Elements. The sides of the faces were merged with the Smudge tool under the dark mode with a pressure of 50% to account for the shadowing of the sides of the faces. A black border was added around the faces about 0.5 inches above the eyebrows, 0.5 inches to the side of the eyes, and 0.5 inches below the chin. All faces were scaled to 2986298 pixels. 2.1.3 Procedures. Participants were told that this was a study about first impressions and that they should rely on their `gut' instinct in judging the faces. They were told that the experiment consisted of two parts of equal length. In one of the parts, participants were instructed to attend to the upper half of the face and to ignore the lower half. The upper half was defined as the half above the midpoint of the bridge of the nose and the lower half was defined as the half below that point. In the other part, they were instructed to attend to the lower half and to ignore the upper half. The order of the two parts was counterbalanced across participants.

518 A Todorov, V Loehr, N N Oosterhof To avoid repetition of the facial stimuli in the two parts of the experiment, the 60 face identities were randomly split into two sets. As described above, each facial identity was a source of four different faces. Thus, each set contained 120 faces. In addition, we included trials with only the halves that participants were instructed to judge. For example, the ``attend to the upper half of the face'' part included 60 trials with the upper facial halves of the 30 trustworthy and 30 untrustworthy faces. The total number of trials was 360 (180 trials per experimental part). Orthogonal to the counterbalancing of the instruction parts, the two sets of facial images were counterbalanced across participants. Participants were presented with two practice trials to become familiar with the procedures. Each trial started with a fixation point ( ) presented for 500 ms at the center of the screen. Then, a facial image (a whole face or a half) was presented with ``Trustworthy?'' presented above the image. The image remained on the screen until the participant's response. Participants responded by pressing the À' key on the keyboard which was labeled `yes' or by pressing the `;' (semicolon) key on the keyboard which was labeled `no'. The inter-trial interval was 1000 ms. The order of the trials was randomized for each participant. The overall design was a 2 (instruction: attend to and judge lower vs upper half of the face)62 (trustworthiness of judged half: trustworthy vs untrustworthy)63 (context: single half vs half aligned with a trustworthy half vs half aligned with an untrustworthy half) within-subjects design ANOVA. (1) Mean judgments reported in this and the other experiments are based on the proportion of ``yes'' responses. We also analyzed the response times for the judgments. Before the main analyses, for each participant, we removed outlier response times that were 2.5 SD longer than the participant's mean response time. 2.2 Results 2.2.1 Analysis of judgments. As expected, participants judged trustworthy halves (mean, M ˆ 0:63, SDˆ 0:20) more positively than untrustworthy halves (M ˆ 0:27, SDˆ 0:12), (F 125, ˆ 153:61, p 5 0:001). More importantly, the effect of context was also significant (F 250, ˆ 32:63, p 5 0:001), (there were no other significant effects). As shown in figure 2, participants judged both trustworthy and untrustworthy halves more positively when aligned with trustworthy halves (M ˆ 0:51, SDˆ 0:16) than when aligned with untrustworthy halves (M ˆ 0:37, SDˆ 0:16), (t 25 ˆ 8:64, p 5 0:001). Participants' judgments were largely influenced by the negative context (untrustworthy halves). Whereas judgments of single facial halves (M ˆ 0:48, SDˆ 0:15) were more positive than judgments of these halves in the context of untrustworthy halves (t 25 ˆ 5:67, p 5 0:001), they were not significantly different from judgments in the context of trustworthy halves (t 25 ˆ 1:62, p ˆ 0:12). Mean trustworthiness 1.0 0.8 0.6 0.4 0.2 0.0 trustworthy untrustworthy Trustworthiness of `to-be-judged' halves single halves halves aligned with trustworthy halves halves aligned with untrustworthy halves Figure 2. Mean trustworthiness judgments of trustworthy and untrustworthy facial halves as a function of their alignment with trustworthy and untrustworthy halves (experiment 1). Error bars show 1 SEM. (1) Note that the stimuli could also be coded in terms of the congruence of the to-be-judged and to-be-ignored halves rather than the valence of the to-be-ignored halves. Statistically, the interaction effect of congruence and trustworthiness is equivalent to the main effect of context when the stimuli are coded in terms of their valence.

Obligatory holistic processing 519 2.2.2 Analysis of response times. The analyses revealed two significant effects: an interaction of context and trustworthiness (F 252ˆ, 5:41, p 5 0:007), and a three-way interaction of context, trustworthiness, and instructions (F 252, ˆ 3:51, p 5 0:037). When the analysis was limited to judgments of whole faces (and not single halves; there were no significant effects for the analysis of judgments of halves), the latter interaction was not significant. The interaction of context and trustworthiness remained significant (F 126, ˆ 11:29, p 5 0:002), indicating that, whereas participants were faster at judging trustworthy halves in positive contexts than in negative contexts, they were faster at judging untrustworthy halves in negative than in positive contexts. (2) In other words, participants were faster at judging halves in congruent contexts (M ˆ 1170 ms, SD ˆ 657 ms) than in incongruent contexts (M ˆ 1213 ms, SD ˆ 684 ms) (see table 1). Table 1. Differences in response times between judgments of facial halves aligned with halves congruent in valence and judgments of halves aligned with halves incongruent in valence, significance values of the differences, and effect sizes for all experiments. Experiment n Difference=ms Significance, p Effect size, d 1 27 43 0.002 0.06 2 48 60 0.009 0.14 3 20 38 0.212 0.15 4a 38 35 0.001 0.21 4b 39 31 0.001 0.16 2.3 Discussion This experiment provided evidence that trustworthiness judgments from emotionally neutral faces rely on holistic representations. Although participants were instructed to ignore facial information, they were unable to do so. Specifically, participants perceived the same facial halves more positively when aligned with trustworthy than when aligned with untrustworthy halves. This effect did not depend on whether participants were instructed to pay attention to the lower or the upper half of the face. Further, the experiment also provided evidence that facial cues indicating untrustworthiness are more heavily weighted in judgments than cues indicating trustworthiness. When halves were aligned with untrustworthy halves, judgments changed to a larger extent than when they were aligned with trustworthy halves. 3 Experiment 2 The signature of holistic face processing is that it is dependent on the orientation of the face (Maurer et al 2002). That is, to the extent that trustworthiness judgments rely on holistic processing, the difference between judgments of halves aligned with trustworthy halves and judgments of halves aligned with untrustworthy halves should be reduced when the faces are inverted. To test this prediction, experiment 2 included trials on which the faces were inverted. Because the instructions to attend to the upper or lower half of the face did not affect judgments in experiment 1, participants were instructed to attend to the upper half of the face. 3.1 Method 3.1.1 Participants. Fifty-five undergraduate students participated in the experiment for partial course credit. (2) This interaction effect is equivalent to a test of the main effect of congruence. Because across all four experiments, this effect did not consistently vary for judgments of trustworthy and untrustworthy halves, we report the overall congruence effect collapsed across these judgments.

520 A Todorov, V Loehr, N N Oosterhof 3.1.2 Face stimuli. We randomly selected 40 of the 60 base faces from experiment 1. As explained in section 2.1.2, each facial identity was a source of four different faces (untrustworthy, trustworthy, and two composite versions), resulting in 160 faces. In addition we included the upper halves (eye region) of the 40 trustworthy and 40 untrustworthy faces. To avoid repetition of the facial stimuli between upright and inverted faces, the 40 base faces were randomly split into two sets, each set generating 120 stimuli. These two sets were counterbalanced across participants. 3.1.3 Procedures. The procedures were the same as in experiment 1. Participants were told that they would see a series of faces, some upright, others inverted, and others with just the eye region of the face, and were instructed to attend only to the eye region of the face and to ignore the rest of the face. The experiment began with three practice trials to familiarize participants with the procedure. The order of the trials was randomized for each participant. The overall design was a 2 (face position: upright vs inverted)62 (trustworthiness of judged half: trustworthy vs untrustworthy)63 (context: single half vs half aligned with a trustworthy half vs half aligned with an untrustworthy half) within-subjects design ANOVA. 3.2 Results 3.2.1 Analysis of judgments. As in experiment 1, both main effects of trustworthiness and context were significant. Participants judged trustworthy halves (M ˆ 0:69, SDˆ 0:19) more positively than untrustworthy halves (M ˆ 0:34, SDˆ 0:16), (F 154ˆ, 327:27, p 5 0:001). Participants also judged both trustworthy and untrustworthy halves more positively when aligned with trustworthy (M ˆ 0:56, SD ˆ 0:16) than with untrustworthy halves (M ˆ 0:43, SD ˆ 0:18), (t 54 ˆ 10:48, p 5 0:001), (F 2, 108 ˆ 43:34, p 5 0:001 for the overall effect). As in experiment 1, whereas judgments of single halves (M ˆ 0:54, SD ˆ 0:17) were more positive than judgments of these halves in the context of untrustworthy halves (t 54 ˆ 6:04, p 5 0:001), they were not significantly different from judgments of the halves in the context of trustworthy halves (t 54 ˆ 1:88, p ˆ 0:07). More important, both main effects of trustworthiness and context were qualified by interactions with the position of the face, indicating that both effects were significantly reduced when the faces were inverted (figure 3). The difference between judgments of trustworthy and untrustworthy halves was reduced from 0.43 (SD ˆ 0:17), (t 54 ˆ 18:81, p 5 0:001) when the stimuli were upright to 0.27 (SD ˆ 0:17), (t 54 ˆ 12:04, p 5 0:001) when they were inverted (F 154, ˆ 46:27, p 5 0:001 for the interaction). Similarly, the difference between judgments of halves in positive and negative contexts was reduced from 0.21 (SD ˆ 0:14), (t 54 ˆ 11:43, p 5 0:001) when the faces were upright to 0.06 (SD ˆ 0:10), (t 54 ˆ 4:44, p 5 0:001) when they were inverted (F 154 ˆ 58:27, p 5 0:001 for the interaction). (3), Finally, there was a significant interaction of trustworthiness and context (F 2, 108 ˆ 8:04, p 5 0:001), primarily indicating that the effect of context was larger for judgments (3) Although the effect was substantially reduced for inverted faces, it was nevertheless significant, suggesting that such context effects could be detected in studies which have sufficient statistical power. In fact, the size of the current effect is comparable to the effect reported in the only other study of the composite-face effect in social judgments (Abbas and Duchaine 2008). In the latter study, participants preferred the upper halves of faces that were aligned with attractive lower halves. Although this effect was substantially reduced for inverted faces, it was higher than chance when the faces were aligned (M ˆ 0:55) but not when the faces were misaligned (M ˆ 0:51). This difference was marginally significant but the sample size in the study was smaller (n ˆ 25) than the sample size for the current experiment (n ˆ 55). Further, small but significant composite-face effects for inverted faces have also been observed in other paradigms (Goffaux and Rossion 2006, their experiment 3).

Obligatory holistic processing 521 Mean trustworthiness (a) 1.0 0.8 0.6 0.4 0.2 0.0 single halves halves aligned with trustworthy halves halves aligned with untrustworthy halves trustworthy untrustworthy trustworthy untrustworthy Trustworthiness of `to-be-judged' halves Trustworthiness of `to-be-judged' halves (b) Figure 3. Mean trustworthiness judgments of trustworthy and untrustworthy facial halves as a function of their alignment with trustworthy and untrustworthy halves for (a) upright faces and (b) inverted faces (experiment 2). Error bars show 1 SEM. of trustworthy (M ˆ 0:75, SD ˆ 0:18 vs M ˆ 0:59, SD ˆ 0:22) than untrustworthy (M ˆ 0:39, SDˆ 0:18 vs M ˆ 0:28, SDˆ 0:17) halves. (4) 3.2.2 Analysis of response times. The only significant effect was a three-way interaction of context, trustworthiness, and face position (F 294, ˆ 11:71, p 5 0:001). There were no significant effects for judgments of inverted faces. For upright faces, the only significant effect was the interaction of trustworthiness and context (F 294, ˆ 11:02, p 5 0:001 for all judgments, and F 147, ˆ 7:39, p 5 0:009 for judgments of whole faces). As in experiment 1, participants were faster at judging halves in congruent contexts (M ˆ 1156 ms, SD ˆ 423 ms) than in incongruent contexts (M ˆ 1216 ms, SD ˆ 444 ms), (see table 1). 3.3 Discussion When the faces were upright, the pattern of findings was very similar to the pattern of findings of experiment 1 (figure 2 and figure 3a), showing strong evidence for the composite-face effect. The same facial halves were judged more positively when aligned with trustworthy than with untrustworthy halves. This effect was substantially reduced when the faces were inverted (figure 3b). Similarly, the difference between judgments of trustworthy and untrustworthy halves was reduced when the faces were inverted. Finally, this experiment also replicated the finding that negative contexts were more potent in changing face perception than positive contexts, consistent with evidence that negative information is weighted more heavily than positive information in social judgments (Rozin and Royzman 2001). 4 Experiment 3 In this experiment, in addition to face orientation, we also manipulated the alignment of facial halves to further demonstrate that the changes in judgments as a function of contexts are due to a proper composite-face effect or a fusion of the facial halves. Specifically, in aligned- and misaligned-halves conditions (see figure 1), the information is identical and, hence, differences in judgments could be attributed to differences in (4) Although this interaction was not significant in experiment 1, most likely because of lack of statistical power, the pattern of means was similar (M ˆ 0:70, SDˆ 0:22 vs M ˆ 0:53, SDˆ 0:22, for judgments of trustworthy halves in positive and negative contexts, respectively; and M ˆ 0:32, SD ˆ 0:15 vs M ˆ 0:20, SDˆ 0:12, for judgments of untrustworthy halves). The interaction was marginally significant (F 1 25ˆ 3:93, p 5 0:059) when the analysis was limited to whole faces. In fact, the effect size, Z 2, ˆ 0:13, was slightly larger than the effect size for the corresponding interaction in experiment 2, involving upright faces, Z 2 ˆ 0:11. The same considerations apply for experiment 3, where the corresponding effect size for upright, aligned faces was 0.10 (see figure 4a).

522 A Todorov, V Loehr, N N Oosterhof processing of this information. We expected that the difference between judgments of halves aligned with trustworthy halves and judgments of halves aligned with untrustworthy halves should be reduced when the faces are misaligned or inverted. In other words, this difference should emerge only for upright, aligned faces. 4.1 Method 4.1.1 Participants. Twenty undergraduate students participated in the experiment for $8 payment. 4.1.2 Face stimuli. To create the misaligned faces, we employed a MatLab script to systematically alter the faces from experiment 2. Each image (face and black background combined) was 2986298 pixels. The faces alone were 1926136 pixels. Each whole face was cut at 65 pixels down from the top of the face to create two halves. The top part of each upright image was shifted 34 pixels to the left and the bottom part of each upright image was shifted 34 pixels to the right (see third row of figure 1). The bottom part (eye area) of each inverted image was shifted 34 pixels to the left, and the top part (mouth area) of each inverted image was shifted 34 pixels to the right. This ensured that the rightmost edge of the top part of the face was aligned exactly halfway with the rightmost edge of the bottom part of the face, so that the midpoint of the bridge of the nose was aligned with the left edge of the face (or would be if rotated 1808 counterclockwise for inverted faces). 4.1.3 Procedures. The instructions, images, and procedures were the same as in experiment 2. The major differences were that (a) we did not use single facial halves and (b) the halves were presented not only as aligned but also as misaligned. For each participant, 20 of the base face identities appeared in all four combinations (untrustworthy, trustworthy, and two composite versions) in upright position, and the other 20 appeared in all combinations in inverted position; these 160 images were also presented as misaligned (see third row of figure 1). The order of the 320 trials was randomized for each participant. The overall design was a 2 (face position: upright vs inverted)62 (alignment: halves aligned vs misaligned)62 (trustworthiness of judged half: trustworthy vs untrustworthy)62 (context: half aligned with a trustworthy half vs half aligned with an untrustworthy half) within-subjects design ANOVA. 4.2 Results 4.2.1 Analysis of judgments. As in the first two experiments, both main effects of trustworthiness and context were significant. Participants judged trustworthy halves (M ˆ 0:63, SDˆ 0:21) more positively than untrustworthy halves (M ˆ 0:33, SDˆ 0:19), (F 119, ˆ 40:43, p 5 0:001). Participants also judged both trustworthy and untrustworthy halves more positively when aligned with trustworthy (M ˆ 0:51, SDˆ 0:17) than with untrustworthy (M ˆ 0:46, SDˆ 0:18), (F 119, ˆ 11:42, p 5 0:003) halves. More importantly, the effect of context was modulated by both the orientation and the alignment of the faces. Replicating experiment 2, the difference between judgments of halves in positive and negative contexts was reduced from 0.09 (SD ˆ 0:09; t 19 ˆ 4:33, p 5 0:001), when the faces were upright to 0.01 (SD ˆ 0:06; t 5 1), when they were inverted), (F 119, ˆ 15:21, p 5 0:001 for the interaction). Similarly, the difference between judgments of halves in positive and negative contexts was reduced from 0.09 (SD ˆ 0:11; t 19 ˆ 3:67, p 5 0:002), when the faces were aligned to 0.01 (SD ˆ 0:08; t 5 1) when they were misaligned (F 119, ˆ 5:71, p 5 0:027 for the interaction). As in experiment 2, the orientation of the faces modulated the effect of trustworthiness. The difference between judgments of trustworthy and untrustworthy halves was reduced from 0.34 (SD ˆ 0:24; t 19 ˆ 6:13, p 5 0:001), when the stimuli were upright to 0.25 (SD ˆ 0:19; t 19 ˆ 5:82, p 5 0:001), when they were inverted (F 119 ˆ 6:02, p 5 0:024 for the interaction).,

Obligatory holistic processing 523 There were two other significant effects in the overall analysis. The interaction of position and alignment was significant (F 119, ˆ 18:94, p 5 0:001), indicating that judgments of misaligned halves were more positive than judgments of aligned halves when the faces were upright but not when the faces were inverted. Finally, there was a significant four-way interaction (F 119, ˆ 8:03, p 5 0:011). To establish that this interaction does not qualify the interpretation of the effects of context and trustworthiness as a function of face orientation and alignment, we conducted separate analyses for (a) upright, aligned faces (figure 4a); (b) upright, misaligned faces (figure 4b); (c) inverted, aligned faces (figure 4c); and (d) inverted, misaligned faces (figure 4d). The effect of context was significant only for upright, aligned faces (F 119, ˆ 17:60, p 5 0:001), and not for any of the remaining combinations of orientation and alignment. In contrast, the effect of trustworthiness was significant for all four combinations, although its size was largest for upright, aligned faces. The interaction of trustworthiness and context was not significant in any of the analyses (but see footnote 4). 1.0 halves aligned with trustworthy halves halves aligned with untrustworthy halves Mean trustworthiness Mean trustworthiness (a) 0.8 0.6 0.4 0.2 0.0 1.0 0.8 0.6 0.4 0.2 (b) 0.0 trustworthy untrustworthy trustworthy untrustworthy Trustworthiness of `to-be-judged' halves Trustworthiness of `to-be-judged' halves (c) (d) Figure 4. Mean trustworthiness judgments of trustworthy and untrustworthy facial halves as a function of their alignment with trustworthy and untrustworthy halves for (a) upright, aligned faces; (b) upright, misaligned faces; (c) inverted, aligned faces; and (d) inverted, misaligned faces (experiment 3). Error bars show 1 SEM. 4.2.2 Analysis of response times. The only significant effect was the interaction of face position and alignment (F 119, ˆ 5:77, p 5 0:027), indicating that participants were faster at judging aligned than misaligned parts when they were inverted but not when they were upright. Although the congruence effect observed in experiments 1 and 2 was not significant, it was comparable in size for upright, aligned faces (table 1). Participants were faster at judging halves in congruent contexts (M ˆ 961 ms, SD ˆ 261 ms) than in incongruent contexts (M ˆ 999 ms, SD ˆ 261 ms), (t 19 ˆ 1:29, p ˆ 0:21). We return to this issue after we report the results of the next two experiments.

524 A Todorov, V Loehr, N N Oosterhof 5 Experiment 4 Although participants were unable to completely disregard information from the task-irrelevant facial halves, the pattern of means for the conditions in which the implications of the to-be-judged and to-be-ignored halves were inconsistent (eg trustworthy and untrustworthy) suggests that participants were able to partially control the influence of context. For example, in all three experiments, participants judged trustworthy halves aligned with untrustworthy halves more positively than untrustworthy halves aligned with trustworthy halves. As described in section 5.2, these differences were significant for all experiments. The objective of this experiment was to explore the processes underlying the emergence and potential top ^ down control of holistic processing of faces. Because the faces were presented for unlimited time in the previous studies, it is unclear whether obligatory, bottom ^ up driven holistic processing could have emerged before potential top ^ down control. To test this hypothesis, we presented trustworthy and untrustworthy halves aligned with either trustworthy or untrustworthy halves for 33, 50, 67, 100, 167, and 500 ms. We were specifically interested in whether context effects would emerge after exposures as brief as 33 ms and whether top ^ down control or effects of instructions would emerge after longer exposures. Consider judgments of faces with inconsistent to-be-judged and to-be-ignored (task-irrelevant context) halves. If participants are unable to ignore the task-irrelevant context, context effects should emerge after extremely brief exposures. That is, participants should not be able to discriminate between trustworthy and untrustworthy halves when aligned with untrustworthy and trustworthy halves, respectively. However, with longer exposures, they should be able to discriminate between these halves, suggesting that additional time allows for top ^ down control of face processing. Alternatively, if the pattern of discrimination remains stable across time so that trustworthy halves aligned with untrustworthy halves are judged more positively than untrustworthy halves aligned with trustworthy halves, this would suggest that even after extremely brief exposures, participants could exercise some control on holistic processing of faces. In addition to the main experiment, in which participants were instructed to attend to the upper half of the face and ignore the lower half, we conducted a control experiment in which participants were not given specific instructions to attend to any part of the face. The objective of this experiment was to rule out the possibility that potential differences in judgments can be attributed to processing biases to weigh information from one half of the face more heavily than information from the other half. Finally, these experiments also allowed us to test an additional hypothesis. In contrast to previous experiments that used rapid exposure to faces (Bar et al 2006; Todorov et al 2009; Willis and Todorov 2006), the trustworthy and untrustworthy faces (faces for which the to-be-judged and to-be-ignored halves were congruent) were generated to clearly fall into two distinct categories. Thus, it is possible to test whether the change in judgments is different for trustworthy and untrustworthy faces as a function of time exposure. For example, previous studies have shown that participants are more sensitive to changes at the negative than at the positive end of the trustworthiness dimension (Oosterhof and Todorov 2008). Consistent with this finding, judgments of untrustworthy faces may be affected to a lesser extent than judgments of trustworthy faces by the increase in time exposure. That is, participants would need less exposure to form a judgment of untrustworthy than of trustworthy faces. 5.1 Method 5.1.1 Participants. Thirty-eight undergraduate students participated in the main experiment (4a) and thirty-nine participated in the control experiment (4b) for partial course credit.

Obligatory holistic processing 525 5.1.2 Face stimuli. The same stimuli as in experiment 1 were used, except that we did not use single halves. 5.1.3 Procedures. In experiment 4a, participants were told to make judgments based solely on the upper half of the face and to ignore the lower half. In experiment 4b, they were not given specific instructions to attend to any part of the face. These were the only differences between the experiments. The task began with three practice trials. Each trial started with a fixation point ( ) presented for 500 ms at the center of the screen. Then, a face was presented either for 33, 50, 67, 100, 167, or 500 ms and immediately replaced by a mask. The word ``Trustworthy?'' was presented above the mask. The mask remained on the screen until the participant's response. Participants responded by pressing the À' key on the keyboard, which was labeled `yes', or by pressing the semicolon `;' key on the keyboard, which was labeled `no'. The inter-trial interval was 1000 ms. The order of the trials was randomized for each participant. To counterbalance the timing conditions between faces, the 60 original facial identities (corresponding to 240 face stimuli used in the experiments) were randomly divided into 6 groups of 10 facial identities. The corresponding 40 face stimuli (per group) were counterbalanced across timing conditions between participants. The overall design was a 2 (trustworthiness of judged part: trustworthy vs untrustworthy)62 (context: judged half aligned with a trustworthy half vs half aligned with an untrustworthy half)66 (time presentation: 33 vs 50 vs 67 vs 100 vs 167 vs 500 ms) within-subjects design ANOVA. Note that this design properly describes the main but not the control experiment, in which the first two factors are not defined for the participants. 5.2 Results 5.2.1 Experiment 4a. As expected, participants judged trustworthy halves more positively than untrustworthy halves and this difference increased with the increase in time exposure (figure 5a). Similarly, they judged halves more positively when aligned with trustworthy than with untrustworthy halves and this difference increased with the time exposure. Correspondingly, both main effects of trustworthiness (F 137, ˆ 116:63, p 5 0:001) and context (F 137, ˆ 127:76, p 5 0:001) were significant, but qualified with interactions with time exposure (F 5, 185 ˆ 18:80, p 5 0:001; and F 5, 185 ˆ 3:62, p 5 0:004, respectively); the main effect of time was also significant (F 5185 ˆ 3:49, p 5 0:005). Mean trustworthiness 1.0 0.8 0.6 0.4 0.2 trustworthy upper half aligned with trustworthy lower half trustworthy upper half aligned with untrustworthy lower half untrustworthy upper half aligned with trustworthy lower half untrustworthy upper half aligned with untrustworthy lower half 0.0 0 50 100 150 200 250 300 350 400 450 500 0 50 100 150 200 250 300 350 400 450 500 Time exposure to faces=ms Time exposure to faces=ms (a) (b) Figure 5. Mean trustworthiness judgments of trustworthy and untrustworthy facial halves as a function of their alignment with trustworthy and untrustworthy halves and time exposure to faces. Faces were presented for 33, 50, 67, 100, 167, and 500 ms. (a) Participants were instructed to judge the upper and ignore the lower halves of the faces (experiment 4a). (b) Participants were not given specific instructions to attend to any half of the face (experiment 4b). Error bars show 1 SEM.,

526 A Todorov, V Loehr, N N Oosterhof Importantly, both effects of context and trustworthiness were significant for 33 ms exposure (F 137, ˆ 11:62, p 5 0:002; and F 137, ˆ 9:78, p 5 0:003, respectively). Finally, there was a significant interaction of trustworthiness and context (F 137, ˆ 5:39, p 5 0:026), indicating that the effect of context was larger for judgments of trustworthy (M ˆ 0:72, SDˆ 0:20 vs M ˆ 0:52, SDˆ 0:21) than untrustworthy (M ˆ 0:44, SD ˆ 0:28 vs M ˆ 0:28, SDˆ 0:19) halves. Next, we analyzed the judgments of faces for which the valence of the to-be-judged and to-be-ignored halves were inconsistent. In all three previous experiments, although participants were unable to ignore the effect of context, they judged upright trustworthy halves aligned with untrustworthy halves more positively than untrustworthy halves aligned with trustworthy halves (M ˆ 0:53, SDˆ 0:22 vs M ˆ 0:32, SDˆ 0:15; t 25 ˆ 6:89, p 5 0:001, in experiment 1; M ˆ 0:54, SD ˆ 0:25 vs M ˆ 0:35, SD ˆ 0:21; t 54 ˆ 6:07, p 5 0:001, in experiment 2; M ˆ 0:55, SD ˆ 0:26 vs M ˆ 0:34, SD ˆ 0:17; t 19 ˆ 4:33, p 5 0:001, in experiment 3). The findings in the 500 ms exposure condition, the condition closest to the conditions in the first three experiments, were very similar (M ˆ 0:62, SD ˆ 0:22 vs M ˆ 0:43, SD ˆ 0:21; t 37 ˆ 4:86, p 5 0:001). These findings show that the effect of context did not completely trump the effect of trustworthiness of the judged facial half, suggesting that participants were able to partially control the effect of context. However, as shown in figure 5a (the middle two lines), after 33 ms exposure to faces, judgments of trustworthy halves aligned with untrustworthy halves and judgments of untrustworthy halves aligned with trustworthy halves were indistinguishable. Significant differences in judgments emerged only after 100 ms exposure (t 35 ˆ 2:43, p 5 0:020). These findings are consistent with the hypothesis that additional time exposure allowed participants to focus on the judged half and partially ignore the context half. 5.2.2 Experiment 4b. Because participants in the main experiment were instructed to focus on the eye region of the face, an alternative interpretation of the above findings is that the eye information is more heavily weighed in participants' judgments; hence; the more positive judgments of trustworthy upper halves aligned with untrustworthy lower halves. However, this interpretation is inconsistent with the control experiment in which participants were not given instructions to focus on any facial half (figure 5b). Under those conditions, there were no significant differences between judgments of upper trustworthy halves aligned with untrustworthy halves and judgments of upper untrustworthy halves aligned with trustworthy halves. If anything, for exposures shorter than 500 ms, the latter judgments were more positive. 5.2.3 Judgments of trustworthy and untrustworthy faces in both experiments. The findings for judgments of faces with congruent halves were similar in both experiments. First, differences between trustworthy and untrustworthy faces were significant after 33 ms exposure (t 37 ˆ 4:24, p 5 0:001 in experiment 4a; and t 38 ˆ 9:57, p 5 0:001 in experiment 4b). Second, these differences increased over time (F 5, 185 ˆ 21:51, p 5 0:001 in experiment 4a; and F 5, 190 ˆ 30:79, p 5 0:001 in experiment 4b) for the interaction of trustworthiness and time. Third, this increase was primarily driven by the change in judgments of trustworthy faces (figure 5). Although the linear component was significant for judgments of untrustworthy faces in experiment 4a (F 137, ˆ 13:48, p 5 0:001), it was smaller than the linear component for judgments of trustworthy faces (F 137, ˆ 31:64, p 5 0:001). In experiment 4b, the linear component was not significant for judgments of untrustworthy faces (F 5 1) but was significant for judgments of trustworthy faces (F 138, ˆ 58:80, p 5 0:001). 5.2.4 Analysis of response times. In experiment 4a, with increased exposure to faces, participants' judgments became faster (F 5 185 ˆ 30:29, p 5 0:001). In addition to the,

Obligatory holistic processing 527 main effect of time, there were significant interactions of trustworthiness and context (F 137, ˆ 31:98, p 5 0:001), and of trustworthiness, context, and time (F 5185, ˆ 2:56, p 5 0:029). The former interaction indicated that, as in the previous experiments (table 1), participants were faster at judging halves in congruent contexts (M ˆ 563 ms, SD ˆ 157 ms) than in incongruent contexts (M ˆ 599 ms, SD ˆ 167 ms). The three-way interaction indicated that this congruence effect was larger in the 500 ms condition (M ˆ 446 ms, SD ˆ 132 ms vs M ˆ 532 ms, SD ˆ 228 ms) than in the other conditions (M ˆ 587 ms, SD ˆ 169 ms vs M ˆ 612 ms, SD ˆ 165 ms). When the analysis excluded the 500 ms condition, the three-way interaction was no longer significant, although the interaction of context and trustworthiness remained significant (F 137, ˆ 16:03, p 5 0:001). In experiment 4b, the only significant effects were the effects of time (F 5190, ˆ 3:96, p 5 0:002) and the interaction of trustworthiness and context (F 138, ˆ 12:15, p 5 0:001), (F 5 1 for three-way interaction). As in experiment 4a, participants' judgments became faster with increased exposure to faces, although the effect was not as large. Participants were also faster at judging halves in congruent contexts (M ˆ 549 ms, SD ˆ 175 ms) than in incongruent contexts (M ˆ 580 ms, SD ˆ 192 ms). As noted above, with the increased exposure to faces there was a greater change in judgments of trustworthy than untrustworthy faces. However, there were no such differences in response times for the judgments. In both experiments, when the analysis was limited to trustworthy and untrustworthy faces, the only significant effect was the effect of time (Fs 5 1 for all effects involving face trustworthiness). In both experiments, judgments became faster with the increased exposure to faces. However, one interesting difference between the two experiments was that the rate of change in response speed was greater in experiment 4a than in experiment 4b. As shown in figure 6, this change was linear in the latter experiment (F 137, ˆ 6:93, p 5 0:012 and F 5 1 for the quadratic trend), but nonlinear in the former experiment (F 137, ˆ 49:65, p 5 0:001 for the linear trend, and F 137, ˆ 22:93, p 5 0:001 for the quadratic trend). At the level of the mean response times, a logarithmic function best described the change of response times in experiment 4a. Response times=ms 700 650 600 550 500 450 attend to upper part only (experiment 4a) no specific instructions (experiment 4b) 400 0 50 100 150 200 250 300 350 400 450 500 Time exposure to faces=ms Figure 6. Response times for judgments as a function of time exposure to faces and experiment. In experiment 4a, participants were instructed to judge the upper and ignore the lower halves of the faces. In experiment 4b, participants were not given specific instructions to attend to any half of the face. The lines show the best fitted functions (logarithmic in experiment 4a and linear in experiment 4b). Error bars show 1 SEM. 5.2.5 Analysis of response times across experiments. As shown in table 1, in all experiments there was a small but consistent congruence effect. Participants were faster to make judgments of facial halves aligned with halves that were congruent in their valence than of halves that were incongruent in valence. Across experiments, this difference

528 A Todorov, V Loehr, N N Oosterhof was about 41 ms for upright, aligned faces and was not apparent for judgments of inverted ( 13 ms in experiments 2 and 3) or misaligned faces (6 ms in experiment 3). The average effect size (d), (5) weighed by the respective sample sizes for the studies was 0.15, and the effect sizes across studies were highly homogeneous (w 2 4 ˆ 0:09). These findings are broadly consistent with findings from composite face paradigms, in which there is a well-defined correct response. As in these paradigms, judgments are slower when the facial halves are incongruent. 5.3 Discussion After extremely brief exposures to faces, participants were unable to discriminate between upper trustworthy halves aligned with untrustworthy halves and upper untrustworthy halves aligned with trustworthy halves. However, with exposures longer than 100 ms, they judged the former more positively than the latter. These findings are consistent with the hypotheses that bottom ^ up driven, holistic processing of faces is obligatory and that additional time allows for partial top ^ down control of task-irrelevant information. The control is partial because the halves that participants were instructed to ignore nevertheless had a large effect on their judgments (see figure 5a). The hypothesis that partial top ^ down control of task-irrelevant information requires longer exposures to faces than bottom ^ up driven, holistic processing is consistent with the response times in the two experiments. As shown in figure 6, participants who were instructed to attend to the upper halves of the faces and ignore the lower halves (experiment 4a) were slower than participants who were not given such instructions (experiment 4b) for exposures shorter than 100 ms but faster for exposures longer than 100 ms. The effect of partial top ^ down control on judgments emerged with the latter exposures. Consistent with previous findings (Todorov et al 2009), differences between judgments of trustworthy and untrustworthy faces were detectable after 33 ms exposure to faces. These differences substantially increased with the increase in face exposure from 33 to 100 ms and relatively little with longer exposures. This effect was almost entirely driven by changes in judgments of trustworthy faces. Whereas these judgments become more positive with the increase in time exposure, judgments of untrustworthy faces changed to a much smaller extent. 6 General discussion All experiments in which participants were instructed to attend to one of the facial halves and ignore the other showed strong evidence for composite-face effect in social judgments. Judgments of the same halves were more positive when aligned with trustworthy than with untrustworthy halves despite instructions to ignore the latter. This effect was substantially reduced when the faces were inverted or the facial halves misaligned. These findings add to a long line of evidence for holistic processing of faces in a variety of tasks: perception of identity (Young et al 1987), emotions (Calder et al 2000), gender (Baudouin and Humphreys 2006), race (Michel et al 2007), and attractiveness (Abbas and Duchaine 2008). The robustness of the composite-face effect across tasks is consistent with the idea of a common representational system that allows for extraction of multiple judgments and categorizations of faces (Calder and Young 2005). Along those lines, we have argued that trustworthiness judgments are based on resemblance of emotionally neutral faces to emotions signaling approach/avoidance behaviors (Oosterhof and Todorov 2008, 2009; Todorov et al 2008). (5) To estimate the effect sizes, we used the formula developed by Dunlap et al (1996). This formula takes into account the correlation between the dependent variables, which is critical for significance tests of repeated-measures designs.