View-Specific Coding of Face Shape Linda Jeffery, 1 Gillian Rhodes, 1 and Tom Busey 2

Transcription

1 PSYCHOLOGICAL SCIENCE Research Article View-Specific Coding of Face Shape Linda Jeffery, 1 Gillian Rhodes, 1 and Tom Busey 2 1 University of Western Australia, Crawley, Australia, and 2 Indiana University, Bloomington ABSTRACT Monkey and human cortex contain view-specific face neurons, but it remains unclear whether they code face shape. We tested the view specificity of faceshape coding by inducing figural face aftereffects at one viewpoint (3/4 left) and testing generalization to different viewpoints (front view and 3/4 right). The aftereffects were induced by adaptation to consistent figural distortions (contracted or expanded), which shifts the distortion perceived as most normal toward the adapting distortion. The strong aftereffect that was observed at the adapting view was significantly and substantially reduced for both front-view test faces and mirror-image (3/4 right) test faces, indicating view specificity. The limited transfer across mirror views is strong evidence of view specificity, given their figural similarity. The aftereffects survived a size change between adaptation and test faces (Experiment 2), a result that rules out low-level adaptation as an explanation. These results provide strong evidence that face-shape coding is view-specific. Human face identification is remarkably robust to changes in viewpoint considering that the images projected on the retina by two different faces seen from the same view are more similar than the images projected by the same face seen at different views. A single exposure to a face from one viewpoint can be sufficient for recognition of this face at other views (Bruce, Valentine, & Baddeley, 1987; Bülthoff & Edelman, 1992), although subtle decrements in recognition can occur as test views are rotated away from learned views (Liu & Chaudhuri, 2002; Newell, Chiroro, & Valentine, 1999; Troje & Bülthoff, 1996; Troje & Kersten, 1999). A central question in face and object perception is whether this robustness to changes in viewpoint reflects underlying view-invariant representations or the use of multiple view-specific representations (e.g., Biederman & Bar, 1999; Biederman & Gerhardstein, 1995; Hayward & Tarr, 1997; Tarr & Bülthoff, 1995). Single-cell recordings in the monkey inferotemporal cortex have identified both view-specific and view-invariant face-selective cells (Perrett et al., 1985, 1991; Wang, Tanaka, & Tanifuji, 1996). View-specific neural populations have also been found in human lateral occipital cortex, a region involved in high-level shape and object coding (Grill-Spector et al., 1999). These were identified using functional magnetic resonance (fmr) adaptation, in which repeated stimulation of the same neural population reduces the MR response. Little adaptation was observed when faces were presented at changed viewpoints, suggesting that different neural populations code different viewpoints. Further evidence for view-specific face neurons in humans comes from behavioral adaptation studies (Fang & He, 2005; Suzuki, 2005). For example, adapting to a face rotated 601 to the left makes a front-view test face appear rotated somewhat to the right, as would be predicted if adaptation changes the sensitivity of view-specific face neurons (Fang & He, 2005). However, none of these studies shows that view-specific representations code face shape, because view-specific neurons have not been directly linked to shape coding. Perrett, Hietanen, Oram, and Benson (1992) observed that cells in the macaque that are sensitive to identity are often view-selective, but also noted that most view-sensitive cells are not sensitive to identity and may code other important view-specific properties, such as eye gaze and head direction, which in humans are coded separately from identity (Haxby, Hoffman, & Gobbini, 2000, 2002). We directly investigated whether the representations of human face shape 1 underlying identity are view-specific by inducing figural shape aftereffects (Rhodes, Jeffery, Watson, Clifford, & Nakayama, 2003; Watson & Clifford, 2003; Webster & MacLin, 1999) in one viewpoint and testing for generalization Address correspondence to Linda Jeffery, School of Psychology, University of Western Australia, 35 Stirling Highway, Crawley, WA 6009, Australia, linda@psy.uwa.edu.au. 1 We use the term shape broadly to include any individuating information about the structure of a face. It could include details of individual features as well as their spatial relations. Volume 17 Number 6 Copyright r 2006 Association for Psychological Science 501

2 View Specificity of Face Coding to new viewpoints. If view-specific neurons code face shape, then such aftereffects should be significantly reduced or eliminated when views change between adaptation and test. Alternatively, if the neurons are view-invariant, then one would expect similar aftereffects for all test views. Given that viewspecific neurons may be broadly tuned (Perrett et al., 1991), we chose test views that differed by 451 and 901 from the adapting view to ensure that view-specific populations could be isolated. A systematic reduction in generalization as test view diverged from adapting view would be evidence for view-specific neurons with broad tuning. Given the figural similarity of a view and its mirror image, failure of the aftereffects to generalize to the mirror view would provide especially strong evidence for view specificity. We adapted participants to three-dimensional figural distortions in faces that were rotated 451 to the left (3/4 left) around the vertical axis and tested for aftereffects in faces at three viewpoints: the same viewpoint (3/4 left), the front view, and the mirror viewpoint (3/4 right). Faces were distorted by either expanding the internal features outward from the center of the face or contracting them inward. We measured the aftereffect by asking participants to rate the perceived normality of the test faces both before and after adaptation. Note that different faces were shown in the adaptation and test phases because we wanted to adapt the neurons coding face shape generally, not to adapt specific identities, as in the face-identity aftereffect (Leopold, O Toole, Vetter, & Blanz, 2001). Adaptation to a consistent distortion in faces of different identities can be thought of as shifting the norms used to code faces in the direction of the adapting distortion (Rhodes & Jeffery, in press). The size of the aftereffect was measured by the shift in the distortion perceived as most normal (see Rhodes et al., 2003). A second experiment included test faces that differed in size from the adapting faces, to rule out adaptation of low-level coding mechanisms as an explanation of our results (e.g., Zhao & Chubb, 2001). EXPERIMENT 1 Method Twenty-seven (4 male) University of Western Australia undergraduate psychology students participated for course credit. Stimuli Adapting and test faces were constructed from three-dimensional laser-scan models (see Busey & Zaki, 2004, for details). To remove some surface artifacts, we morphed together pairs of faces to create 16 male faces. To make the distortions, we warped control points on each face toward (contracted) or away from (expanded) the three-dimensional center of the face in percentage increments of the distance between each point and the center. The warping was applied to the texture and structural maps and then rendered using POV-Ray (Persistence of Vision Raytracer, ). Adapting faces comprised 8 male faces expanded 25% (125) or contracted 25% ( 25) and rotated 451 to the left (3/4 left). Eight different male faces were used to create the test stimuli. Eleven distortion levels, ranging from 25% contracted to 25% expanded ( 25, 15, 10, 6, 2, 0, 12, 16, 110, 115, 125; see Fig. 1), were created at each of three different views: rotated 451 left (3/4 left), front view (01), and a mirror image of the 451-left stimulus (3/4 right), created by flipping the image around the vertical axis. Fig. 1. Seven of the 11 distortion levels for a 3/4-left test face (top row), front-view test face (middle row), and 3/4-right test face (bottom row). The numbers across the bottom indicate the distortion levels. 502 Volume 17 Number 6

3 Linda Jeffery, Gillian Rhodes, and Tom Busey Faces were presented in color on a black background on a 19-in. monitor. Both the adapting and the test images measured 13.9 cm wide by 13.4 cm high and subtended approximate visual angles of at a viewing distance of approximately 40 cm. Procedure Participants were first exposed to the range of variation in normality in the stimulus set by viewing 32 of the test faces that spanned the distortion levels and viewpoints. They were then asked to rate how normal each of the 264 test faces looked, using a 10-point scale ranging from 1, unusual, to 10, normal. Each test face was shown for 1 s, surrounded by a box, with the instruction rate. The rating scale was then displayed and remained on the screen until the participant responded. Faces were presented in random order. An adaptation period followed. Participants were randomly assigned to one of the two adapting conditions (contracted or expanded). The eight adapting faces were shown repeatedly, in random order, for 750 ms each, with a 200-ms interstimulus interval (ISI) between faces. The presentation continued for a total of 2.5 min. The test faces were then rated again for normality. Each test face was preceded by a 7,600-ms top-up adaptation period (cf. Rhodes et al., 2003) in which the eight adapting faces were presented once each for 750 ms, with a 200- ms ISI, in random order. Trials were divided into eight 10-min blocks. The session took approximately 90 min to complete. Results The mean rating for each distortion level was calculated for each of the three test viewpoints both before and after adaptation, for each participant. The most normal distortion for each test view, before and after adaptation, was measured by fitting thirdorder polynomials to each set of distortion-level means and calculating the distortion at which the fitted curve reached a maximum. Three participants (one male) were excluded from further analyses because at least one of their functions was erratic and showed poor fit (R 2 <.7). This left 12 participants in each of the adapting conditions (contracted and expanded). To index the size of the aftereffect, we determined for each participant the amount the most normal distortion shifted in the predicted 2 direction after adaptation. We calculated this shift by subtracting the most normal distortion after adaptation from the most normal distortion before adaptation. All shifts in the predicted direction were assigned a positive sign, and shifts in the opposite direction were scored as negative. The means are shown in Figure 2. A one-way repeated measures analysis of variance (ANOVA) found a large significant effect of test view, F(2, 46) , p <.001, p rep >.999, Z Figure 2 shows that the most normal distortion shifted in the predicted direction, toward the adapting distortion, for test faces presented at the same view as 2 This direction was the direction predicted by a classic repulsive aftereffect. Fig. 2. Mean shift in the most normal distortion as a function of the view of the test face (3/4 left, front, 3/4 right), collapsed across adapting conditions. Error bars show 1 SE on either side of the mean. the adapting faces (3/4 left). A planned comparison showed that the shift in the most normal distortion was reduced when test faces were rotated 451 away from the adapting faces (front view), t(46) , p <.001, p rep >.99, d Similarly, the shift in the most normal distortion was significantly less for test faces presented at mirror views (3/4 right) than for test faces at the adapting view, t(46) , p <.001, p rep >.99, d > The size of the shift did not differ significantly between test faces presented at the front view and test faces presented 3/4 right, t(46) , p >.05, p rep 5.42, d A significant aftereffect occurred at all views (ts > 2.67, ps <.004, p rep s >.97; ds , 0.69, and 1.28 for 3/4 left, front, and 3/4 right, respectively). EXPERIMENT 2 Figural aftereffects in the perception of face shape generalize across changes in size (Zhao & Chubb, 2001) and tilt in the picture plane (Rhodes et al., 2003; Watson & Clifford, 2003), ruling out purely low-level retinotopic adaptation as the source of these aftereffects and implicating adaptation of high-level face-coding mechanisms (see also Rhodes et al., 2004). In Experiment 2, we sought to replicate Experiment 1 using test faces that differed in size from the adapting faces, to confirm that our results reflect adaptation of higher-level mechanisms. Method Twenty-eight undergraduates (11 male) participated for course credit. The stimuli and procedure were the same as in Experiment 1 except that participants used a chin rest to maintain a viewing distance of 47 cm from a 17-in. monitor. Also, adapting faces measured 15.8 cm wide by 15 cm high and subtended a visual angle of , and test faces measured 8.0 cm wide by 7.5 cm high and subtended a visual angle of Volume 17 Number 6 503

4 View Specificity of Face Coding Participants were randomly assigned to adapting condition (contracted or expanded). Results Aftereffects were measured as for Experiment 1. Four participants (3 male) were excluded from further analysis because their functions were erratic and showed poor fit (R 2 <.7), leaving 12 participants in each adapting condition (contracted and expanded). Figure 2 shows that the pattern of results found in Experiment 1 was replicated in Experiment 2. A one-way repeated measures ANOVA found a large significant effect of test view, F(2, 46) , p <.001, p rep >.99, Z The most normal distortion shifted in the predicted direction for test faces presented at all views. Planned comparisons showed that the shift was smaller when test faces were presented at the front view, t(46) , p <.001, p rep >.99, d , and the mirror view, t(46) , p <.001, p rep >.99, d , than when they were presented at the adapting (3/4 left) view. There was no significant difference in the size of the aftereffect for front-view and 3/4-right test faces, t(46) , p >.05, p rep 5.70, d Adaptation produced significant shifts in the mean most normal distortion at each viewpoint (ts > 3.00, ps <.003, p rep s >.97; ds , 0.72, and 1.06 for 3/4 left, front, and 3/4 right, respectively). DISCUSSION Consistently distorted faces presented at a 3/4-left view produced robust figural aftereffects in faces presented at the same view. A 451 change in view (to a front view) more than halved the aftereffect. This result suggests view-specific coding of face shape. For mirror-image test views (3/4 right), there was also a substantial reduction in the size of the aftereffect. The limited transfer across figurally similar mirror views is particularly noteworthy. These results provide strong evidence for view specificity. The same pattern of results was obtained with a size change between adapting and test views, ruling out purely low-level retinotopic adaptation as the source of the aftereffects. The reduction in aftereffect produced by a 451 change in view was also much greater than that produced by a large change in image size (cf. results for Experiments 1 and 2 in Fig. 2), further ruling out an account based on low-level features of the images. The data strongly suggest that face shape is coded by view-specific neurons. Figural adaptation, which generalizes across identities (as in the current study), systematically shifts the face-shape norm (Rhodes et al., 2003). The finding that shifts induced at one viewpoint show only limited transfer to other views indicates that norms for face shape may be view-specific (see also Jiang, Blanz, & O Toole, 2006, in this issue). The present results suggest that neurons coding face shape are sensitive to a 451 change in view, which is comparable to the view tuning of monkey view-specific face cells (Perrett et al., 1991). Tuning width is determined by the size of the reduction from the maximal response as viewpoint is changed. A drop to half the maximal response defines the range (Perrett et al., 1991). It therefore seems reasonable to interpret a reduction in the size of an aftereffect by more than half as indicative of view specificity and suggestive regarding the tuning range. However, Jiang et al. (2006) interpret a similar reduction in the size of face-identity aftereffects across views as consistent with viewindependent coding. Their data seem equally consistent with a view-specific account in which neurons coding face shape are broadly tuned. Future studies that parametrically vary the size of the view change between adapting and test faces will enable the tuning curves of human face neurons to be mapped. Good candidate locations for these view-specific shape-coding mechanisms are the lateral occipital cortex and fusiform face area, both of which appear to contain view-specific face-sensitive populations (Grill-Spector et al., 1999; Kourtzi & Grill-Spector, 2005). To conclude, we have used figural aftereffects to probe the view selectivity of human mechanisms for coding face shape. A 451 shift in test view more than halved the aftereffect, and the transfer was no greater for the figurally similar mirror view. Although view-invariant coding mechanisms may be able to handle limited view dependence (Hayward, 2003), they cannot handle these results. Studies of fmr adaptation have identified view-specific neurons in face-coding areas (Grill-Spector et al., 1999). The present results suggest that these neurons may play a role in coding face shape, and ultimately face identity. Acknowledgments This work was supported by the Australian Research Council. We thank Dave Perrett and Darren Burke for helpful discussions, and Daphne Maurer, the Face Lab (University of Western Australia), and two anonymous reviewers for their helpful comments on the manuscript. REFERENCES Biederman, I., & Bar, M. (1999). One-shot viewpoint invariance in matching novel objects. Vision Research, 39, Biederman, I., & Gerhardstein, P.C. (1995). Viewpoint-dependent mechanisms in visual object recognition: Reply to Tarr and Bülthoff (1995). Journal of Experimental Psychology: Human Perception and Performance, 21, Bruce, V., Valentine, T., & Baddeley, A. (1987). The basis of the 3/4 view advantage in face recognition. Applied Cognitive Psychology, 1, Bülthoff, H.H., & Edelman, S. (1992). Psychophysical support for a two-dimensional view interpolation theory of object recognition. Proceedings of the National Academy of Sciences, USA, 89, Busey, T.A., & Zaki, S.R. (2004). The contribution of symmetry and motion to the recognition of faces at novel orientations. Memory & Cognition, 32, Fang, F., & He, S. (2005). Viewer-centered object representation in the human visual system revealed by viewpoint aftereffects. Neuron, 45, Volume 17 Number 6

5 Linda Jeffery, Gillian Rhodes, and Tom Busey Grill-Spector, K., Kushnir, T., Edelman, S., Avidan, G., Itzchak, Y., & Malach, R. (1999). Differential processing of objects under various viewing conditions in the human lateral occipital complex. Neuron, 24, Haxby, J.V., Hoffman, E.A., & Gobbini, M.I. (2000). The distributed human neural system for face perception. Trends in Cognitive Sciences, 4, Haxby, J.V., Hoffman, E.A., & Gobbini, M.I. (2002). Human neural systems for face recognition and social communication. Biological Psychiatry, 51, Hayward, W.G. (2003). After the viewpoint debate: Where next in object recognition? Trends in Cognitive Sciences, 7, Hayward, W.G., & Tarr, M.J. (1997). Testing conditions for viewpoint invariance in object recognition. Journal of Experimental Psychology: Human Perception and Performance, 23, Jiang, F., Blanz, V., & O Toole, A.J. (2006). Probing the visual representation of faces with adaptation: Aview from the other side of the mean. Psychological Science, 17, Kourtzi, Z., & Grill-Spector, K. (2005). fmri adaptation: A tool for studying visual representations in the primate brain. In C.W.G. Clifford & G. Rhodes (Eds.), Fitting the mind to the world: Adaptation and aftereffects in high-level vision (pp ). Oxford, England: Oxford University Press. Leopold, D.A., O Toole, A.J., Vetter, T., & Blanz, V. (2001). Prototypereferenced shape encoding revealed by high-level aftereffects. Nature Neuroscience, 4, Liu, C.H., & Chaudhuri, A. (2002). Reassessing the 3/4 view effect in face recognition. Cognition, 83, Newell, F.N., Chiroro, P., & Valentine, T. (1999). Recognizing unfamiliar faces: The effects of distinctiveness and view. Quarterly Journal of Experimental Psychology, 52A, Perrett, D.I., Hietanen, J.K., Oram, M.W., & Benson, P.J. (1992). Organization and functions of cells responsive to faces in the temporal cortex. Philosophical Transactions of the Royal Society of London: Series B, Biological Sciences, 335, Perrett, D.I., Oram, M.W., Harries, M.H., Bevan, R., Hietanen, J.K., Benson, P.J., & Thomas, N.J. (1991). Viewer-centred and objectcentred coding of heads in the macaque temporal cortex. Experimental Brain Research, 86, Perrett, D.I., Smith, P.A., Potter, D.D., Mistlin, A.J., Head, A.S., Milner, A.D., & Jeeves, M.A. (1985). Visual cells in the temporal cortex sensitive to face view and gaze direction. Proceedings of the Royal Society of London: Series B, Biological Sciences, 223, Persistence of Vision Raytracer. ( ). POV-Ray [Computer software]. Available from Rhodes, G., & Jeffery, L. (in press). Adaptive norm-based coding of facial identity. Vision Research. Rhodes, G., Jeffery, L., Watson, T.L., Clifford, C.W.G., & Nakayama, K. (2003). Fitting the mind to the world: Face adaptation and attractiveness aftereffects. Psychological Science, 14, Rhodes, G., Jeffery, L., Watson, T.L., Jaquet, E., Winkler, C., & Clifford, C.W. (2004). Orientation-contingent face aftereffects and implications for face-coding mechanisms. Current Biology, 14, Suzuki, S. (2005). High-level pattern coding revealed by brief shape aftereffects. In C.W.G. Clifford & G. Rhodes (Eds.), Fitting the mind to the world: Adaptation and aftereffects in high-level vision (pp ). Oxford, England: Oxford University Press. Tarr, M.J., & Bülthoff, H.H. (1995). Is human object recognition better described by geon structural descriptions or by multiple views? Comment on Biederman and Gerhardstein (1993). Journal of Experimental Psychology: Human Perception and Performance, 21, Troje, N.F., & Bülthoff, H.H. (1996). Face recognition under varying poses: The role of texture and shape. Vision Research, 36, Troje, N.F., & Kersten, D. (1999). Viewpoint-dependent recognition of familiar faces. Perception, 28, Wang, G., Tanaka, K., & Tanifuji, M. (1996). Optical imaging of functional organization in the monkey inferotemporal cortex. Science, 272, Watson, T.L., & Clifford, C.W.G. (2003). Pulling faces: An investigation of the face-distortion aftereffect. Perception, 32, Webster, M.A., & MacLin, O.H. (1999). Figural aftereffects in the perception of faces. Psychonomic Bulletin & Review, 6, Zhao, L., & Chubb, C. (2001). The size-tuning of the face-distortion after-effect. Vision Research, 41, (RECEIVED 6/9/05; ACCEPTED 8/8/05; FINAL MATERIALS RECEIVED 10/10/05) Volume 17 Number 6 505

6