RUNNNING HEAD: Pigeons Attend to Features of Human Faces Pigeons and People Categorize Human Faces Using Similar Features Brett M. Gibson *, Edward A. Wasserman *, Frédéric Gosselin, and Philippe G. Schyns Address for Correspondence: Brett M. Gibson, Ph.D. The University of Iowa Department of Psychology 11 Seashore Hall East Iowa City, IA 52242-1407 Email: brett-gibson@uiowa.edu Ph: 319.335.2406 FAX: 319.335.0191
Bubbled faces... 2 Human faces contain rich information that allows us to recognize hundreds of different individuals and to discern several different emotional expressions 1. We carry out such intricate visual discriminations by attending to diagnostic features of the face 2-4. Perhaps such attentional deployment is performed by specialized neural and perceptual systems for processing face information 5-9. On the other hand, the processing of facial information might be performed by neural and perceptual systems that are devoted to processing other kinds of visual information 10,11. Here, we report that the visual features used by pigeons in human face discrimination tasks overlap considerably with those used by people 3. Organisms like pigeons, which presumably lack specialized systems for processing human faces, nevertheless similarly attend to diagnostic features of the human face. These results suggest that some aspects of facial processing may not require specially evolved brain circuitry. Human social interactions depend on the processing of information provided by faces 12-13. The advent of functional Magnetic Resonance Imaging (fmri) and Event Related Potential (ERP) technologies have allowed researchers to examine the brain activity associated with face processing by humans. Recent fmri studies appear to have isolated a specific area of the brain, the Fusiform Face Area (FFA), as the locus of face processing in humans 5-7. Likewise, recent ERP studies have suggested that the upright frontal view of a human face (or faces of other species, face photographs, paintings, and sketches) can elicit a negative potential 170 ms following stimulus onset (N170) that is typically larger than that to other stimuli 8,9,14-19. Although the results of these fmri and ERP studies seem to be convincing, several other reports have suggested that the processing of facial information may not
Bubbled faces... 3 require specalized neural circuitry and that the centers believed to be specialized for processing facial information may participate in many other visual tasks. For example, the activation of the FFA increases with expertise in recognising novel objects in general 10,11. Simalrly, an enhanced early ERP negative component (N170) appears when people categorise objects within their realm of expertise compared to when they categorise objects outside of that realm 20. If the processing of facial information can be performed by general visual systems, then the landscape of human facial features might be used by a wide variety of organisms or artificial perceptual systems to discriminate properties of human faces. Notably, features or regions of the human face that contain the most diagnostic visual information might be used during such discriminations and overlap with those features that are actually used by humans solving similar tasks. To find out, we used a new procedure called bubbles 3,4 to see what features of the face pigeons actually use when they discriminate human emotion and gender. Organisms, like pigeons, which presumably lack specialized systems for processing human faces, may nevertheless attend to diagnostic features of the human face using a general perceptual system. To ascertain the diagnostic information that such generic system would require, we used the data of an ideal observer resolving the same tasks 3. We trained four feral pigeons to discriminate 32 grayscale photographs of faces of eight human males and eight human females exhibiting either happy or neutral expressions (Figure 1a). One pair of birds learned to discriminate whether the face was a male or a female (Gender condition), whereas a second pair of birds learned to discriminate whether the face displayed a happy or a neutral expression (Emotion
Bubbled faces... 4 condition). Training lasted an average of 122 days (± 31 s.e.m.) for pigeons in the Gender condition and 79 days (± 13 s.e.m.) for pigeons in the Emotion condition, at which time all of the birds were averaging well over 70% correct responses. After training, we used the bubbles 3,4 procedure to pinpoint the features of the faces that the pigeons actually used to discriminate both human emotion and gender. Specifically, during testing, we presented the birds with the same 32 faces they had encountered during training, but each image was now only partly visible through a midgray mask (Figure 1b). The overlaying mask was punctured by several small punch holes or bubbles that revealed portions of the face below. From trial to trial, the bubbles were randomly positioned over each face so that, across blocks of extended testing, the entire face was disclosed and the sampling of face information was unbiased. We tested the birds in seven consecutive phases with approximately 320 bubbled images in each phase. The number of punch-holes per stimulus was systematically varied across the seven experimental phases to maintain performance between floor and ceiling levels of accuracy. Successive phases involved stimuli with 20, 30, 40, 50, 40, 30, and 20 punchholes, respectively. We then used the birds discrimination of these bubbled images to ascertain which features of the human faces controlled their behavior. One bird (31B) initially in the Emotion condition completed all seven phases of testing; it was later retrained and tested in the Gender condition. Thus, this bird served in both conditions. Accuracy scores during testing are shown in Table 1. Note that pigeon 31B performed both tasks more accurately than the other pigeons. The accuracy scores of birds in the Emotion condition tended to be somewhat higher than the accuracy scores of birds in the Gender condition, consistent with the difference in the speed of the pigeons
Bubbled faces... 5 discrimination learning. Likewise, human subjects in a companion study also had a more difficult time mastering the gender task than the emotion task 3. Accuracy to the testing stimuli progressively improved with increasing numbers of bubbles. Strong correlations were observed between the number of punch-holes and choice accuracy for each bird (i.e., r 23Y =.75, r 31B_emotion =.91, r 31B_gender =.83, r 72Y =.85, and r 52W = 0.71). Accuracy was generally higher during the second round of testing with the 20, 30, and 40 bubbles stimuli than during the first round of testing. Figure 2 presents the results from the bubbles analysis and displays the pattern of facial features that were used by pigeons in the current project and by humans and ideal observers in a companion project 3. The use of facial features by the pigeons is not random; rather, the birds use features of the face that vary with the task they are required to perform. Pigeons discriminating between happy and neutral expressions tended to use features in the bottom part of the face including the mouth, whereas pigeons discriminating gender used features in the top part of the face including the eyes. The switch in the use of facial features is particularly clear for the bird that served in both conditions (31B: Figure 2). Notably, in both tasks, pigeons use of the entire wealth of facial information was correlated with that of human and ideal observers, as revealed in the companion study 3 for both the Emotion (r emotion-humans =.4822, p < 0.01; r emotion-ideal =.3567, p < 0.01) and Gender r gender-humans =.1418, p < 0.01; r gender-ideal =.0352, p < 0.01) tasks. The bird that served first in the Emotion condition and then in the Gender condition (31B), also used facial features that overlapped with those used by humans (r 31B-emotion-humans =.4480, p < 0.01) and ideal observers (r 31B-emotion-ideal =.2711, p < 0.01) when discriminating emotion. The conglomeration of facial features that this bird used,
Bubbled faces... 6 however, changed following gender retraining and was significantly correlated with those used by human (r 31B-gender-humans =.1126, p < 0.01) and idea observers (r 31B-gender-ideal = -.0118,, p < 0.01also discriminating gender. Furthermore, the correlations between pigeons and humans were reliably larger than those between pigeons and ideal observers in both the Emotion, p < 0.05, and Gender, p < 0.05, discriminations. These results clearly indicate that pigeons use face information in much the same way as do humans, even though neither species optimally uses the most informative regions of human faces to solve the discrimination tasks. Briefly stated, pigeons and humans are similarly biased in their use of face information. We cannot be sure from the present data whether or not the pigeons saw the photographs as representations of human faces; picture-object equivalence in nonhuman animals is a matter of continuing controversy 21. Nor does the fact that our pigeons used particular facial features mean that they did not also use the spatial arrangement of those features in their visual discriminations 22, as has been suggested by other research 23. We are sure that ideal observers lack specialized circuitry for extracting gender and emotion from images of human faces. Nevertheless, ideals reveal the main sources of visual information to which both the human and the pigeon visual systems are sensitive. These dramatically different systems quite effectively discriminate images of human faces containing information about gender and emotion. Their ability to do so is impressive, because differences in human facial information can be quite subtle. The ideals analyses reveal that the features of the face that should command attention importantly depend on the discrimination itself. Interestingly, those features of the
Bubbled faces... 7 human face that support such discriminations by humans and pigeons are similar to one another. What are the implications of these results for humans perception of faces? Without denying the possibility of our species possessing and using specialized neural systems of face discrimination, our results suggest that the diagnostic information underlying emotion and gender discriminations can be extracted without these specialized systems. The ideals use of information suggest that dedicated would systems have specialized to extract the main sources of facial information in the tasks, something the pigeons did learn over the course of a few hours of their lifetime. Our findings thus join others in suggesting that facial information may be processed by general perceptual system 10,11 rather than by specially evolved centers 5-9. The processing of facial information for individual recognition could require specialized circuitry; such circuitry for individual recognition appears to be located in the temporal and frontal corticies of humans 24, monkeys 7, and sheep 25. Methods Subjects Four adult feral pigeons (Columba livia) were studied. The birds were individually housed and maintained at 85% of their ad lib weights by controlled feedings of mixed grain. The birds had free access to water that contained a vitamin supplement. All of the pigeons had served in other unrelated studies in which they were shown visual stimuli that were quite different than those used here. Procedure Each pigeon was trained and tested in one of four operant conditioning chambers 26. During each trial, 1 of the 32 face photographs was randomly selected and
Bubbled faces... 8 appeared in the center of a monitor in the front wall of the operant chamber. Each facial image (5.72 x 5.72 deg of visual angle) had normalized hairstyle, global orientation, and lighting 27 and was identical to the stimuli previously used during testing with humans and ideal observers 3,4. The pigeon then was required to peck at the face in the display area. Following that peck, red and green choice areas were illuminated to the upper left and upper right of the display area. One pair of birds was trained to discriminate whether the face was male or female (Gender condition), where another pair of birds was trained to discriminate whether the face displayed a happy or neutral expression (Emotion condition). The choice areas (red or green) and the display relationships were balanced within each condition. A correct choice resulted in the delivery of a food pellet reward into the chamber, whereas an incorrect choice resulted in the trial being repeated until a correct response was made. Acquisition continued until accuracy appeared asymptotic. Each daily session contained 5 blocks of 32 training trials. Each daily session of testing contained 6 blocks. The first block involved 32 warm-up trials that were identical to those given during training. During each of the 5 subsequent blocks, the pigeons were presented with the 32 training faces plus 2 bubbled testing faces (see below) from each of the four different display categories (male-happy, male-neutral, female-happy, female-neutral; see Figure 1b). Each bubble revealed a portion of the face below and was.18 degree of visual angle in size. The bubbles were positioned over and adjacent to the face according to a random distribution, so that each bubbled testing image was unique. Forty testing images were presented in each session. Each day of testing alternated with a day of training if the training criterion was maintained.
Bubbled faces... 9 The number of punch-holes per stimulus also was varied across seven experimental phases to maintain accuracy between floor and ceiling levels. Successive phases comprised stimuli with 20, 30, 40, 50, 40, 30, and 20 punch-holes, respectively. On average, there were 319 testing trials per phase (std = 2.98) from a maximum of 320. Bubbles analysis A correct response by the pigeon on a bubbled testing trial indicates either that the features of the face that were seen through the bubbled mask divulged enough diagnostic information to support a correct response or that the pigeon responded correctly by chance. An incorrect response suggests that the features of the face that were revealed on that trial were not adequate to support a correct response. We generated a ratio score that specified the number of times a specific region of the input space led to a successful response over the total number of times that region had been revealed during the course of the experiment. The ratio score of a particular pixel indicates the probability that the pigeon responded correctly when this particular pixel was revealed by a bubble. Because performance changes as functions of the information revealed and familiarity, we normalized (Z-score) the results within each phase that met our performance criteria (i.e., Z-score = (ratios mean(ratios)) / std(ratios)). The sum of all normalized ratio scores was itself normalized to enable comparisons between tasks and subject groups. We analyzed only those phases in which choice accuracy was significantly greater than chance (p <.05, with p derived from binomial tests: {N! / [i! * (N - i)] *.5 i *.5 N - i }, with N the number of trials per phase, i the number of correct trials, and a sum performed from i to N) 28. For the emotion task, 11 phases (out of 14) met criterion. For the gender task, 15 phases (out of 21) met criterion.
Bubbled faces... 10 To assist with the interpretation of results, a composite diagnostic mask was constructed; it associated the score for each region of the input space with a grayscale shade, so that input areas that had a score that was reliably above chance were open or bubbled, thereby revealing the regions associated with the face below. Regions of the input space that were used no more than chance were colored gray and those that were below chance were black. This composite diagnostic mask was then placed over a training image for analysis by visual inspection (Figure 2). Monte-Carlo simulations We performed Monte-Carlo simulations to assess the statistical significance of the correlations between the overall use of information in pigeons, and humans and ideal observers, in each task. We generated 10,000 ratio score images, per pigeon and per phase that satisfied our accuracy criteria, by performing on all the bubbled masks that were presented the computations described above, with the difference that correct and incorrect trials were randomly permutated 29. We thus ended up, for each group of pigeons, with 10,000 random normalized ratio scores of the sort that we would have obtained under to the nil hypothesis (i.e., no region is especially diagnostic in the faces). Finally, for each pigeon group, we correlated these random normalized ratio score images with the human and the ideal observer ones to produce two distributions of correlations. The levels of significance were directly extracted from these distributions. We performed another set of Monte-Carlo simulations to assess, for each task, the statistical significance of the difference between the correlation between the pigeons use of visual information and the humans, and the correlation between the pigeons use of visual information and the ideals.
Bubbled faces... 11 References 1. Bruce, V., & Young, A. In the Eye of the Beholder: The Science of Face Perception. (Oxford University Press, 1998). 2. Schyns P. G. Cognition 67, 147-179 (1998). 3. Gosselin, F. & Schyns, P. G. Vis. Res. 41, 2261-2271 (2001). 4. Schyns, P. G., Bonnar, L. & Gosselin, F. Psych. Sci., (2002). 5. Kanwisher, N., McDermott, J., & Chun, M. M. Journal of Neuroscience, 17, 4302 4311(1997). 6. Kanwisher, N., Tong, F., & Nakayama, K. Cognition, 68, B1-B11 (1998). 7. Perrett, D. I., Mistlin, A. J., & Chitty, A. J. Trends in Neuroscience. 10, 358-364 (1987). 8. Carmel, D. & Bentin, S. Cognition 83, 1-29 (2002). 9. Bentin, S., Allison, T., Puce, A., Perez, A. & McCarthy, G. J. Cogn. Neurosci. 8, 551-565 (1996). 10. Gauthier, I., Tarr, M. J., Anderson, A. W., Skudlarski, P., & Gore, J. C. Nature Neuroscience 2, 568-573(1999). 11. Tarr, M. J. & Gauthier, I. Nature Neuroscience, 3, 764-769 (2000). 12. Farah, M. J. Behavioural Brain Research, 76, 181-189 (1996).
Bubbled faces... 12 13. Nachson, I. J. Clin. and Exp. Neuropsych., 17, 256-275 (1995). 14. Taylor. M., McCarthy, G., Saliba, E. & Degiovanni, E. Clin. Neurophysiol. 110, 910 915 (1999). 15. Sagiv, N. & Bentin, S. J. Cogn. Neurosci. 13, 937-951 (2001). 16. Eimer, M. Neuroreport 9, 2945-2948 (1998). 17. Eimer, M. Neuroreport 11, 2319-2324 (2000). 18. Eimer, M. Brain Res. Cogn. Brain Res. 10, 145-158 (2000). 19. Rebai, M., Poiroux, S., Bernard, C., & Lalonde, R. Int. J. Neurosci. 106, 209-226 (2001). 20. Tanaka, J. W. & Curran, T. Psychol. Sci. 12, 43-47 (2001). 21. Watanabe, S. in Picture perception in animals. (ed. Fagot, J.) 71-90 (Psychology Press: East Sussex, 2000). 22. Cooper, E. E., & Wojan, T. J. Journal of Experimental Psychology: Learning, Memory, and Cognition. 26, 470-488 (2000). 23. Wasserman, E. A., Kirkpatrick-Steger, K., Van Hamme, L. J., & Biederman, I. Psychological Science. 4, 336-341 (1993). 24. Kreiman, G., Koch, C., & Fried, I. Nature. 408, 357-361 (2001). 25. Kendrick, K. M., dacosta, A. P., Leigh, A. E., Hinton, M. R., & Peirce, J. W. Nature. 414, 165-166 (2001).
Bubbled faces... 13 26. Wasserman, E. A., Hugart, J. A., & Kirkpatrick-Steger, K. Journal of Experimental Psychology: Animal Behavior Processes. 21, 248-252 (1995). 27. Oliva, A., & Schyns, P. G. Cognitive Psychology. 41, 176-210 (1999). 28. Siegel, S. Nonparametric statistics for the behavioral sciences. (McGraw-Hill, 1956). 29. Hammersley, J. M., & Handscomb, D. C. Monte Carlo Methods. (Chapman and Hall, 1964).
Bubbled faces... 14 Figure Captions Figure 1. Examples of the images used during experimentation. (a) Four of the 32 images used in training. The figure shows one of the eight males (left column) and one of the eight females (right column) exhibiting either a happy (top row) or a neutral (bottom row) expression. (b) Example of four bubbled faces used in testing. These images are constructed with the same training images shown in the top portion of the figure with the addition of the overlaid bubbled mid-gray mask (see Methods). An example of the testing displays with 20 bubbles (Phase 1: top left), 30 bubbles (Phase 2: top right), 40 bubbles (Phase 3: bottom left), and 50 bubbles (Phase 4: bottom right). Figure 2. Results of the bubbles analysis following testing with humans, pigeons, bird 31B, and ideal observers on the gender (top row) and expression (bottom row) discrimination tasks. The regions of the face that were used significantly above chance are in red and reveal the underlying face to help with interpretation. The human and ideal data have been reported previously 3, but they were reanalyzed here to be comparable with the pigeon data. The ideal observer captures all of the regions in the image (in this case, the face) that have the highest local variance between the discriminations (M versus F and H versus N) 27. Thus, the ideal observer provides the standard of all of the diagnostic information that is available in the stimulus set to solve each task. Each image displays a composite diagnostic mask indicating the regions of the face that were reliably diagnostic across all blocks of testing and all of the participants in each condition. Each diagnostic mask is placed over the same female face in this figure only for illustrative purposes.
Bubbled faces... 15 Table 1. A summary of the accuracy scores to the bubbled testing displays for each of the seven phases of testing for all of the birds (23Y, 31B, 52W, 72Y) in the Gender and Expressions conditions. Bird 31B was trained and tested on both discrimination problems. A star after a score indicates above chance performance as assessed by binomial tests for the respective testing periods.
Bubbled faces... 16
Bubbled faces... 17
Bubbled faces... 18