The Pennsylvania State University. The Graduate School. Department of Psychology AN EXAMINATION OF HOLISM IN THE VISUAL PROCESSING OF FACES

Transcription

1 The Pennsylvania State University The Graduate School Department of Psychology AN EXAMINATION OF HOLISM IN THE VISUAL PROCESSING OF FACES USING THE CROWDING EFFECT AND GENERAL RECOGNITION THEORY A Thesis in Psychology by Brianna Sullivan 2008 Brianna Sullivan Submitted in Partial Fulfillment of the Requirements for the Degree of Master of Science August 2008

2 The thesis of Brianna Sullivan was reviewed and approved* by the following: Michael J. Wenger Associate Professor of Cognitive Psychology Thesis Advisor Cathleen M. Moore Professor of Cognitive Psychology Reginald B. Adams, Jr. Assistant Professor of Social Psychology Melvin M. Mark Professor of Social Psychology Head of the Department of Psychology * Signatures are on file in the Graduate School. ii

3 ABSTRACT This study examined the extent to which holism in visual perception can be revealed by way of the presence or absence of crowding. Martelli, Majaj, and Pelli (2005) used crowding to propose an operational definition for holism. Specifically, they argued that holistic perception of an object is implicated if that object can be identified when the entire object is presented within an isolation field (defined as an area proportional to one-half eccentricity). Conversely, partsbased processing is implicated if identification is impaired when the entire object is within an isolation field, with an attenuation or elimination of that impairment when each part of the object is isolated by critical spacing. Martelli et al. found evidence of crowding increases in threshold contrast as a function of eccentricity for faces and words suggesting that foveally-presented objects are processed holistically, and peripherally-presented objects are processed by parts. This operational definition is considered from the perspective of general recognition theory (GRT, Ashby & Townsend, 1986). GRT provides theoretical characterizations of perceptual and decisional independence and separability, with violations of independence and separability allowing for multiple characterizations of holism. In this study, accuracy of identification responses was used to link Martelli et al. s operational definition to the definitions of holism provided by GRT. Two sets of face stimuli were presented under conditions modeled on those used by Martelli et al. The faces were used to replicate the patterns documented by Martelli et al.: specifically, evidence for the benefit of a facial context in foveal presentation, and impairment in peripheral presentation benefit and impairment that were eliminated in both presentations when critical spacing isolated the featural parts of the face stimuli. In addition, the GRT analyses revealed disparities between the current operational and theoretical definitions of holistic processing which suggest that the visual crowding effect cannot serve as a method for defining holism in face processing. These results contribute to a more systematic definition of holism, and an improved understanding of the visual processing of faces. iii

4 TABLE OF CONTENTS List of Tables v List of Figures vi INTRODUCTION Holistic Face Processing Operational Definitions of Holism Crowding Effect Martelli, Majaj and Pelli (2005) Experiments General Recognition Theory Current Experiments GENERAL METHODS Participants Stimuli Procedure RESULTS Experiment Evidence of Context Effects and Crowding Evidence of Holism Experiment Evidence of Context Effects and Crowding Evidence of Holism DISCUSSION Limitations Future Directions Conclusions REFERENCES APPENDIX

5 LIST OF TABLES Table 1: Experiment 1 threshold ratios and upper and lower bounds of the 95% confidence intervals Table 2: Summary Truth table 1 relates the results of the macro-analyses for one of the features (e.g., the nose) to inferences regarding Perceptual Separability (PS) and Decisional Separability (DS). Note. This table replicates details presented in Wenger and Ingvalsen, 2002, Table 4, p Table 3: Summary Truth table 2 relates the results of the micro-analyses to inferences regarding Perceptual Independence (PI) and Decisional Separability (DS). Note. This table replicates details presented in Wenger and Ingvalsen, 2002, Table 5, p Table 4: Experiment 1 multidimensional signal detection analyses summary of violations Table 5: Experiment 1 signal detection macro-analysis marginal estimates for d and c Table 6: Experiment 1 signal detection macro-analysis for marginal response invariance (MRI) Table 7: Experiment 1 signal detection micro-analyses for sampling independence for the foveal presentation of stimuli Table 8: Experiment 1 signal detection micro-analyses for sampling independence for the peripheral presentation of stimuli Table 9: Experiment 2 threshold ratios and upper and lower bounds of the 95% confidence intervals Table 10: Experiment 2 multidimensional signal detection analyses summary of violations Table 11: Experiment 2 signal detection macro-analysis marginal estimates for d and c Table 12: Experiment 2 signal detection macro-analysis for marginal response invariance (MRI) Table 13: Experiment 2 signal detection micro-analyses for sampling independence for the foveal presentation of stimuli Table 14: Experiment 2 signal detection micro-analyses for sampling independence for the peripheral presentation of stimuli v

6 LIST OF FIGURES Figure 1: Schematic of visual field illustrating changes in critical spacing as a function of eccentricity, at 1.5 and 8 degrees eccentricity Figure 2: Experiment 1 Nose and Face Stimuli a. Nose alone (narrow, r) b. Nose alone (wide, w) c. Eyes(r)-Nose(r)-Mouth(w) d. Eyes(r)-Nose(r)-Mouth(r) e. Eyes(r)-Nose(w)-Mouth(w) f. Eyes(w)-Nose(r)-Mouth(w) g. Eyes(w)-Nose(w)-Mouth(w) h. Eyes(w)-Nose(w)-Mouth(r) i. Eyes(r)-Nose(w)-Mouth(r) j. Eyes(w)-Nose(w)-Mouth(r) Figure 3: Experiment 2 Nose and Face Stimuli a. Nose alone (narrow, r) b. Nose alone (wide, w) c. Eyes(r)-Nose(r)-Mouth(w) d. Eyes(r)-Nose(r)-Mouth(r) e. Eyes(r)-Nose(w)-Mouth(w) f. Eyes(w)-Nose(r)-Mouth(w) g. Eyes(w)-Nose(w)-Mouth(w) h. Eyes(w)-Nose(w)-Mouth(r) i. Eyes(r)-Nose(w)-Mouth(r) j. Eyes(w)-Nose(w)-Mouth(r) Figure 4: Contour example explaining the GRT representations a. Theoretical representation of equal-likelihood contours b. Statistical representation of geometrical contours Figure 5: Examples of GRT equal-likelihood contours illustrating violations a. Violation of Perceptual Independence (PI) b. Violation of Perceptual Separability (PS) c. Violation of Decisional Separability (DS) d. No violations Figure 6: Experiment 1. Contrast threshold ratios Figure 7: Experiment 1. Contours indicating distributions of evidence for Observer a. Eyes-Nose foveal b. Eyes-Nose peripheral c. Eyes-Mouth foveal d. Eyes-Mouth peripheral e. Nose-Mouth foveal f. Nose-Mouth peripheral vi

7 Figure 8: Experiment 1. Contours indicating distributions of evidence for Observer a. Eyes-Nose foveal b. Eyes-Nose peripheral c. Eyes-Mouth foveal d. Eyes-Mouth peripheral e. Nose-Mouth foveal f. Nose-Mouth peripheral Figure 9: Experiment 1. Contours indicating distributions of evidence for Observer a. Eyes-Nose foveal b. Eyes-Nose peripheral c. Eyes-Mouth foveal d. Eyes-Mouth peripheral e. Nose-Mouth foveal f. Nose-Mouth peripheral Figure 10: Experiment 1. Contours indicating distributions of evidence for Observer a. Eyes-Nose foveal b. Eyes-Nose peripheral c. Eyes-Mouth foveal d. Eyes-Mouth peripheral e. Nose-Mouth foveal f. Nose-Mouth peripheral Figure 11: Experiment 1. Contours indicating distributions of evidence for Observer a. Eyes-Nose foveal b. Eyes-Nose peripheral c. Eyes-Mouth foveal d. Eyes-Mouth peripheral e. Nose-Mouth foveal f. Nose-Mouth peripheral Figure 12: Experiment 2. Contrast threshold ratios Figure 13: Experiment 2. Contours indicating distributions of evidence for Observer a. Eyes-Nose foveal b. Eyes-Nose peripheral c. Eyes-Mouth foveal d. Eyes-Mouth peripheral e. Nose-Mouth foveal f. Nose-Mouth peripheral Figure 14: Experiment 2. Contours indicating distributions of evidence for Observer a. Eyes-Nose foveal b. Eyes-Nose peripheral c. Eyes-Mouth foveal d. Eyes-Mouth peripheral e. Nose-Mouth foveal f. Nose-Mouth peripheral vii

8 Figure 15: Experiment 2. Contours indicating distributions of evidence for Observer a. Eyes-Nose foveal b. Eyes-Nose peripheral c. Eyes-Mouth foveal d. Eyes-Mouth peripheral e. Nose-Mouth foveal f. Nose-Mouth peripheral Figure 16: Experiment 2. Contours indicating distributions of evidence for Observer a. Eyes-Nose foveal b. Eyes-Nose peripheral c. Eyes-Mouth foveal d. Eyes-Mouth peripheral e. Nose-Mouth foveal f. Nose-Mouth peripheral viii

9 Every absurdity has a champion to defend it. Oliver Goldsmith, ix

10 INTRODUCTION The study of psychology, like any other field of human inquiry, is studded with debates whether information processing is parallel or serial, how working memory functions, whether or not attention can be unconscious. In the subfield of vision research, one of the most significant controversies concerns the concept of holism. Also known as configurality or gestalt, holism refers to the idea that the whole is different from the sum of the parts (Farah, Wilson, Drain, & Tanaka, 1998; O Toole, Wenger, & Townsend, 2001). In terms of visual processing, the holistic encoding hypothesis posits that human beings encode and recognize certain visual objects as unitary psychological entities, rather than representing objects by their component parts and then mentally assembling the parts into a single object. The aim of this study was intended to contribute to the investigation of the conceptual and empirical integrity of extant operational definitions for holism. Previous studies have employed a variety of methods to assess holism in face processing: the inversion effect, superiority and inferiority effects, Thatcher effect, composite face effect, and more, all of which are discussed in detail below. Within this context, the research most relevant to the present study was performed by Martelli, Majaj, and Pelli (2005), who used a visual effect known as crowding. Crowding represents a failure of object perception, when the parts of an object are spaced sufficiently close to each other that they appear to overlap and crowd each other in perception, which impedes visual processing. Since the problem resides in the interference caused by parts, Martelli and colleagues used crowding to define holism, as a whole, unitary object by definition has no components that can interfere with each other. They therefore hypothesized that when crowding did not impair object identification, the object was being processed holistically, whereas when crowding impaired identification, the object was instead being processed by its parts, which interfered with each other. However, Martelli and associates (2005) definition for holism, like those associated with the other visual effects listed above, is problematic because it is operational. While operational definitions have the great advantage of being able to provide a plenitude of converging evidence to distinguish and add layers of dimensionality to a concept, they also have the disadvantage of lacking a strong theoretical foundation. Without being able to define the concept in terms of clear 1

11 theoretical structures, it instead must be defined in terms of behavioral results, which are subject to circular reasoning, redundancy, and terminological imprecision. To avoid these problems, it is beneficial to append to these operational definitions a theoretical definition of holism, to assess the validity of such definitions through the theoretical lens of general recognition theory. In this study, general recognition theory (GRT, Ashby & Townsend, 1986) was applied to the collection of evidence of visual crowding, in an attempt to reconcile the operational crowding definition for holism with the theoretical GRT definition for holism. After discussing previous work on operational definitions of holism, the crowding effect, Martelli, Majaj, and Pelli s (2005) experiments, this document details General Recognition Theory and its use in defining holism, before describing the current experiments. Both experiments replicated Martelli et al. s first two experiments on crowding but in conjunction with GRT analysis, and the results indicated a fundamental disconnect between the GRT theoretical and crowding operational definitions for holism. Details are provided in the results, after which the discussion provides summary and addresses further use of theoretically-based definitions of holism in examining face processing, to gain a clearer and more detailed assessment of the holistic hypothesis for faces. Holistic Face Processing As mentioned, several visual effects have been used to provide support for holistic encoding generally, including the object superiority effect, and inversion or orientation effects. The object superiority effect occurs when people are better able to identify an object when it is presented in context, such as simple lines in an organized geometrical structure (Weisstein & Harris, 1974), or a door in a house, or a nose in a face (Tanaka & Farah, 1993). Inversion or orientation effects refer to the increased difficulty of recognizing an object when it is presented upside-down relative to when it is presented upright (e.g., Palmer, Rosch, & Chase, 1981). This effect provides support for holism because people s impaired performance is thought to reflect the disruption of information they perceive about the configuration of the face information integrated over the entire face region compared to information about individual features, like the mouth or nose, for which people show no impairment. Holistic 2

12 encoding has been applied to many aspects of object perception and recognition, in particular, the study of face processing. According to the holistic encoding hypothesis, faces are special in that they are perceived as wholes to a much greater extent than is true for other visual objects, which are hypothesized to be perceived by their component parts (Farah, Tanaka & Drain, 1995; Farah, et al., 1998). For example, it is argued that a building is recognized by its windows, roof, and doors; while a face is recognized as a completely integrated whole object, and not by its eyes, nose, and mouth. Although earlier researchers suggested that faces are processed as gestalts that the overall holistic structure of the face is more crucial for perception and recognition than its component featural parts (e.g., Bradshaw & Wallace, 1971) the modal contemporary version of the holistic hypothesis of face processing was developed by Farah and colleagues (1995; 1998). It is referred to as the holistic encoding hypothesis, that faces are primarily represented as undifferentiated wholes, and that the perceptual process for faces involves little or no explicit representation of compositional parts. In the holistic encoding hypothesis, whole face perception transcends local feature perception, and the visual system goes directly from the elementary features to whole object representation without recognizing intermediate parts (Farah et al., 1998). The two empirical effects that provide the strongest evidence for this hypothesis are face-specific instantiations of the object superiority and orientation effects mentioned above: the face superiority effect and the face inversion effect. According to the face superiority effect, people are better able to discriminate a facial feature if it is presented in the context of a face than if it is presented alone or in a scrambled face (Tanaka & Farah, 1993; Tanaka & Sengco, 1997). In Tanaka and Farah s (1993) experiments, participants were asked to identify a facial feature the nose presented alone, and in normal and scrambled versions of a face. The results suggested that identification was improved for a feature in the context of a normal face, and that normal faces were recognized more holistically than scrambled faces. People were better at identifying normal faces because all the pieces fit together in a unitary whole there was a distinct benefit for the presence of a 3

13 rational face context, compared to no context or a nonsense one. Processing improved for a sensible whole, leading Tanaka and Farah (1993) to infer a greater degree of holism for normal faces. Similarly, in the inversion effect, people have a much harder time recognizing a face when it is presented upside-down than when it is presented upright (Farah et al., 1995). In their study, Farah, Tanaka, and Drain (1995) asked participants to study upright whole faces and upright faces divided into featural parts, then to identify whole and parts-based faces presented upright and inverted. Farah and associates believed that the significant improvement in accuracy scores from inverted to upright whole faces, compared to the non-significant difference between inverted and upright parts-based faces, suggested that the face inversion effect is associated with holistic perception. Once more, the improvement in identification accuracy was taken to mean that there was a significant distinction in processing, with inversion revealing that faces seen as wholes are being processed differently, and more efficiently, than faces seen as a collection parts. Farah et al. (1997) stated that people are better at seeing faces as wholes than any other way, and this is evidence of holistic encoding. Additionally, in a 1998 study, Farah and colleagues presented participants with upright and inverted features and faces, again asking participants to identify a feature by itself, or in the context of a face. The results revealed a distinct impairment in performance associated with inverted whole faces compared to upright ones, but no difference in performance for inverted versus upright features alone. Since inversion affected whole faces so much more profoundly than features, this again suggested to the experimenters that faces possess a unique quality; that they are processed differently than individual features, and that the difference is that face identification and recognition is holistic. Operational Definitions of Holism Through their studies, Farah and colleagues have provided three operational definitions for holism, based on comparisons of conditions. An operational definition of a concept is one in which the concept is defined solely in terms of measurable outcomes, fitting the definition and theoretical implications to the behavioral results on an experimental task. In contrast, a researcher using a theoretical definition begins with the definition, then develops a task to test it, working from the definition to the 4

14 behavioral implications. Exploring holistic face processing through a variety of techniques provides valuable converging evidence, and the use of multiple methods with different specific implementations of tasks and stimuli also has the advantage of ensuring that the results cannot be credited to the nature of any one experimental method (McKone, 2004). Rather than being the consequence of an item effect, attributable to the particular type of task or stimulus, the evidence across studies can show that the holism defines face processing in all kinds of experimental conditions. Unfortunately, the use of operational definitions means that each description of holism is based on a different experimental context and so uses different terms, which in turn means that the resulting definitions cannot be generalized across studies, because there is no cohesive logic for assembly. As a result, the definitions tend to involve a circular reasoning, a profusion of terminology, some of which may be overlapping and some exclusive, and a lack of definitional precision, which can be demonstrated in the following studies. Although there are similarities in the three studies by Farah and associates, the definitions of holism that result holism based on accuracy scores for scrambled versus normal faces and for inverted versus upright faces and features are distinct from each other. This indicates a significant problem associated with operational definitions: they are restricted to being described in terms of the tasks researchers use. This difficulty becomes even more prominent because Farah and colleagues (1993; 1995; 1998) are not the only ones to have examined holism, and other researchers have used different experimental tasks which have resulted in the creation of several more diverse operational definitions of holism. Reviewed below are a set of the most prominent operational definitions of holism in the context of face perception and memory, provided to both demonstrate the range of approaches and indicate the motivation for a theoretically-grounded alternative approach. One such definition is based on the experimental paradigms used to illustrate the Thatcher effect. The Thatcher effect occurs when the photograph of a human face is manipulated so that the eyes and mouth are inverted in an upright face. Then, when the face itself is inverted, participants who view the face in both conditions rate the inverted face as appearing less grotesque than the upright one, even though the features remain inverted in both cases (e.g., Thompson, 1980). For example, Bartlett and 5

15 Searcy (1993) presented participants with normal and Thatcherized faces in upright and inverted conditions, and asked them to rate the grotesqueness of the image on a scale from one to seven. They found that grotesqueness ratings were slightly increased by inversion for normal faces, but dramatically decreased by inversion for Thatcherized faces. Given their hypothesis that the grotesqueness ratings are based on the processing of holistic information, and that the Thatcher illusion reflects a disruption in holistic processing when faces are inverted, Bartlett and Searcy (1993) devised a new operational definition for holism. They concluded that holism is indicated when participants give a higher grotesqueness rating for an upright than an inverted Thatcherized face. Another visual effect that has been used to operationally define holism is the composite face effect. In the composite face effect, face stimuli are created by combining complementary halves of two different faces (either top-bottom or right-left halves), and the combination causes interference in the identification of facial features within one half, or of one half itself (e.g., Young, Hellawell, & Hay, 1987; Singer & Sheinberg, 2006). Young and colleagues (1987) discovered that reaction times for identification of a facial feature were longer for the composite faces than in normal faces composed of halves from the same individual. To the researchers, this implied that all parts of a face interact holistically in identification, and when the parts indicate different identities, identification is disrupted. Hence, their operational definition for holism based on the composite face effect: that holism is indicated when reaction times are shorter for faces composed of halves from the same face, and longer for faces composed of halves from two different faces, since, in the latter, holistic processing is disrupted. While the different researchers above each focused on using a single effect to define holism, McKone and colleagues have employed a variety of methods and tasks to examine holistic processing. They have attempted to compile an argument based on converging operations, pulling together evidence from different sources to arrive at a single conclusion, and, as with Farah and associates, the result is a multitude of operational definitions. McKone, Martini and Nakayama (2001) investigated the categorical perception of face identity. Categorical perception refers to the identification of stimuli into sharply discrete categories as a function of some physical manipulation, rather than a smooth (and somewhat 6

16 gradual) change in identification responses across the range of manipulation. To assess learned categorical perception for face identity, Beale and Keil (1995) used a pair of photographs of famous faces (presidents Clinton and Kennedy), and morphed them together to produce intermediate images between a pair of endpoint faces. Their subjects were asked to perform binary classification of the faces to determine the predicted category boundary. Then, categorical perception was demonstrated when participants showed better discrimination between pairs of stimuli that crossed the individuallydetermined category boundary (35% Clinton morph vs. 55% Clinton morph) than between equidistant pairs on the same side of the boundary (20% morph and 40% morph). In their experiments, McKone and colleagues (2001) presented participants with upright and inverted faces and parts at high levels of noise, using discrimination tasks to assess categorical perception. They found that categorical perception occurred for upright faces, but not inverted faces or parts, which they believed represented a qualitative difference between the processing of faces in upright, inverted and part-based conditions, reflecting the effects of holism. This operational definition states that holism is apparent where discrimination accuracy is higher for faces presented at high noise levels on opposite sides of a category boundary than for faces that are on the same sides of a boundary, inverted, or separated into parts. However, aside from the aforementioned problems associated with operational definitions, there are other issues with using categorical perception, discussed by Massaro (1987). The discrimination tasks, such as the ones used by McKone in this study, encourage participants to categorize rather than discriminate. As a result, participants encode stimuli categorically and base their discrimination decision on the category labels, so better discrimination between items from different categories than items in the same category cannot provide conclusive evidence of categorical perception (Ellison & Massaro, 1997). Thus, categorical perception appears to be an unreliable tool for defining holism another risk of operational definitions, that the definition is only as valid as the task providing the behavioral outcomes that define it. McKone (2004) has also employed Mooney faces extremely high-contrast versions of faces (Mooney, 1957) to assess holistic face processing. In her experiments, McKone (2004) used Mooney faces in order to present a challenge to face processing by using stimuli in which it is hard to perceive 7

17 faces, and particularly difficult to discriminate featural parts. Participants were asked to identify the Mooney faces in upright and inverted conditions, and their identification accuracy scores were significantly higher for the upright faces than for the inverted faces ones, which, to McKone, indicated that holistic processing could occur in the absence of parts-based contributions to identification. From these results, she operationally defined holism as being indicated when identification accuracy was higher for upright versus inverted Mooney faces. In another study, Robbins and McKone (2003) used images of two pairs of twins to assess participants ability to learn to determine identity in another challenging condition: discriminating between identical twins. Images of identical twins were used in order to make learning based on features as difficult as possible, since they should be essentially identical, and therefore to encourage holism by providing maximum need for the perceptual system to develop holistic representations of the faces. Participants underwent training sessions to be able to identify the twins, and then were tested with the images presented upright and inverted. The results showed that identification accuracy was significantly higher for upright versus inverted images for both pairs of twins. This led Robbins and McKone (2003) to conclude that even in conditions designed to facilitate holism, practice with inverted faces does not induce holistic processing, while in the upright orientation, twin discrimination is supported by holistic processing. Their operational definition follows, that if identification accuracy is higher for upright twin faces than inverted twin faces, holism is indicated. In one further study comparing upright and inverted images, Martini, McKone, and Nakayama (2006) made use of transparency displays to examine holism in face processing. In transparency displays, an upright and inverted image of the same face are superimposed over each other in transparency. Then, by manipulating the physical contrast of the component images, the experimenters can measure the degree of perceptual dominance of an image as a function of its orientation, which is aimed at quantifying the exact dependence on orientation in human face perception. Martini and colleagues (2006) showed participants the transparency displays, and when contrasts were equal, participants found that the upright face dominated the inverted one by appearing to have a higher contrast. In fact, the inverted and upright 8

18 components were not judged to be equal until the contrast ratio between upright and inverted images was approximately 33:67. Martini et al. (2006) suggested that identification of upright faces employs the highly sensitive functional mechanism that processes visual information holistically, rather than using independent processing subroutines for individual parts, which they proposed is activated as a default mechanism to identify inverted faces. Based on that conception, their results seem to indicate the functional dominance of holism in faces the strong propensity of people to see and process an upright face first, because it is done holistically, before an inverted face, which cannot be processed holistically and must be done by parts. This ordering exists even when the two images are equalized for contrast, and persists until the contrast ratio is unbalanced enough to force people to do the reverse. From this, Martini and associates derived another operational definition for holism that holistic processing is implied when the contrast ratio for inverted and upright faces in a transparency display is equal, and participants still identify the upright image as dominant. As this review demonstrates, there is a wide range of operational definitions of holism, based on the fact that many operational methods have been used as a means for establishing holistic processing of faces. The work proposed here examines another recently-proposed operational definition, one that uses the visual effect known as crowding to define holism. Crowding Effect Visual crowding occurs when people find it harder to identify or recognize a target object when it is presented with surrounding contours (Bex, Dakin, & Simmers, 2003). Crowding can occur between objects, and within a single object (between its component parts), an effect that is referred to as internal crowding. An object crowds itself when all its parts fall within a critical spacing, the distance that parts need to be separated from each other in order to allow for identification or recognition (Martelli, Majaj, & Pelli, 2005). Critical spacing defines an isolation field, the region over which the features of an object are integrated to identify or recognize the object (see Figure 1), and the sizes of critical spacing and the diameter of the isolation field are assumed to be the same, roughly half the eccentricity (Bouma, 1970). Pelli, Palomares, and Majaj (2004) argued that crowding is distinct from ordinary visual masking. In 9

19 ordinary masking, the target signal appears to disappear, suppressed by the mask, and this effect is similar for the fovea and the periphery. In contrast, when crowding occurs, the target signal is visible but ambiguous, incorporating features from the mask the surrounding contours and its effects are only evident in the periphery. Although Louie, Bressler, Whitney (2007) propose that crowding may occur at multiple stages in the visual system, based on their finding of selective crowding for high-level representations of face, most research attributes the origin of crowding to problems in feature integration, which is part of what differentiates crowding from ordinary masking. Ordinary masking impairs people s abilities to discriminate a target signal by stimulating the same receptive fields of the visual feature detectors that pick up the signal, and so interferes with both feature detection and feature integration. In crowding, the mask and target signal stimulate different feature detectors that reach the same feature integrator, causing excessive integration over an inappropriately large area that includes the flanking masks as well as the signal (Pelli et al., 2004). Nandy and Tjan (2007) describe this as occurring because feature integration is a competitive process, in which crowding reduces the amount of valid features (the target object) and increases the amount of invalid features (the surrounding contours) that the visual system uses. Since interference takes place at the level of integration in crowding, the crowding effect makes it possible for researchers to separate the two visual processes of detection and integration. Additionally, crowding has the potential to provide a new method for examining holism, and its alternative, the parts-based hypothesis of face processing. The holistic definition posited by Martelli, Majaj and Pelli (2005) states that identification or recognition of an object occurs when the whole object is within an isolation field, as if the whole object consists of a single part. Even when the whole object is within a critical spacing, there will be no crowding, because a single part cannot crowd itself. Therefore, Martelli and associates argue, the experimental stimuli should show no effect of crowding there should be no change in the threshold level of contrast for the stimuli when each stimulus is presented within an isolation field, which would provide support for the holistic processing hypothesis. However, the alternative hypothesis states that 10

20 parts-based processing is implicated if identification or recognition is impaired when the entire object is within an isolation field, and the impairment is attenuated or eliminated when each part of the object is isolated from the rest by a critical spacing. Otherwise, the parts overlap and blend together crowd each other and impair visual processing. If the experimental stimuli show evidence of crowding (an increase in threshold contrast) when all of the elements are presented together within a single isolation field, it suggests that those stimuli were not perceived as wholes, and would in turn imply support for the partsbased hypothesis of processing. The parts-based hypothesis posits that the visual system processes faces and other objects in the same way, involving an intermediate stage of recognition that is specialized for compositional parts. Called the hierarchical parts-based hypothesis by Martelli, Majaj, and Pelli (2005), it states that the parts of a face are vital to face representation. They fall in a hierarchy of perception in between the representation of elementary features (the lines and shades that compose the shapes of the eyes, nose, mouth, and outline of the face) and the representation of the whole object (the face). Evidence supporting the parts-based hypothesis appears in the object superiority effect for words, in visual search processing of words and faces, and in the crowding effect. Pelli, Farell, and Moore (2003) found that words show the same object-superiority effect that faces do: people were better able to identify a letter when it was presented in the context of a word, than if it was presented alone or in a non-word context. So, faces appear to be processed more like other visual objects, including words, than previous research might suggest a finding that underscored the importance of further exploration of the holistic processing of faces and words, such as that conducted by Martelli, Majaj, and Pelli (2005). Their investigation consisted of a set of experiments that used the crowding effect to try and gain a clearer and more precise understanding of face processing. Martelli, Majaj and Pelli (2005) Experiments Martelli and colleagues (2005) performed a series of experiments to examine the holistic and parts-based processing hypotheses, to determine if the crowding effect could be used to reveal holism in two types of stimuli, faces and words. Since previous research had indicated that words are not processed 11

21 holistically, but instead recognized by parts (Pelli et al., 2003), the researchers planned to present both faces and words under conditions of crowding to see if faces, like words, would show crowding and be recognized by parts, or if they would instead resist crowding and be recognized as whole objects. In this way, the presence or absence of crowding in stimuli could be used to provide a definition for holism. Martelli et al. (2005) tested seven participants using photos and caricatures of faces, focusing on mouths, and using an alphabet of five letters (cgprx) plus vowels. In the first experiment, which investigated the effects of crowding in words and faces, participants were asked to identify mouths or letters presented alone or within face or word contexts, and presented foveally or peripherally. In the second experiment, participants again viewed mouths and letters in the fovea or periphery, but this time, the experimenters manipulated critical spacing and isolation fields so that participants would have to identify the letter or mouth target when the whole object was within critical spacing, or when parts were isolated by critical spacing, to assess internal and external crowding. In their procedure, the fixation point appeared in the center of the computer monitor screen for 200 ms, and then 400 ms after it appeared, the target stimulus would appear, also for 200 ms. For peripheral viewing, the target was always presented in the right visual field, at 2, 4, 6, 8, or 12 from fixation (0 eccentricity). After the target stimulus disappeared, the response screen appeared showing all the possible stimuli candidates at 80% signal contrast contrast being defined as the ratio of luminance increment of the signal to the background luminance (Martelli et al., 2005) and participants were instructed to click on the stimulus matching the original target. The experimenters adjusted the contrast of the target stimuli presented using the QUEST staircase procedure (Watson & Pelli, 1983), in which contrast is adjusted down for a criterion number of correct responses, and up for a criterion number of incorrect responses. They estimated each participant s threshold contrast in a 40-trial run, with an asymptotic accuracy criterion of 82% (Watson & Pelli, 1983). To assess the effects of presentation with or without context, Martelli et al. (2005) used threshold ratios the threshold contrast for objects presented alone divided by threshold contrast for objects presented in context, as a function of eccentricity. A ratio greater than one indicated an advantage of 12

22 context, while a ratio less than one indicated a disadvantage, or impairment associated with crowding. Their results for words, face photos, and face caricatures showed that for all three, ratios were above one when stimuli were presented foveally, indicating an object superiority effect in the fovea for both words and faces. However, with peripheral presentation, the ratios for all three stimulus types decreased. At 2, ratios were approximately equal to one, but at 4, 6, and 8, ratios became progressively smaller, indicating a growing disadvantage, or object inferiority effect, for both faces and words presented peripherally at 8 eccentricity, the inferiority effect for words increased by a factor of five, face photos by four, and face caricatures by seven indicating that object identification was impaired by crowding. When the experimenters examined their results for threshold contrast as a function of critical spacing and eccentricity, they found that, in foveal presentation, the threshold contrast was unaffected by critical spacing. So, for both words and faces, when an object was presented at zero degrees eccentricity, altering critical spacing had no effect on participants ability to accurately identify the object. Conversely, when a word or face object was presented in the periphery, at 4, 6, 8, or 12 eccentricity, threshold contrast had to increase as eccentricity increased, or as critical spacing decreased. When peripherally-presented objects were presented in narrow critical spacing, participants required higher levels of contrast to accurately identify them. These findings allowed Martelli and colleagues to draw a variety of conclusions. From the first experiment, they determined that both words and faces showed an object superiority effect in foveal presentation, and an inferiority effect in peripheral presentation. If the superiority effect had occurred exclusively for faces at both presentation locations, it would have provided support for the holistic hypothesis by indicating that faces, unlike words, are more recognizable as whole objects, no matter where they are presented. Instead, their evidence for crowding suggested that objects are processed by parts when presented in the periphery, thus providing support for the parts-based hypothesis. Additional support for the latter was provided by the threshold contrasts from the second experiment, in which they found that crowding could be internal as well as external, which result implied that peripheral identification was only possible when object parts were isolated from each other by critical spacing. In 13

23 the periphery, identification of an isolated part is essential for identification of a whole object, such that identification within critical spacing was impaired for whole objects, but unimpaired for object parts. Therefore, Martelli et al. (2005) concluded that even in objects which are not themselves perceived as wholes, the component parts must be represented holistically in order for identification or recognition to occur. In other words, object parts (features and letters) are recognized holistically, while whole objects (faces and words) are recognized by parts consistent with their hierarchical parts-based hypothesis, in which the most crucial components of visual processing are object parts. Although their findings and conclusions offer fascinating implications for visual processing hypotheses of faces, Martelli, Majaj, and Pelli s (2005) arguments have weaknesses as well as strengths, which revolve primarily around the fact that their definitions of holistic and parts-based processing are operational. These difficulties apply to all the operational definitions listed previously in the section above. First, operational definitions are subject to circular reasoning, for example: Martelli and colleagues state that finding evidence of internal crowding indicates that an object has component parts, but also that if an object has component parts, it will show evidence of internal crowding. Secondly, these operational definitions are all based on shifts in performance of participants, which is potentially problematic, because people s performance can shift in ways that may not have any implications for holism. For example, when an image is presented in inverted orientation or the periphery, a participant could simply be more conservative about making an identification judgment, compared to an upright or foveally presented image a shift in response bias that may not be consistent with holism. Finally, all the definitions are restricted to the methods the experimenters used. They are distinct from those of other researchers in the field, because all are using different tasks, and then developing a multitude of definitions based on those tasks. The resulting terminology is diverse, imprecise, and possibly redundant, which makes it difficult, if not impossible, to generalize findings across studies in order to draw cohesive conclusions about holism. 14

24 General Recognition Theory A solution to these difficulties lies in the use of a theoretically motivated approach to defining holism identifying theoretical concepts, developing tasks to test them, and then drawing conclusions about behavioral patterns. The work proposed here is focused on theoretical definitions; it replicates the Martelli et al. (2005) study to the extent of presenting faces in central and peripheral locations to find evidence of crowding, but extended within the theoretical context of general recognition theory (GRT). GRT is a multidimensional generalization of signal detection theory developed by Ashby and Townsend (1986). It provides a meta-theoretical language in which to define the encoding and processing of information, by offering a precise set of terminology phrased in terms of theoretical dependence or independence, in which object parts are either bound together or separated in psychological representation (Ashby & Townsend, 1986). Unfortunately, there is no way to use Martelli et al. s existing data to compare the influence of facial context associated with foveal versus peripheral presentation using GRT, because in order to perform the GRT analyses it is necessary to have an account of participants identification responses for all of the components of the stimuli that are of interest, particularly their hit and false alarm rates. Since there was no target-absent condition, and participants only had to make responses on a single dimension of the stimulus (the middle letter or mouth), there are no false alarm rates, nor multiple dimensions of the stimulus (for example, two letters, or the eyes and mouth), which are required to draw the GRT inferences. Without this information, there is no way to conclusively determine whether the two types of presentation lead to qualitatively different forms of processing, as defined theoretically by GRT. The experiments presented here were designed to make that distinction, replicating the crowding effect in order to systematically define crowding and holism, and to better understand the behavioral implications of the crowding effect for holistic and parts-based hypotheses of face processing. In Martelli and associates experiments, the existence of crowding, as assessed by threshold measures indicating the superiority or inferiority effects for context, was used to operationally define holism. However, GRT provides theoretical characterizations of perceptual and decisional independence and separability, and violations of that independence and separability allow for multiple characterizations of 15

25 holism. By tying the two together, the threshold measure evidence used to infer holistic versus partsbased processing can be compared to the GRT evidence of holistic or non-holistic processing, to determine whether crowding can be used as a valid tool for defining holism. The GRT characterizations rely on three theoretical constructs: perceptual independence (PI), perceptual separability (PS), and decisional separability (DS; Ashby & Townsend, 1986; see also Wenger & Ingvalson, 2002, 2003). The three constructs can be illustrated using an example from one of the sets of stimuli used in this experiment, which consist of two sets of eight faces (see Figures 2 and 3). Each face has three interior feature dimensions that participants were asked to judge (eyes, nose, and mouth), but they were only required to make judgments about two of the dimensions at a time (eyes and nose, nose and mouth, or eyes and mouth), while the third feature was held constant. Each feature has two variations: narrow and wide, so, for the example, a participant would be making judgements about the Eyes-Nose feature pairing, based on four possible images (Eyes-narrow Nose-narrow, Eyes-narrow Nosewide, Eyes-wide Nose-narrow, Eyes-wide Nose-wide). Each time a stimulus is presented, encoding is assumed to provide a level of perceptual information regarding each of the two dimensions. Assuming the level of perceptual information varies across presentations of the images (Ashby & Lee, 1993), the results of numerous presentations would produce four bivariate distributions of perceptual information, one for each of the possible paired combinations of the dimensions. In this study, the distribution of perceptual information was assumed to be reflected in the distribution of identification responses. As Martelli et al. (2005) used identification accuracies in their analyses, accuracies become the common measure allowing for the link between Martelli et al. s methodology and operational definitions and the definitions of holism provided by GRT analysis. This is particularly important because Martelli et al. focus on the shifts in threshold as a function of accuracy; however, anything that shifts accuracy also shifts threshold estimates, and consideration of the three GRT constructs shows that accuracies can be shifted in at least four different ways, as shown in Figure 5. This figure provides examples of the theoretical representations of GRT contours, depictions of the distributions of stimulus-response 16

26 identification data and the inferences about perceptual and decisional preservations or violations that can be drawn from them. A stimulus is considered perceptually independent if the perceptual information about one dimension of a stimulus is statistically independent of the perceptual information about a second dimension of a stimulus. If it is assumed that the distributions of perceptual information are bivariate Gaussians, then this implies that the dimensions are uncorrelated. It is important to note in considering PI, the comparison is within a single stimulus. Formally, PI is defined in terms of the relationship between the joint probability distribution for the perceptual evidence for one stimulus and the associated marginal probability distributions. PI is said to hold if the perceptual effects of two dimensions of a stimulus are statistically independent, that is if f rw (e,n) = g r (e)g w (n). The n and e indicate the variables representing the amount of perceptual evidence for the nose and eyes, respectively, of the stimuli faces. The f rw is the joint probability distribution function for the specific combination of the eyes at level r (narrow) and the mouth at level w (wide); g r (e) and g w (n) are the associated marginal pdfs. To clarify this equation with an example, PI would be preserved if the level of perceptual information about the width of the eyes was not correlated with the width of the nose. On the other hand, PI would be violated if a variation in the level of perceptual evidence in support of a narrow nose was correlated with a variation in support of wide eyes so that the presentation of a narrow nose increases the amount of evidence in support of wide eyes versus narrow ones. In signal detection theory, a stimulus can be represented as a single probability distribution, such as the standard bell curve. However, in GRT, when the stimuli are multidimensional (as in this case, with the pairings of features eyes-nose, eyes-mouth, and nose-mouth, each with wide and narrow levels), the combination of components is represented by a bivariate normal multidimensional probability distribution (Ashby & Townsend, 1986; Richler et al., 2008). Then, a cross-section of these multidimensional distributions is taken to simplify the visual representation, and this cross-section provides the contours of 17

27 equal-likelihood, in which circles represent the areas of the distributions for each response to a stimulus, as can be seen in Figure 4a. In this theoretical representation, evidence suggesting the possible violation or preservation of PI can be drawn from the shapes of the response distributions. If these shapes are truly circular, there is no evidence of a correlation in the distribution of responses, and PI is preserved. However, if the response distributions are distorted, such that, for example, one or more of the shapes form an ellipse, this indicates a correlation in the data for a stimulus, and therefore, a violation of PI (Figure 5a). If the stimuli are processed holistically, the evidence is expected to show a violation of PI, since, if the stimulus is being perceived as a whole, the perceptual information about the two dimensions of the stimulus should be highly correlated, as the object parts are bound together in perceptual representation. This would correspond to an absence of crowding, as indicated by threshold measures showing an object superiority effect for context. Foveal holism associated with the absence of crowding would appear in the GRT contours as the areas of the distribution of accuracies being increased by a positive correlation between a narrow nose and wide eyes, for example that would shape the distribution into an ellipse. In crowding, the distribution of accuracies is altered where variations in accuracy are linked to variations in threshold levels of contrast. If a stimulus is affected by crowding, a higher level of contrast is necessary for correct identification of the stimulus. So, evidence for crowding will consist of an increase in level of contrast, associated with a decrease in accuracy, while increased accuracy and no contrast change would indicate the absence of crowding. On the other hand, a preservation of PI, indicating the independence of parts in the perceptual representation, would be consistent with parts-based processing, as would evidence for the presence of crowding, seen in threshold assessments indicating an object inferiority effect. The contours would indicate that when objects are presented in the periphery, the correlation disappeared, with a decrease in accuracy accompanied by an increase in the observer s threshold contrast, the behavioral indicator of visual crowding. Stimulus dimensions are considered perceptually separable (PS) if the distribution of perceptual evidence for one dimension is unchanged across the levels of the other dimension; unlike PI, which is 18

28 specific to the joint distribution of perceptual evidence for a single stimulus, PS requires a comparison of distributions across stimuli. Formally, PS is said to hold if the perceptual effect of dimension 1 does not depend on the level of dimension 2, that is if g h,r (n) = g h,w (n), h = r,w; and if the perceptual effect of dimension 2 does not depend on the level of dimension 1, or if g j,r (e) = g j,w (e), j = r,w. The first equation represents the marginal probability distribution function for the nose across levels r and w, or wide and narrow, respectively, while the second represents the marginal pdf for the eyes across the levels r and w, for the wide and narrow shapes of the eyes. In the GRT contours, the centers of the circles represent the centers of the distributions of the responses to the stimuli, and the preservation of PS is indicated where the centers are equally spaced from each other so that they form a square. A violation of PS is indicated where the spacing between is distorted for one or more of the centers so that they are no longer appear to be orthogonal to each other (Figure 5b). For example, PS would be preserved if the observers distribution of perceptual evidence about the eyes does not change across levels of the nose. PS would be violated if the amount of evidence in support of the eyes being narrow is larger when the nose is narrow rather than wide. Again, holistic processing would be supported by a violation of PS, and according to Martelli and associates (2005) should be tied to evidence of an object superiority effect from the threshold contrasts, while a preservation of PS, and evidence of crowding from the object inferiority effect, would conversely provide support for parts-based processing. In detail, PS could be violated in foveal presentation if the entire distribution of the area of identification accuracies for a stimulus shifted, changing the alignment of the distributions in a way that reflected increased accuracy associated with the absence of an increase in threshold level for contrast, and altering their d-primes (d s). In comparison, no shift in distribution for peripheral presentation, reflecting preservation of PS, would be accompanied by a comparative reduction in accuracy and an increase in threshold, indicative of a crowding effect. 19

29 Finally, two stimulus dimensions would be considered decisionally separable (DS) if the decisional criterion that observers used for one dimension of the stimulus was invariant across the levels of the second dimension. If observers used the same criterion to judge eyes when they looked at narrow or wide mouths, DS would be preserved, but DS would be violated if they used a stricter criterion to judge eye width when they looked at a wide mouth compared to a narrow one. Under an assumption of linear decision bounds, DS would be preserved if those bounds were orthogonal to the coordinate axis of the representational space. This is shown in the GRT contours by the lines intersecting the representational space, which indicate the decision bounds between the distributions. Where the lines are orthogonal, the preservation of DS is indicated. However, where the lines are unevenly slanted so that the decision bounds do not divide the distributions equally, it indicates the violation of DS (Figure 5c). As with PI and PS, a violation of DS indicates that the observer s response to object parts indicates that they are bound together in dependency of representation and processed holistically, while a preservation suggests that parts are separated in representation, and processed by parts. In terms of crowding, evidence of object superiority from the thresholds would indicate holism, according to Martelli et al. (2005), while in GRT observers could be more liberal in their identification of stimulus in foveal presentation, which would be reflected in a shift in the decision bounds of the distributions with an increase in accuracy. However, if the threshold evidence indicated an inferiority effect, or that liberal bias could vanish with peripheral presentation, and instead, accuracy would be decreased, associated with an increase in threshold that would be interpreted as evidence of crowding. For holistic processing to be upheld, the results of these experiments would have to show that there was a violation of at least one of these constructs associated with presentation of the stimuli. Although even one violation, associated with the absence of changes in threshold that indicate a crowding effect, would provide support for holistic processing, it is possible for two or even three violations to occur within a stimulus set. However, there is one additional explanation associated with the crowding effect for a shift in performance between foveal and peripheral presentations of the stimuli. What could occur is that presentations of a stimulus in both the fovea and periphery show no violations (Figure 5d). 20

30 There are no correlations, all the distributions are aligned equally, and the decision bounds remain constant but the whole geometry of the distributions shifts when objects are presented in the periphery. This would indicate impaired performance due to crowding when objects are presented in the periphery, unrelated to any holism in visual processing. In this instance, the object superiority and inferiority effects would be apparent, but without being associated with any evidence for making allowable inferences about holism. This possibility also illustrates one of the benefits of GRT analyses: its conclusions are not dependent on levels of behavioral performance, and significant results can be gathered even from poor performance. These four possibilities contain important implications for Martelli et al. s study (2005). They state that their results indicate holistic processing associated with foveal presentation, based on the absence of crowding, compared to parts-based processing associated with the presence of crowding in peripheral presentation. However, as the above examples indicate, the presence or absence of crowding may reflect a simple detriment or alteration in performance, not consistent with holism, or may be the result of perceptual or decisional separation, which are weaker forms of holism, rather than perceptual independence. This uncertainty reveals the necessity of reconsidering Martelli and colleagues conclusions and their operational definition of holism from the perspective of GRT. Current Experiments In the two experiments of this study, GRT analyses were related to the threshold measures of crowding used in Martelli and associates experiments (2005) to compare operational and theoretical definitions of holistic processing, and determine whether or not crowding can correctly be considered diagnostic of holistic versus parts-based processing. The first experiment replicated Martelli et al. s (2005) experiment 1, applying GRT to compare the influence of facial context associated with peripheral and foveal presentation, and to determine whether either or both types are associated with holistic or parts-based processing. There were three sets of hypotheses, one associated with evidence for crowding and one with evidence about the GRT constructs, pertaining to holistic and parts-based processing, respectively. The 21

31 presence of crowding is reflected by an increase in threshold contrast that occurs when the overlap or blurring of object parts makes the object difficult to identify without being presented at higher contrast, and produces an object inferiority effect. Therefore, it was predicted that if holistic processing occurred, regardless of the location of object presentation, the threshold contrast for peripheral presentation would be equal to the threshold for foveal presentation. There would only be an effect of object superiority, since the stimuli should be perceived as single parts enhanced by their contexts, not as consisting of multiple parts subject to crowding by their contexts. So if the stimuli could be processed holistically in both foveal and peripheral presentation, there would be little if any evidence for crowding, the thresholds for foveal and peripheral presentation would be equal, and the threshold measures would indicate an object superiority effect. Additionally, the GRT analyses were expected to show a violation of at least one of the three constructs, indicating dependency and holism, for both foveal and peripheral presentation. In contrast, if parts-based processing occurred, it was predicted that the thresholds for peripheral presentation would be significantly higher than the thresholds for foveal presentation, with an object inferiority effect occurring the periphery, while the GRT analyses showed that in the periphery, there are no violations, indicating complete independence. However, in the fovea, there are two alternatives: if objects are processed holistically in the fovea in accordance with Martelli et al. (2005) s findings that foveally-presented objects showed no crowding, and the lack of crowding reflected holism at least one construct should be violated. Conversely, if processing is parts-based, independent of presentation location, there would be no violations, indicating complete independence. The third possibility was that evidence for crowding would not correspond with evidence about holism provided by the GRT constructs, if the presence of crowding does not provide an accurate indication of the occurrence of holistic processing. In this case, it was predicted that differences in the thresholds for foveal and peripheral presentation would not be correlated with any particular pattern in the GRT results: both foveally- and peripherally-presented faces could be associated with the same presence or absence of violations. 22

32 The second experiment was also a replication, of experiment 2 performed by Martelli et al. (2005), in which critical spacing and isolation fields were manipulated. In conjunction with GRT analyses, the manipulation of spacing in this experiment determined whether or not GRT conclusions for peripheral presentation alter in any way when all parts are separated into their own isolation fields through enlargement of the stimuli. Since isolating the features should improve identification and eliminate the inferiority effect of context, this should affect the thresholds for contrast for the faces. If the results from the first experiment indicated that in both presentation locations the thresholds were equal and processing was holistic, then the results for threshold and for GRT would not be predicted to significantly alter in this experiment. However, if the results from the first experiment showed a disparity in the thresholds between foveal and peripheral presentation, suggestive of parts-based processing, than it was predicted that these results would show a comparative decrease in peripheral thresholds to approximately equal that of foveal thresholds as crowding is attenuated. If crowding evidences nonholistic, parts-based processing, then it is predicted that relief from crowding should result in more holistic and less parts-based processing, and therefore, more violations of the GRT constructs, compared to the first experiment. However, as in the first experiment, if crowding does not accurately reflect holism in processing, then even if the thresholds for foveal and peripheral presentation change, and even if the number of GRT violations change between the first and second experiments, it was predicted that there will be no difference in the patterns of GRT conclusions between foveal and peripheral presentation. In both of these experiments, assessing Martelli et al. s operational definition of holism with the theoretical definitions of GRT under the conditions of the first and second experiments allowed for the comparison of operational and theoretical definitions, and offered a better understanding of holism in visual processing. together below. GENERAL METHOD Given the high level of similarity between the two experiments, the methods for both are outlined 23

33 Participants In this study, there were a total of nine participants, with five in the first experiment and four in the second experiment. All participants were recruited through postings on University listserves, and had either normal or corrected-to-normal vision and unencumbered use of their hands. Participants were paid eight dollars per hour, and to discourage attrition over the ten day testing period, participants had the financial incentive of being paid two dollars extra per hour if they completed the entire ten blocks. No subject from the first experiment participated in second. Stimuli The stimulus set for the first experiment consisted of ten face stimuli, with three interior facial features (eyes, nose, and mouth) presented in two conditions (wide and narrow, see Figure 2). Two were composed of a nose presented in isolation on a gray background, and the remaining eight were whole faces, in which one, two, or all three features were altered from narrow to wide. In the first experiment, the stimuli size was fixed with mouth size at 1.5 (corresponding with Martelli et al., 2005) and whole face size at 3. For the second experiment, the stimuli were altered so that each featural part was separated from the rest by critical spacing, into its own isolation field. Stimuli were presented at 8 eccentricity, and since critical spacing is equal to roughly half eccentricity (Bouma, 1970), the featural parts needed to be separated by 4 in order to be contained in their own isolation fields. Therefore, the whole face size was increased to 6, but the features within remained fixed at the sizes of the first experiment, determined by mouth size 1.5 (see Figure 3). This enabled an examination of the effects of manipulating critical spacing in foveal and peripheral presentation. The range in luminance contrast for stimuli was between %. Procedure In both experiments, the procedure was identical; the only change was in the manipulation of spacing in the face stimuli used in the second experiment. Participants were seated in front of a computer monitor, at a fixed viewing distance of 76 cm, and chin rests were used. Participants were presented with the stimuli on a VGA monitor that was 45 cm on the diagonal, and presentation consisted of the series of 24

34 face stimuli presented against a mid-scale gray background with a luminance value of cd/m 2. To assess threshold level for contrast for each participant, stimuli were initially presented to participants beginning at 80% contrast (the ceiling anchor used by Martelli et al., 2005), then adjusted based on performance. An adaptive staircase procedure was used, with three correct responses resulting in a decrease in threshold and one incorrect response resulting in an increase in threshold. Presentation of the noses in isolation and within the whole faces was designed to assess the effects of context on identification accuracy when stimuli were presented in the fovea versus the periphery. The foveal stimuli appeared in the center of the monitor, replacing the fixation point, while the peripheral stimuli appeared at eight degrees eccentricity in the right visual field (as in Marelli et al., 2005), with the fixation point remaining present for the duration of the trial. In each self-initiated trial, the participant was presented with a fixation point, then a mask, the stimulus, and another mask, each appearing for 200 ms. In the first experiment, the second mask was followed by a response screen with all of the response alternatives, each presented at 80% contrast and labeled with a digit. In the second experiment, the increased size of the face stimuli (double that of the first experiment) meant that the four response alternatives were too large to be presented together on the monitor. Therefore, paper response screens were constructed and mounted on the monitor so that four faces appeared on either side of the computer screen, labeled with digits. After the second mask, an arrow appeared on the screen, directing participants to make their responses based on the response screen to either the left or the right of the monitor. In both experiments, participants were instructed to indicate the face that they saw by pressing its designated number on the numeric keypad of the computer, and they received feedback on the accuracy of each response. On the first day of each experiment, participants completed learning and practice sessions to improve their identification accuracy of the noses in isolation and in context. The practice sessions were considered completed after participants were able to achieve 100% accuracy on two practice blocks. After the final practice block, participants identified noses (as wide or narrow) presented in the fovea and periphery for the first two sessions of the experiment, with individual threshold assessed using the 25

35 adapted staircase procedure. In the remaining eight sessions of the experiment, only the whole face stimuli were used, and participants were asked to identify pairs of features (eyes and nose, eyes and mouth, or nose and mouth). Participants completed learning and practice blocks for each of the pairedfeatures during the third, fourth, and fifth sessions of the experiment, and were allowed to proceed to the experimental test blocks after demonstrating identification accuracy with all of the face stimuli. Stimuli were presented foveally and peripherally, while stimulus contrast level was kept constant at 80%, so that the areas of distribution of their identification accuracies could be analyzed with respect to any possible violations of the GRT constructs PI, PS, and DS. RESULTS Experiment 1 In the first experiment, it was predicted that the results would reveal a relationship between the behavioral effects of crowding and the evidence of holism according to GRT. Crowding is determined by an increase in threshold for level of contrast for peripheral versus foveal faces when a target feature is presented in the context of other features, relative to when it is presented alone which, according to Martelli and associates (2005) indexes holism. In contrast, the violations of perceptual and decisional dependence and separability the definitions of holism within GRT are obtained by analyses of response frequencies, specifically in the form of identification/confusion matrices. The GRT analyses also make it possible to determine whether or not foveal presentation will regularly produce evidence for perceptual holism, and whether or not peripheral presentation will regularly produce evidence for partsbased processing, as found by Martelli et al. (2005). All analyses were performed on the data within individual observers, with an α level of 0.05 used throughout. Evidence for Context Effects and Crowding Context effects can provide evidence of an object superiority effect, in which the context surrounding an object part improves the recognizability of that part, or of an inferiority effect, in which the surrounding context interferes with identification of the object part. In this experiment, the object part 26

36 was the mouth alone, relative to the mouth presented in the whole face context. A two-alternative forcedchoice staircase using a three-down, one-up rule for adjusting contrasts was used to estimate threshold level of contrast for identification. This procedure converges at an accuracy level of 0.79 (Levitt, 1971). Thresholds for each participant were estimated using the geometric mean of the log values of the final fifteen contrast reversals in each block for each type of stimulus (mouth alone or whole face) at each location (foveal or peripheral). Threshold ratios were calculated, using threshold estimates for the mouth presented alone divided by the threshold estimates for the mouth presented in the context of the whole faces. Once threshold ratios were obtained for foveal and peripheral presentation, the reliability of any differences relative to a criterion value of 1 was assessed, using confidence intervals for each ratio. These were estimated as: R ± z α/2 s n f i i= 1 2 /F R represents the foveal or peripheral ratios, while z α/2 is the z-score associated with the upper and lower α/2 percentage point of the standard normal distribution. f 2 i represents the case weight associated with the i-th ratio (i = 1), so F is equal to locations, or n = six), and s is the standard deviation of the ratio, calculated as n i= 1 f i, the number of groups (three feature pairings at two n 2 s = 1 2 f i ( Ri R) F 1 i= 1 While F and f i 2 represent the same values as in the confidence interval, R indicates the mean ratio for the three foveal or peripheral blocks, and R i indicates the ratio for each of three separate foveal or peripheral blocks, so that the full equation for each participant would be written as s = 1 n F 1 = i 1 f 2 i ( R F1 2 R) ( R F 2 2 R) ( R F 3 2 R). 27

37 For all five participants, the threshold ratios were higher in foveal than in peripheral presentation, and for four out of five, the ratios were significantly different from each other (Figure 6 and Table 1). Observers 1 and 4 showed both a significant foveal superiority effect, and a significant peripheral inferiority effect. For Observer 3, only the foveal superiority effect was significant, and for Observers 2 and 5, only the peripheral inferiority effect was significant (Table 1 and Figure 6). The presence of object inferiority for Observers 1, 2, 4, and 5 indicated that presenting the mouth in the face impaired participants ability to identify the mouth. Identifying mouth stimuli in peripheral presentation was more difficult for participants, and required higher levels of contrast to counter the object inferiority effect and achieve identification accuracy comparable to the fovea. This evidence of crowding partially (i.e., for only two of the five observers) replicates the results of Martelli et al. (2005) s Experiment 1. Their results showed the same pattern: face context improved identification for features in the fovea, with no evidence that the parts conflicted with each other in representation, suggesting a possible holism for foveal vision. However, context created a clear disadvantage for feature identification in the periphery, where the parts of the face crowded and interfered with each other, strongly implying parts-based processing in peripheral vision according to Martelli et al. s hypothesis. Evidence for Holism The identification accuracies for feature pairs were analyzed to determine if there was any evidence to suggest violations of PI, PS, and/or DS. For each participant, six identification/confusion matrices were created, containing the response frequencies for each type of feature pairing (Eyes-Nose, Eyes-Mouth, and Nose-Mouth) at each location of presentation (foveal and peripheral). These response frequencies were then used to estimate a set of marginal and conditional performance measures, including the signal detection parameters z(fa), d, and c for both detection and discrimination sensitivity and bias, along with measures of marginal response invariance and sampling independence. The patterns of equalities and inequalities in these measures were used to support inferences regarding PI, PS, and/or DS. The relationship between tests of the equalities and the allowable inferences are summarized in Tables 2 and 3, with the formal analyses supporting this logic detailed in Kadlec and Townsend (1992). The proofs linking data to the theoretical inferences are based on 28

38 sufficiency arguments only. For example, a violation of PS is always accompanied by inequalities in the d s for the dimensions of the features (e.g., PS is violated where the d for the Eyes is significantly unequal to the d for the Nose), but unequal d s are not always accompanied by violations of PS. Tests for equivalences were performed at two levels of specificity, referred to the signal detection macro-analyses and micro-analyses. The macro-analyses support inferences about perceptual and decisional separability, and include tests for equivalence of marginal sensitivity and bias, and for marginal response invariance (MRI) for each stimulus feature. Assessment of MRI uses response frequencies for the conjunction of two features; for example, to assess MRI for the feature pairing of Eyes and Mouth, tests the equality P(e i m n E i M n )+P(e i m w E i M n ) = P(e i m n E i M w )+P(e i m w E i M w ) which indicates that the marginal response invariance holds for the Eyes (E) at level i, where i can be narrow (r) or wide (w) across the levels of the Mouth (M), either narrow (r) or wide (w), with uppercase letters indicating stimuli and lowercase indicating responses. The marginal tests for z(fa), d, and c of the Eyes across levels of Mouth would be written as: z(fa;e, M r ) = z(fa;e,m w ) and d (E, M r ) = d (E,M w ) and c(e, M r ) = c(e, M w ). The reliability of all differences was established using a Z-test (following Kadlec, 1992). All of the comparisons are designed to reveal the preservation or violation of PS and/or DS, based on whether the equations indicate equivalence or lack thereof. A reliable lack of equivalence indicates shifts in the marginal distributions and marginal response criteria, suggestive of violations of PS or DS. The micro-analyses are tests of sampling independence, and equivalence on measures of discriminability and bias for one feature conditionalized on the level and response of the other feature. 29

39 Evidence of sampling independence indicates that the joint probability of reporting a particular conjunction of stimulus states will be equal to the product of each of the marginal probabilities, equation P(e w m r E i M j ) = [P(e r m w E i M j ) + P(e r m n E i M j )] x [P(e w m r E i M j )+P(e r m r E i M j )], i,j = r,w. For the Eyes conditionalized on the responses when the Mouth is narrow, evidence of sampling independence is based on whether the z(fa), d, and c values for a correct response (the accurate identification of the stimulus) are or are not equivalent to the corresponding values for an incorrect response (the misidentification of the stimulus), expressed as z(fa;e M = correct) = z(fa;e M = incorrect) and d (E M = correct) = d (E M = incorrect), and c(e M = correct) = c(e M = incorrect). The test of sampling independence provides evidence that allows for inferences about perceptual independence, while the tests on the conditionals allow for inferences about both DS and PI. Sampling independence can offer inferences about PI because its test is conducted at the level of responses to individual stimuli, for all possible stimulus states. Therefore, any obtained violations of this equality suggest the possibility that the distribution is distorted in a way that indicates a correlation in the underlying bivariate distribution, and thus a violation of PI. The macro-analysis results for the first experiment of this study are presented in Tables 5 and 6, which provide a more detailed summary of the evidence in support of the violations of Perceptual and Decisional Separability (PS and DS) that appear in Table 4. Parts-based face processing, at either or both presentation locations, would be associated with preservation all of the GRT constructs (PI, PS, or DS), indicating complete separability and independence. However, holistic face processing would be associated with violations of one or more of the three GRT constructs, indicating dependence within 30

40 perceptual representation. Inferences consistent with the contrast threshold crowding inferences and with Martelli et al. s (2005) findings would indicate a complete preservation of the GRT constructs associated with peripheral presentation, suggestive of parts-based processing, and at least one violation associated with foveal presentation, in accordance with holistic processing. The inferences supported by the multidimensional signal detection analyses of the results from the first experiment, the marginal and conditional discrimination of measures of d and c reveal that, for all five participants, violations of the GRT constructs PS and DS occurred in both foveal and peripheral presentation. Although there was no consistent pattern of violations, indicating a considerable degree of individual difference in observers, examining results across participants there were 16 violations associated with foveal presentation, and 19 with peripheral presentation (Table 4). Specifically, the tests on the marginal d s and cs (Table 5) and marginal response invariance (Table 6) indicated that there were six violations of PS and ten of DS in foveal presentation, and four violations of PS and fifteen of DS in peripheral presentation, but the tests on conditionals and sampling independence indicated no violations of PI at all (Tables 7 and 8). Additionally, the results showed more consistent violations when the pair of features were adjacent (the Eyes-Nose and Nose-Mouth), and thus more likely to crowd each other, compared to when the feature pair was more distant (the Eyes-Mouth). In the fovea, there were four adjacent-feature vs. two distant-feature PS violations and seven adjacent vs. three distant DS violations; in the periphery, four adjacent vs. zero distant PS violations and ten adjacent vs. five distant DS violations. These results are shown in Tables 4 through 8, and depicted in Figures 7 through 11. These figures are graphical representations of the inferred perceptual space for each observer in each presentation condition, based on the marginal measures of sensitivity and bias, and on the tests for sampling independence, and are based on a set of simplifying assumptions (Figure 4b). First, it is assumed that, if a stimulus is at level 1 on a given dimension, the relevant coordinate for that dimension is assumed (for graphical purposes only) to be 0. Second, the distances between the points on the margins (the centers of the distributions, represented as filled circles) are the marginal d s. Third, the decision bound locations (estimated by the marginal cs) are estimated relative to the centers and placed on the 31

41 coordinate axes. The lines that pass through the points of the marginal decision bounds are a graphical device to more clearly show changes in the bounds, not the bounds themselves. Fourth, the squares surrounding each of the circles represent the relative frequency of each response for each of the stimuli, with the area being proportional to the relative frequency of a response. The statistical representations can reveal possible violations of PI, PS, and DS. If the distances between the centers on the two pairs of sides (top and bottom, right and left) are not equal (as shown by inequalities in the marginal response invariance and d s), it indicates a possible violation of PS. If the lines connecting the marginal decision bounds are tilted and not orthogonal to the coordinate axes (as shown by inequalities in the marginal response invariance and cs), it indicates a possible violation of DS. And if two or more of the squares are filled in (as shown by inequalities in sampling independence), it indicates the possible violation of PI. There is one contour for each stimulus feature-pair at each location, so a total of six contours per observer. When the inferences from the GRT analyses regarding holism are compared to the findings about crowding and context effects, they indicate a significant disparity, and contrast strongly with the assumption that threshold measures of crowding can provide evidence indicative of the presence or absence of holism. Whereas the evidence of context effects and crowding generally replicate Martelli and colleagues findings (2005), and would according to their hypotheses suggest holistic processing for foveally-presented objects and parts-based processing for peripherally-presented objects, the evidence from the GRT measures indicate that holism can be inferred in both foveal and peripheral presentation. In fact, the most consistent violations were in the peripheral condition. This suggests that not only are faces perceived holistically regardless of presentation location (albeit with only the weaker forms of holism indicated by PS and DS), but that holism seems to be more indicated where crowding is present than where it is absent. Experiment 2 In the second experiment, the goal was to determine whether the threshold and GRT conclusions would change given a manipulation consistent with that used in Martelli and colleagues (2005) 32

42 Experiment 2. They increased size of critical spacing in their stimuli in order to place component parts in individual isolation fields, and found that increasing critical spacing had no effect on foveal presentation, but made an enormous difference in contrast levels for peripheral presentation. When critical spacing was increased, participants no longer required significantly higher levels of contrast to maintain identification accuracy in the periphery, which reinforced the conclusion that peripheral faces were highly subject to changes in spacing and contrast. This provided support for their hypotheses that foveal faces are unaffected by changing conditions and therefore processed holistically, while peripheral faces were highly affected by changes in spacing and contrast, and therefore reflected parts-based processing. In the second experiment of the current study, critical spacing was manipulated through expansion increasing total stimulus size while maintaining the sizing of the interior features from the first experiment to determine whether or not context effects and GRT conclusions would be affected by altered crowding conditions in the periphery. The results from Experiment 1 suggested that Martelli and associates use of crowding to operationally define for holism was invalid, but it was possible that increased critical spacing would have the effect that presentation alone did not: an absence of violations for peripheral presentation that would suggest that altering conditions of crowding can define the presence or absence of holism. All the procedural details of Experiment 2 were identical to those of Experiment 1, with the exception of the altered stimuli and the constructed response screens, which were mounted on either side of the monitor with arrows appearing on the computer screen to direct participants to respond. Evidence for Context Effects and Crowding The thresholds for contrast level and identification frequencies for each participant were assessed using the same methods as in the first experiment. Calculations of the confidence interval of the means of the foveal and peripheral ratios indicated that the four participants all showed threshold ratios for contrast that were higher in the foveal than peripheral conditions, but none of the ratios were significantly different from each other (Table 9 and Figure 12). Only one participant, Observer 2, had a foveal ratio that was reliably different from one, but it was also less than one, a foveal inferiority effect that was a reversal of the pattern found in the first experiment. Similarly, only Observer 1 showed a peripheral 33

43 inferiority effect with a ratio reliably less than one. Otherwise, there were no significant effects the foveal ratios of Observers 1, 3, and 4 were not reliably different from one, nor were the peripheral ratios of Observers 2, 3, and 4. This indicates that separating features into their own isolation fields not only attenuates crowding and the object inferiority effect, as Martelli et al. (2005) found, but also eliminates an object superiority effect. And, contrary to Martelli et al. s (2005) findings that threshold was unaffected by critical spacing in the fovea, these results showed an impact for increased critical spacing on both foveal and peripheral presentation: the elimination of both superiority and inferiority effects for context. The elimination of context effects also indicates the elimination of crowding in stimuli presented in the periphery. As the threshold ratios indicate, the increased critical spacing, designed to relieve crowding by placing each feature into its own isolation field, prevents the features from interacting to crowd one another. These results correspond with Martelli and colleagues (2005) findings that increased critical spacing prevented a crowding effect in the periphery. Evidence for Holism As in the first experiment, the GRT results from all four participants failed to support the hypothesis that only foveal presentation would be associated with violations of PI, PS, or DS, while peripheral presentation would be associated with complete preservation of the constructs (see Tables 10 to 14 and Figures 13 to 16). Again, the signal detection micro-analyses measures of sampling independence and conditional values of z(f), d, and c indicated that no violations of perceptual independence could be inferred from the data (Tables 13 and 14). However, there were instances of inequalities in the signal detection macro-analyses measures of marginal values for d and c (Table 11), and in the marginal response invariance (Table 12), which indicated violations of both perceptual and decisional separability. The summary of violations based on inferences drawn from the macro- and micro-analysis data is presented in Table 10. As before, there were more consistent violations of DS (17 total) than PS (8 total), but in contrast, the violations were more consistent in the fovea than in the periphery (see Table 10). There were only 10 violations in the periphery, three of PS and seven of DS, compared to 15 in the fovea, with five of PS and 34

44 ten of DS. This suggests that separating the features with expanded critical spacing increases people s dependence on holism in foveal vision, but, since overall there were fewer violations than in the first experiment twenty-five instead of thirty-five total that holistic perception is reduced, but far from completely eliminated, by this distortion of face stimuli. Furthermore, the findings concerning adjacent versus distant pairs of features also were altered from the first experiment. Rather than generally showing more consistent violations for adjacent pairs, results were split more evenly, with more violations of perceptual separability associated with distant pairs, and more violations of decisional separability for adjacent pairs. In foveal presentation, there were two adjacent versus three distant PS violations, but six adjacent versus four distant DS violations; in peripheral presentation, zero adjacent versus three distant PS violations, and four adjacent versus three distant DS violations. There was an increase in relative violations of PS in the second experiment compared to the first, in the feature-pairs and in the overall summary of violations specifically, the change in the magnitude of the differences in d was less than the change in magnitude of the differences in c from Experiments 1 to 2. This seems to suggest that increasing critical spacing to isolate the features improved the quantity or quality of perceptual information that participants could derive from the stimuli, so that they relied less on decisional information in their holistic processing of the faces. When related back to the crowding and context effects, these results show the same incongruity as was found in the first experiment. Although all of the findings crowding, context effects, and GRT violations were affected by the distortion of the face stimuli, compared to the first experiment, the pattern of a dissociation between the threshold and GRT measures remains the same. Even when the crowding and context effects were attenuated by the isolation of feature parts in both the fovea and the periphery, violations of PS and DS remained, indicating a holism that, however weak, remained evident regardless of stimuli distortion or presentation location. This lack of correspondence further undermines the link between crowding and holistic processing as posited by Martelli and associates operational definition for holism. 35

45 DISCUSSION The two experiments of this study were designed to assess the validity of using the visual crowding effect (Martelli et al., 2005) to distinguish holistic from parts-based processing of faces. Martelli and associates suggested that increased thresholds for contrast and a crowding effect in peripheral presentation of stimuli reflected parts-based processing, while unchanged thresholds and object superiority effect in foveal presentation reflected holistic processing, but the findings of these experiments indicate otherwise. The results from the first experiment replicated the differences in threshold based on visual location found by Martelli et al., that participants contrast thresholds for peripherally-presented faces were significantly higher than their thresholds for foveally-presented faces. Moreover, the threshold ratios indicated a significant context advantage or object superiority effect for faces in the fovea, and an even stronger crowding impairment or object inferiority effect for faces in the periphery, just as Martelli and colleagues found. However, the GRT conclusions show that these changes in contrast cannot be linked to evidence of holism. Object superiority in the fovea and crowding impairment and inferiority in the periphery were both associated with violations of perceptual and decisional separability, rather than the foveal violations and peripheral preservations that would have been expected based on Martelli et al. s conclusions of holistic processing in the fovea and parts-based processing in the periphery. Instead, the widespread violations of PS and DS found for all observers indicate that they perceived and responded to the face stimuli in a weakly holistic way. Moreover, there was no significant evidence for perceptual independence violations of PI are suggested by tests for marginal response invariance, sampling independence, z(fa), d and c, but even when evidence from all those tests were combined, the evidence for PI violations failed to be sufficient which shows that there is more to holism than simply perception, particularly perceptual dependence within a single stimulus. The fact that the violations were of perceptual and decisional separability, rather than perceptual independence, indicates only limited evidence for holism in the internal representation of the face stimuli. While evidence of violations of PI would indicate a strong form of holism perceptual dependence specific to each stimulus state 36

46 violations of PS and DS are only specific to perceptual and decisional information across the entire stimulus set, which are weaker and more limited forms of holism (O Toole et al., 2001; Wenger & Ingvalson, 2002, 2003). Violations of perceptual separability suggest that the perceptual information for a given feature depends on the information for the second feature across the full set of face stimuli. Besides, violations of PS were less consistent than violations of DS. Violations of decisional separability indicate that holism in internal representation may be due to a shift in the decision criterion people use to judge the identity of features, such that the condition of one feature influenced how observers set their criteria for judging the other feature. Rather than being purely perceptual, the holistic processing of faces in both the fovea and the periphery appears to involve an influential decisional component, which suggests that holism has as much or more to do with decisional influences and top-down information as perception alone in shaping how people process faces. Rather than providing evidence for parts-based processing, the stronger evidence for holism associated with conditions of crowding makes it clear that Martelli and colleagues (2005) operational definition is not consistent with the strong theoretical definition of holism provided by GRT, and therefore that the presence or absence of crowding should not be used to define holism in faces. In the second experiment, the face stimuli were altered to increase critical spacing between the features, and there were corresponding changes in the threshold ratios for contrast of the observers. All of the effects found in the first experiment significant differences between foveal and peripheral ratios, foveal object superiority and peripheral object inferiority disappeared. None of the foveal and peripheral ratios were significantly different from each other, and the effects of context were attenuated as well, so that there was neither an advantage for foveally-presented faces nor an impairment for peripherally-presented faces. Separating the features into their own isolation fields eliminated context effects in the periphery, as Martelli et al. (2005) noted, but also in the fovea, which was not previously observed. The researchers were focused on the effects of changing isolation fields on relieving crowding in the periphery; if they noted corresponding effects relieving superiority in the fovea, it was not included in their publication. The results of the GRT analyses indicate that perceptions of and responses to the 37

47 stimuli were also affected by the increased critical spacing. While violations of perceptual and decisional separability were found in both foveal and peripheral presentation of the face stimuli, the proportionate number of perceptual compared to decisional separability violations increased, relative to the first experiment, and there were more total violations in the fovea than in the periphery. However, the overall violations were much less consistent. Altering the spacing between features increased participants reliance on perceptual versus decisional holism, and generally diminished their ability to see the faces as whole objects united in representation, but did not eliminate it entirely. Limitations Although the conclusions of this study are interesting, they are not without limitations. First, although it did replicate the context effects found by Martelli and associates, the effects were much smaller. The reason for this could be due to item effects that these stimuli were not able to capture contextual elements that may have been unique to the items used in their experiment or to a difference in learning trials. Where Martelli et al. s participants completed a 2,000-trial learning phase, participants in this experiment completed a learning phase including a maximum of 90 trials, which may have been insufficient time for them to learn the stimuli well enough to produce strong context effects. However, the context effects found were still significant for a subset of observers, so it is unlikely that more practice would have dramatically affected the outcomes of the threshold ratios or GRT conclusions. A second possible limitation is that this study focused solely on holism in faces, rather than comparing faces with other visual objects. However, it should be noted that this comparison, while interesting, is not entirely relevant to the central issue of this study: the extent to which crowding (and its absence) is a diagnostic for holistic processing. Future Directions Previous studies have used general recognition theory to examine holistic perception of objects as well as faces, particularly Wenger and Ingvalson (2002; 2003) and Richler, Gauthier, Wenger, and Palmeri (2008). Wenger and Ingvalson (2002; 2003) presented images of human faces, cat faces, cars, and shapes in upright and inverted orientations to participants, and found evidence of holistic processing 38

48 with a distinct decisional basis for all of the stimuli, although holistic processing was disrupted in the inverted condition. Richler and associates (2008) used a sequential response paradigm in which participants were cued to first identify one half of the composite face, then the second, thus incorporating both selective attention as used by Young et al. (1987) and complete identification as used by GRT experiments (Townsend & Ashby, 1986) to examine the composite face effect. They discovered a congruency effect of greater discriminability when the two face halves matched, versus when one half was the same and one different, but that the effect decreased when the halves were systematically misaligned findings consistent with those of Young et al. (1987) and Singer and Sheinberg (2006). However, they also found that the magnitude of misalignment of the two face halves, believed to be associated with the disruption of holistic processing, actually corresponded with the magnitude of the violations of DS. This result, like those of Wenger and Ingvalson (2003; 2003), suggests a strong decisional component in holistic face processing. Furthermore, the variety of different stimuli used in those studies, as well as those employed in the current experiments, indicates that GRT can be used to provide a strong theoretical definition of holism that holds for the processing of all kinds of objects, including faces, presented throughout the visual field. However, while the above studies have explored the inversion and composite effects, no previous GRT study has examined objects other than faces under conditions of crowding in the periphery, so it is possible that different effects would emerge. Most significantly, however, it would be interesting to further investigate evidence of the strong decisional component found in holism particularly in peripheral presentation, under poor visual conditions. Although the results of these experiments reveal that crowding cannot be used as a valid diagnostic for holism, it still has the potential to offer a great deal of insight into basic coding of the visual world, including retinotopic mapping and decreased cell density in the periphery. Peripheral vision provides humans with less information, and what we are getting is of poorer quality, but we still need to be able to use it to navigate the world. But where perceptual information is impaired, the GRT conclusions suggest that holism allows for decisional knowledge-based information to fill in. Therefore, it would be useful to present other objects, abstract features, and word stimuli in the periphery, to see if they also show an 39

49 increase in violations, particularly decisional ones, relative to foveal presentation. Such experiments could reveal evidence that the decisional contribution of holism applies to processing aside from that of faces, and increase the precision of our understanding of how holistic processing functions within the visual system. Conclusion Although the visual crowding effect has great potential for use in cognitive behavioral studies, those future experiments rest on the foundation of the conclusions of this research. By employing GRT analyses to explore Martelli, Majaj, and Pelli s (2005) use of crowding to operationally define holism in face processing, this study was able to identify the flaws in one operational definition for holism, and provide clear results of holistic processing in faces using GRT-based theoretical definitions of holism. The clarification of definitions should benefit vision science by improving terminological precision, expanding available methodologies to include GRT, and facilitating the broader applicability and utility of findings about holism across the field. The results are a more detailed and precise characterization of holism, a clearer conception of the crowding effect and its uses in exploring vision, and a better understanding of how the visual system processes faces whether they appear normal or distorted, at the corner of our eyes or at the center of our vision. 40

50 REFERENCES Ashby, F. G. & Lee, W. W. (1993). Perceptual variability as a fundamental axiom of perceptual science. In S.C. Masin (Ed.), Foundations of perceptual theory ( ). Amsterdam: Elsevier. Ashby, F. G. & Townsend, J. T. (1986). Varieties of perceptual independence. Psychological Review, 93, Bartlett, J. C. & Searcy, J. (1993). Inversion and configuration of faces. Cognitive Psychology, 25, Beale, J. M. & Keil, F. C. (1995). Categorical effects in the perception of faces. Cognition, 57, Bex, P. J., Dakin, S. C., & Simmers, A. J. (2003). The shape and size of crowding for moving targets. Vision Research, 43, Bouma, H. (1970). Interaction effects in parafoveal letter recognition. Nature, 226(241), Bradshaw, J. L. & Wallace, G. (1971). Models for the processing and identification of faces. Perception and Psychophysics, 9, Ellison, J. W. & Massaro, D. W. (1997). Featural evaluation, integration, and judgment of facial affect. Journal of Experimental Psychology: Human Perception and Performance, 23(1), Farah, M. J., Tanaka, J. N., & Drain, H. M. (1995). What causes the face inversion effect? Journal of Experimental Psychology: Human Perception and Performance, 21(3), Farah, M. J., Wilson, K. D., Drain, H. M. & Tanaka, J. N. (1998). What is special about face perception? Psychological Review, 105(3), Kadlec, H. (1995). Multidimensional signal detection analysis (MSDA) for testing separability and independence: A Pascal program. Behavioral Research Methods, Instruments & Computers, 4, Kadlec, H. & Townsend, J. T. (1992). Implications of marginal and conditional detection parameters for the separabilities and independence of perceptual dimensions. Journal of Mathematics Psychology, 36, Klein, S. A. (2001) Measuring, estimating, and understanding the psychometric function: A commentary. Perception & Psychophysics, 63(8), Leder & Carbon, C.-C. (2005). When context hinders! Learn-test compatibility in face recognition. Quarterly Journal of Experimental Psychology, 58, Levitt, H. (1971). Transformed up-down methods in psychoacoustics. Journal of the Acoustical Society of America, 49, Louie, E. G., Bressler, D. W. & Whitney, D. (2007) Holistic crowding: Selective interference between configural representations of faces in crowded scenes. Journal of Vision, 7(2),

51 Martelli, M., Majaj, N. J., & Pelli, D. G. (2005). Are faces processed like words? A diagnostic test for recognition by parts. Journal of Vision, 5, Martini, P., McKone, E. & Nakayama, K. (2006). Orientation tuning of human face processing estimated by contrast matching in transparency displays. Vision Research, 46(13), Massaro, D. W. (1987). Speech perception by ear and eye: A paradigm for psychological inquiry. Hillsdale, NJ: Erlbaum. McKone, E. (2004). Isolating the special component of face recognition: Peripheral identification and a Mooney face. Journal of Experimental Psychology: Learning, Memory, and Cognition, 30(1), McKone, E., Martini, P. & Nakayama, K. (2001). Categorical perception of face identity in noise isolates configural processing. Journal of Experimental Psychology: Human Perception and Performance, 27, Mooney, C. M. (1957). Age in the development of closure ability in children. Journal of Psychology, 11, Nandy, A. S. & Tjan, B. S. (2007). The nature of letter crowding as revealed by first- and secondorder classification images. Journal of Vision, 7(2), O'Toole, A. J., Wenger, M. J., & Townsend, J. T. (2001). Quantitative models of perceiving and remembering faces: Precedents and possibilities. In M. J. Wenger & J. T. Townsend (Eds.), Computational, geometric, and process perspectives on facial cognition: Contexts and challenges (1-38). Mahwah, NJ: Erlbaum. Palmer, S. E., Rosch, E., & Chase, P. (1981) Canonical perspective and the perception of objects. Attention and performance IX, ed. John B. Long & Alan Baddeley ( ). Hillsdale, NJ: Erlbaum Associates. Pelli, D. G., Farell, B. & Moore, D. C. (2003). The remarkable inefficiency of word recognition. Nature, 423(6941), Pelli, D. G., Palomares, M. & Majaj, N. J. (2004). Crowding is unlike ordinary masking: Distinguishing feature integration from detection. Journal of Vision, 4(12), Richler, J. J., Gauthier, I., Wenger, M. J. & Palmeri, T. J. Holistic processing of faces: Perceptual and decisional components. Journal of Experimental Psychology: Learning, Memory, and Cognition, 34(2), Robbins, R. & McKone, E. (2003) Can holistic processing be learned for inverted faces? Cognition, 88, Singer, J. & Sheinberg, D. L. (2006). Holistic processing unites face parts across time. Vision Research, 46, Tanaka, J. N. & Farah, M. J. (1993). Parts and wholes in face recognition. Quarterly Journal of Experimental Psychology A, 46(2),

52 Tanaka, J. N. & Sengco, J. A. (1997). Features and their configuration in face recognition. Memory and Cognition, 25(5), Thompson, P. (1980). Margaret Thatcher: A new illusion. Perception, 9, Watson, A. B.& Pelli, D. G. (1983). QUEST: A Bayesian adaptive psychometric method. Perception and Psychophysics, 79(part 4), Weisstein, N. & Harris, C. S. (1974). Visual detection of line segments: An object-superiority effect. Science, 186, Wenger, M. J. & Ingvalson, E. M. (2002). A decisional component of holistic encoding. Journal of Experimental Psychology, 28(5), Wenger, M. J. & Ingvalson, E. M. (2003). Preserving informational separability and violating decisional separability in facial perception and recognition. Journal of Experimental Psychology: Learning, Memory, and Cognition, 29, Wenger, M. J. & Townsend, J. T. (2006). On the costs and benefits of faces and words: Process characteristics of feature search in highly meaningful stimuli. Journal of Experimental Psychology: Human Perception and Performance, 33, Young, A. W., Hellawell, D. & Hay, D. C. (1987). Configurational information in face perception. Perception, 16,

53 APPENDIX: TABLES AND FIGURES 44

54 Table 1. Experiment 1 threshold ratios and upper and lower bounds of the 95% confidence intervals. 45

55 Table 2. Summary truth table 1 relates outcomes of macro-analyses for one of the features (e.g., the nose) to inferences regarding Perceptual Separability (PS) and Decisional Separability (DS). Note. This table replicates details presented in Wenger and Ingvalsen, 2002, Table 4, p.9. T = true; F = False;? = uncertain inference. 46

56 Table 3. Summary truth table 2 relates outcomes of the micro-analyses to inferences regarding Perceptual Independence (PI) and Decisional Separability (DS). Note. This table replicates details presented in Wenger and Ingvalsen, 2002, Table 5, p.9. Results are given for one of the target features (e.g., the eyes, indicated by 1 ) conditional on the other feature (e.g., the mouth, indicated by 2 ). H = hit; M = miss; CR = correct rejection; FA = false alarm; subscripts o and n = the old/new status of the dimension; T = true; F = False;? = uncertain inference. 47

57 Table 4. Experiment 1: summary of inferences regarding violations of PS and DS. Note. X indicates a violation. 48

58 Table 5. Experiment 1 signal detection macro-analysis marginal estimates for d and c. Note. If d s and cs are equal, PS and DS are preserved; inequalities indicate PS and DS violations. 49

59 Table 6. Experiment 1 signal detection macro-analysis for marginal response invariance (MRI). Note. If MRI probability values are equal, PS and DS are preserved; inequalities indicate PS and DS violations. 50

60 Table 7. Experiment 1 signal detection micro-analyses for sampling independence for the foveal presentation of stimuli. Note. S = stimulus, R = response. The numbers stand for the dimension (eyes, nose, or mouth) of each pairing at each level, 1 = narrow (r) and 2 = wide (w). For example, Eyes-nose pairing S11R22 would indicate that for the presentation of stimulus Eyes(r)-Nose(r), the participant s response indicated stimulus Eyes(w)-Nose(w). If SI values are equal, PI is preserved. A single inequality cannot indicate violation; only a set of inequalities for one S across the entire range of R indicates a PI violation. 51

61 Table 8. Experiment 1 signal detection micro-analyses for sampling independence for the peripheral presentation of stimuli. Note. S = stimulus, R = response. The numbers stand for the dimension (eyes, nose, or mouth) of each pairing at each level, 1 = narrow (r) and 2 = wide (w). If SI values are equal, PI is preserved. A single inequality cannot indicate violation; only a set of inequalities for one S across the entire range of R indicates a PI violation. 52

62 Table 9. Experiment 2 threshold ratios and upper and lower bounds of the 95% confidence intervals. 53

63 Table 10. Experiment 2: summary of inferences regarding violations of PS and DS. Note. X indicates a violation. 54

64 Table 11. Experiment 2 signal detection macro-analysis marginal estimates for d and c. Note. If d s and cs are equal, PS and DS are preserved; inequalities indicate PS and DS violations. 55

65 Table 12. Experiment 2 signal detection macro-analysis for marginal response invariance (MRI). Note. If MRI probability values are equal, PS and DS are preserved; inequalities indicate PS and DS violations. 56

66 Table 13. Experiment 2 signal detection micro-analyses for sampling independence for the foveal presentation of stimuli. Note. S = stimulus, R = response. The numbers stand for the dimension (eyes, nose, or mouth) of each pairing at each level, 1 = narrow (r) and 2 = wide (w). For example, Eyes-nose pairing S11R22 would indicate that for the presentation of stimulus Eyes(r)-Nose(r), the participant s response indicated stimulus Eyes(w)-Nose(w). If SI values are equal, PI is preserved. A single inequality cannot indicate violation; only a set of inequalities for one S across the entire range of R indicates a PI violation. 57

67 Table 14. Experiment 2 signal detection micro-analyses for sampling independence for the peripheral presentation of stimuli. Note. S = stimulus, R = response. The numbers stand for the dimension (eyes, nose, or mouth) of each pairing at each level, 1 = narrow (r) and 2 = wide (w). If SI values are equal, PI is preserved. A single inequality cannot indicate violation; only a set of inequalities for one S across the entire range of R indicates a PI violation. 58

68 Figure 1. Note. The circles represent the size of critical spacing, which needs to increase as eccentricity increases and the isolation field decreases as a function of eccentricity. Critical spacing is assumed to be approximately half the eccentricity, so at 1.5 deg, the circle size reflects a radius of (half the diameter, which is itself half the eccentricity), while at 8 deg, the radius is increased to 2. 59

69 Figure 2. (a) (b) (c) (d) (e) (f) (g) (h) (i) (j) 60

70 Figure 3. (a) (b) (c) (d) (e) (f) (g) (h) (i) (j) 61