BIASES IN LOOKING BEHAVIOUR DURING VISUAL DECISION MAKING TASKS. Mackenzie Gavin Glaholt. A thesis submitted in conformity with the requirements

Transcription

1 BIASES IN LOOKING BEHAVIOUR DURING VISUAL DECISION MAKING TASKS By Mackenzie Gavin Glaholt A thesis submitted in conformity with the requirements for the degree of Doctor of Philosophy, Graduate Department of Psychology, in the University of Toronto. Copyright by Mackenzie Gavin Glaholt, 2010

2 ii BIASES IN LOOKING BEHAVIOUR DURING VISUAL DECISION MAKING TASKS Mackenzie Gavin Glaholt Doctor of Philosophy, 2010 Department of Psychology, University of Toronto ABSTRACT In four experiments we used eye-tracking to investigate biases in looking behaviour during visual decision making tasks. In Experiment 1, participants viewed arrays of images of photographic art and decided which image was preferred (from a set of either two or eight alternatives). To analyze gaze behaviour during the decision we identified dwells (where a dwell is a series of consecutive fixations on a decision alternative). This analysis revealed two forms of gaze bias in the period prior to the response. Replicating prior findings (Shimojo, Simion, Shimojo, & Scheier, 2003), just prior to the response we found an increase in the frequency of dwells on the chosen item. In addition, throughout the decision, dwells on the chosen item were longer than dwells on other items. This pattern of biases was extremely similar across preference and non-preference decision instructions, but overall the biases were more pronounced in eight alternative decisions than in two alternative decisions. In Experiment 2 we manipulated the number of decision alternatives while controlling for differences in the stimulus displays. Participants viewed displays containing six everyday items, and chose either which of two sets of three items was the most expensive (two alternative set selection task) or

3 iii which of the six items was the most expensive (six alternative item selection task). Consistent with Experiment 1, participants exhibited greater selectivity in their processing of stimulus information in the six alternative decisions compared to the two alternative decisions. In Experiments 3 and 4 we manipulated stimulus exposure in order to test predictions derived from the Gaze Cascade model (Shimojo et al., 2003). In Experiment 3, participants performed an eight alternative decision in which four of the items had been pre-exposed prior to the decision. In Experiment 4, stimulus exposure was manipulated during the ongoing decision using a gaze-contingent methodology. While these manipulations of stimulus exposure had strong effects on gaze bias, the specific predictions of the model were not supported. Rather, we suggest an interpretation based on prior research, according to which the gaze bias reflects the selective processing of stimulus information according to its relevance to the decision task.

4 iv ACKNOWLEDGEMENTS I am very grateful to my advisor, Eyal Reingold, for providing continuous support, guidance, and training, throughout my graduate studies. I am grateful to committee members Craig Chambers and Bruce Schneider for their generous guidance and feedback on my research program and thesis. I thank Erik Reichle and Elizabeth Johnson for acting as members of my examination committee and providing feedback on my thesis. I also thank Jiye Shen, Mei-chun Wu, and Karolina Hryciuk for their assistance in the laboratory.

5 v Table of Contents TITLE PAGE ABSTRACT ACKNOWLEDGEMENTS LIST OF TABLES LIST OF FIGURES i ii iv vii viii INTRODUCTION 1 Monitoring Information Search 2 Process Tracing with Eye Movement Recordings 6 Gaze Bias and the Gaze Cascade Hypothesis 11 EXPERIMENT 1: GAZE BIAS AS A FUNCTION OF TASK INSTRUCTIONS AND DECISION COMPLEXITY 18 Method 18 Results 21 Discussion 31 EXPERIMENT 2: CONTRASTING SPATIAL AND DECISION-RELATED GAZE BIASES 36 Method 38 Results 40 Discussion 49 EXPERIMENT 3: GAZE BIAS AND PRIOR STIMULUS EXPOSURE 52

6 vi Method 54 Results 57 Discussion 61 EXPERIMENT 4: GAZE BIAS AND LIMITED STIMULUS EXPOSURE 65 Method 67 Results 70 Discussion 74 GENERAL DISCUSSION 76 REFERENCES 86

7 vii List of Tables Table 1 22 Table 2 28

8 viii List of Figures Figure 1 13 Figure 2 24 Figure 3 29 Figure 4 41 Figure 5 44 Figure 6 47 Figure 7 56 Figure 8 59 Figure 9 69 Figure 10 71

9 INTRODUCTION Early research in decision making focused on explaining and predicting decision outcomes, such as choices and judgments. This approach, known as the structural approach, produces statistical models that account for decision makers responses (outcomes) as a function of the stimulus information and parameters of the decision (inputs). While this approach has been quite successful in probing the overall strategies that decision makers apply, the modeling of outcomes cannot identify different stages of the decision process, or changes in the decision strategy that might occur prior to the final response. In order to shed light on these aspects of decision making, researchers have sought ways to observe more directly the cognitive events that occur prior to the final behavioural response. This approach, which was originally applied to research on problem solving, is known as process tracing (for discussion regarding these two approaches, see Abelson & Levi, 1985; Billings & Marcus, 1983; Einhorn, Kleinmuntz, & Kleinmuntz, 1979; Harte & Koele, 1995; Payne, Braunstein, & Carrol, 1978; Svenson, 1979, 1996). There are several methods of process tracing that have been applied in decision making research. These include verbal protocols, information search displays, and passive eye movement monitoring (for reviews see Ford, Schmitt, Schechtman, Hults, & Doherty, 1989; Riedl, Brandstätter, & Roithmayr, 2008). Verbal protocols involve having the decision maker describe what they are thinking or doing (i.e., think aloud ) while making their decision (concurrent verbal protocol), or having them recall their decision process after the decision has been made (retrospective verbal protocol). While 1

10 2 the verbal protocol methodology has been shown to provide information regarding the sequence of information sampled, and can often suggest the decision strategy that is employed by participants, this method has several potential shortcomings. Concurrent verbalization is effectively a secondary task that the decision maker conducts in parallel with the ongoing decision, the burden of which has been shown to reduce decision accuracy (Russo, Johnson, & Stephens, 1989). Retrospective verbal protocols rely on the decision maker having accurate memory for the decision process as it unfolded. However, it has been demonstrated that decision makers retrospective protocols reflect substantial forgetting and confabulation (Russo et al., 1989). In order to corroborate and expand the information revealed by verbal reports, methods have been developed to more directly observe the patterns of information search that are employed by decision makers. Monitoring Information Search One common method of monitoring information search is through the use of information search displays. In these paradigms (also referred to as information display boards, information display matrices, or computer process tracing), decision makers are presented with a matrix of stimulus information (alternatives by column and attributes by row, or vice versa) and they are tasked with making a decision about the alternatives according to some decision rule. Importantly, the information search display paradigm constrains the way in which the decision maker samples information from the display. The information in each cell in the matrix is hidden. Decision makers must access cells individually, and the displayed information is concealed when the decision maker selects another cell. In this way the decision maker s pattern of information search is made

11 3 explicit. A variety of measures of information search can be derived from this method, such as the depth of search, variability of search, and pattern of search (see Riedl et al., 2008 for a review of these measures). These measures can be used to infer the presence of certain decision strategies, and can detect transitions between different stages of processing (Ball, 1997; Billings & Marcus, 1983; Levin, Huneke, & Jasper, 2000; Payne, 1976; Payne, Bettman, & Johnson, 1993). In the early versions of this paradigm, the information in each cell was written on a card in an envelope, and the decision maker would remove, view, and then replace each card individually. Computerized versions of this paradigm present the information matrix on a computer screen, and participants use a pointing device (often a mouse) to reveal individual cells in the matrix. Computerized versions of the paradigm offer obvious advantages in both ease of use for the decision maker, and in precision of the measurements obtained. However, all versions of this method suffer from a potential shortcoming. In these paradigms the decision maker must execute a deliberate manual act in order to sample each piece of information (i.e., select it by hand). In natural decision situations in which a decision maker is presented with multiple alternatives, decision makers sample information from those alternatives by directing their gaze to them. In general, people make rapid eye movements (called saccades) roughly three to four times each second. The periods between saccades where the eye is relatively still, and visual information is extracted, are known as fixations (for a review of eye movement measures see Rayner, 1998). Eye movements are considerably faster than movements of the hand, and they require less deliberate effort to execute. This difference

12 4 has important implications for process tracing in decision making because it has been argued that effort, and basic information processing limitations such as memory, play a significant role in the way that decisions are made (Payne et al., 1993), and hence any method of process tracing that produces artificial demands of these kind may actually alter the decision process. This issue was investigated by Lohse and Johnson (1996), in a direct comparison of the information search display and eye movement monitoring process tracing methods. Consistent with prior findings (Russo, 1978; van Raaij, 1977), they found significant differences in the process data obtained from these two process tracing methodologies. In particular, Lohse and Johnson found that the information search display paradigm produced longer total time per decision, longer time per information acquisition, lower rate of reacquisition, more information searched, and reduced decision accuracy. Furthermore, these differences became more pronounced in decisions that are more complex (i.e., more alternatives or attributes). Lohse and Johnson concluded that compared to eye movement monitoring, the information search display methodology imposes greater demands in effort and working memory. These findings strongly advocate the use of eye movement recordings as a process tracing measure (but see Reisen, Hoffrage, & Mast, 2008 for a critique). In addition, there are several other reasons why eye movement monitoring may be desirable over other methods of monitoring information search. Importantly, unlike information search displays that are primarily sensitive to deliberate information sampling, eye movement recordings capture a broader range of information sampling acts, which are executed with

13 5 or without conscious awareness. In addition, eye movement recordings might be very useful in supplementing and disambiguating concurrent verbal reports or by serving as powerful cues for retrospective verbal protocols. In a recent study, Eger, Ball, Stevens and Dodd (2007) found that eye movements collected passively during ongoing performance were particularly informative when replayed to participants during a retrospective verbal protocol (eye movement-cued retrospective verbal protocol; see also Hansen, 1991; van Gog, Paas, van Merriënboer, & Witte, 2005). This combination of methods has also been successful in the context of usability research, where the pattern of eye movements can yield insight into the processing steps taken by users (for a review of eye tracking in the field of usability see Ehmke & Wilson, 2007; for a more general review of eye tracking in applied contexts, see Duchowski, 2002). One caveat in using eye tracking as a process tracing measure must be acknowledged. The eye tracking methodology assumes that the decision maker s attention is focused at the point of fixation, though research in visual attention has shown that people are able to direct their attention covertly to areas of the visual field away from their point of gaze (Posner, Snyder, & Davison, 1980). However, during natural viewing, attention and eye movements are tightly coupled: the focus of attention tends to shift to a new location just prior to a shift in gaze to that location (see Hoffman, 1998 for a review). Hence, during decision making tasks where participants are allowed to freely view the decision information, eye movements are generally considered to be valid indicator of the distribution of spatial attention.

14 6 In the past, the use of eye tracking in process tracing has been dissuaded by the cost of the equipment, the relatively low fidelity of the data obtained, and the strict requirements for head-stabilization during recording (e.g., bite-bar). Current video-based eye tracking technology allows for head-free eye movement monitoring while participants view a computer or a projection screen, conditions that are ideal for observing computer-based decisions (e.g., online shopping). Also, lightweight portable eye tracking equipment that is mounted on goggles or eye glass frames is available, allowing for the recording of eye movements while people make decisions in natural everyday decision environments (e.g., store shopping). In addition the spatial and temporal resolution as well as the accuracy of present day eye trackers is vastly superior to that of their predecessors. These improvements in eye tracking technology allow for the possibility of a real time manipulation of the stimulus display as a function of the participant s gaze location. This technique, known as gaze-contingent display, allows for a variety of experimental manipulations (as demonstrated in Experiment 1b and Experiment 4). Hence, eye tracking technology has advanced to the point where it can yield high fidelity process tracing data, in almost any decision environment, with minimal intrusion upon the natural behaviour of the decision maker. Process tracing with eye movement recordings The use of eye movement recordings for process tracing in decision making was pioneered by Russo and colleagues (Russo, 1978; Russo & Dosher, 1983; Russo & Rosen, 1975; Russo & Leclerc, 1994). Russo (1978) argued that eye movement monitoring could provide rich process tracing information that corroborates and extends

15 7 the information obtained from verbal reports and information search displays. Due to limitations in the eye tracking technology available at the time, these early studies tended to focus on the spatial distribution of eye movements (e.g., where the decision maker looks; the order in which information is sampled) rather than the duration of individual eye movements (e.g., fixation duration). Nevertheless, these studies developed several process tracing measures that can be derived from the eye movement record. Russo and Rosen (1975) examined gaze transitions between pairs of decision alternatives while participants chose one of six cars (where each car was described by three attributes). Paired comparisons were identified as sequences in the eye movement record where participants looked back and forth between two alternatives (where the sequence A-B-A was classified as a weak pair and the sequence A-B-A-B was classified as a strong pair). Participants verbal reports corroborated the claim that such transition sequences in fact reflected paired comparisons. The authors found that these sequences tended to involve alternatives that were similar (i.e., sharing attributes). In addition, alternatives involved in paired comparisons tended have higher subjective utility (as revealed by a separate subjective rating task in which participants rated alternatives on a continuum between worst and best ), and were re-fixated more frequently, than alternatives that were not involved in paired comparisons. Together these findings suggest that decision makers may effectively narrow a multi-alternative decision to a decision among a smaller set of competitive alternatives. However, in follow-up analyses regarding the relative utility of the alternatives within each pair, it was not possible to determine the exact nature of the processing steps taking place in paired

16 8 comparisons (such as evaluation vs. elimination). Hence while the eye movement record may be quite useful in revealing the set of decision alternatives that are actively being processed, other sources of information may be required to identify the actual (cognitive) processing steps that take place. In an important illustration of the application of eye movements as a process tracing approach, Russo and Leclerc (1994) used the eye movement record to identify different stages of the decision process. Eye movements were recorded through a oneway mirror while participants made consumer decisions among 16 everyday household items placed on a mock store shelf. The time course of each decision was separated into stages based on the first and last time that the participant s gaze re-fixated a decision alternative. Specifically, the first stage was identified as the period prior to the first refixation on an alternative. The first re-fixation marked the onset of the intermediate (second) stage. The termination of the second stage, and the onset of the third stage of processing, was identified by the first re-fixation on an alternative when counting back from the response (i.e., identical to the method for identifying the first stage, but looking at the sequence of fixations in reverse). Russo and Leclerc hypothesized that the first stage might involve either a screening process, where decision alternatives are processed selectively based on their relevance and inferior alternatives are excluded from further processing, or else an orientation process where alternatives are initially surveyed prior to further processing. The second stage was linked to an evaluative stage of processing where competitive alternatives are compared and the majority of the decision work takes place. The final segment of the decision time course was described as a stage

17 9 involving a review of competitive alternatives just prior to the final response. Aspects of this stage structure have been corroborated by research using the information search display methodology (Wedell & Senter, 1997). Measurements of the spatial distribution of eye movements have also been used to characterize decision makers pattern of information search, variability of information search, and depth of information search, during multi-alternative decisions (Day, Lin, Huang, & Chuang, 2009; Lohse & Johnson, 1996; Pieters & Warlop, 1999; Reisen, et al., 2008; Russo & Dosher, 1983; Rosen & Rosenkoetter, 1976; Selart, Kuvaas, Boe, & Takemura, 2006). Specifically, to index the pattern of information search (developed by Payne, 1976), researchers compare the number of alternative-wise gaze transitions (i.e., moving one s gaze from one attribute to another within a single alternative) and the number of attribute-wise gaze transitions (i.e., moving one s gaze from one attribute in a decision alternative to the same attribute in another alternative). In particular, these measures can describe the extent to which the participant s decision strategy involves the holistic encoding and evaluation of decision alternatives. When confronted with a complex decision scenario (e.g., with many alternatives and many attributes), limitations in information processing capacity may prevent decision makers from encoding each decision alternative holistically along all attributes. Instead, participants may adopt heuristic strategies that result in changes in the pattern of search, depth of search, and variability of search (for a review, see Payne et al., 1993). Specifically, increased complexity brings about a shift towards an attribute-wise search pattern, where alternatives are processed along particular attributes (e.g., the most

18 10 important attributes). In addition, participants may ignore some decision information altogether (reduced depth of search), or process some alternatives or attributes more extensively than others (increased variability of search). In general, measures of search pattern, depth of search, and variability of search obtained from eye movement recordings have provided convergent evidence to prior findings using information search displays. However, some differences emerging from these two process tracing methodologies have been documented. Lohse and Johnson (1996) reported that compared to eye movement monitoring, information search displays induced a more alternative-wise search, with greater depth (i.e., more information sampled), but less variability. In contrast, Reisen et al. (2008) found the information search displays produced a more attribute-wise search, with less variability, but with comparable depth of search to the eye movement monitoring method. Further research is clearly required to tease apart the differences between these two methods of monitoring information search. The process tracing studies described above have tended to focus on the spatial distribution of gaze over the decision information display, but not on the duration of individual fixations on different areas of the display. This was largely due to limitations of the eye tracking equipment that was available at the time, particularly the rate at which gaze position was sampled. Sampling rate of gaze position has improved dramatically in modern technology, allowing for a much more precise estimate of the duration of processing of decision alternatives, though to date only a few studies have taken advantage of this improvement (Pieters & Warlop, 1999; Glaholt & Reingold, 2009a,

19 b; Shimojo et al. 2003; Simion & Shimojo, 2006, 2007; Sütterlin, Brunner, & Opwis, 2008). For example, Pieters and Warlop (1999) monitored participants eye movements while they chose one of six different brands within a single product category (rice, shampoo, canned soup, or salad dressing). The six alternatives for each trial were displayed simultaneously on a computer screen. Of particular interest was to test the commonly held belief regarding consumer decisions that products receiving more attention are more likely to be chosen. In addition, the authors asked whether two factors external to the decision would affect the pattern of eye movements observed during the decision: the motivation of the decision maker (decision rewarded or not), and time constraints imposed upon the decision maker (7 second time limit or unlimited). Though the eye tracker used in this study had relatively low temporal precision by current standards (50 Hz sampling of eye position, compared to 1000 Hz or more for modern research eye trackers), an estimate of the duration of individual eye fixations was obtained. Gaze was biased towards the item that was chosen, where participants spent more time fixating items that were chosen compared to items that were not. This was driven by longer, and more frequent, fixations on the chosen item. In addition, while the manipulations of task motivation and of time constraints did have an impact on eye movements overall, they did not have a significant effect on the bias towards the item that was chosen. Gaze Bias and the Gaze Cascade Model Recently, Shimojo, Simion and colleagues (Shimojo et al., 2003; Simion & Shimojo, 2006, 2007) demonstrated a bias in looking behaviour during two-alternative

20 12 forced-choice (2-AFC) preference decisions. Shimojo et al. (2003) showed participants pairs of faces and had them select the face that they found more attractive by pressing one of two corresponding keys. Gaze position was sampled rapidly using a high fidelity modern eye tracking system. The authors employed an analysis of the eye movement data (referred to as a gaze likelihood analysis ), which plots, for each time point in a period prior to the response, the likelihood that observers gaze was directed toward the stimulus that was eventually chosen. This analysis revealed that just prior to the response there was a progressively increasing bias in the observers gaze towards the chosen stimulus. To explain this effect, Shimojo et al. (2003) proposed a model of preference decisions in which gaze plays an active role in the decision process. This Gaze Cascade Model specifies two component processes related to looking behaviour that interact during preference decisions. The first process is preferential looking, where one tends to look longer at the stimulus that one likes (Birch, Shimojo & Held, 1985). The second process is the mere exposure effect, where merely looking at a stimulus increases preference for that stimulus (Kunst-Wilson & Zajonc, 1980; Moreland & Zajonc, 1977, 1982; Zajonc, 1968). Shimojo et al. (2003) suggested that these two processes can combine to create a positive feedback loop (dubbed a Gaze Cascade) that progressively increases the activation of one of the decision options until it exceeds the threshold for response. In this way, rather than treating eye movements as an indirect measure of decision processes, the Gaze Cascade model holds that gaze itself has an active role in the decision process.

21 13 Figure 1: The positive feedback loop between preference and looking behaviour described in the Gaze Cascade model (Simion & Shimojo, 2007). Shimojo et al. (2003) considered and rejected an alternative explanation that may complicate the interpretation of the gaze bias effect. Specifically, gaze bias might reflect at least in part, a post-decision interval occurring prior to response in which participants continue fixating the chosen stimulus. During such a delay, participants may be engaging in a variety of processes such as memorization of, or programming of, the response. Regardless of the cause, here we use the term response-related explanation to refer to the argument that the gaze bias documented by Shimojo, Simion and colleagues is at least in part due to participants tendency to continue fixating on the chosen stimulus after the decision but prior to the recording of the response. Shimojo, Simion and colleagues (Shimojo et al. 2003; Simion & Shimojo, 2006, 2007) advanced several arguments against a response-related explanation. Specifically, they argued that if gaze bias is due to response-related phenomena, it should not be specific to preference decisions and should occur when other forced-choice decision tasks

22 14 are used. In addition, they argued that a critical goal for their research would involve demonstrating an early gaze bias that occurs long before the response and, consequently, is less likely to be contaminated by response-related processes. To see if the gaze bias was manifest in other tasks, Shimojo et al. (2003) had separate groups of observers view the same pairs of faces under different decision instructions (e.g., choosing the face that was more round). Gaze likelihood analysis revealed a much more pronounced gaze bias in the preference task than in other tasks, a finding which Shimojo et al. (2003) suggested is not consistent with a generic response-related explanation (see also Simion & Shimojo, 2006 for a similar task difference). Looking for evidence of an early gaze bias, Simion and Shimojo (2006) employed a gaze-contingent window paradigm in which participants made preference decisions while viewing pairs of faces through a small circular window that was continually centered on their point of gaze. Face information was available inside the window and was masked outside the window. This viewing mode substantially lengthened trial duration, and importantly a gaze bias was observed as early as seven seconds prior to the response. Simion and Shimojo (2006) argued that such an early gaze bias cannot be solely due to response-related processes. Simion and Shimojo (2007) provided additional evidence against the responserelated explanation of the gaze bias effect. On each trial, a pair of faces was displayed for a random duration and participants were asked to choose the more attractive face. After the disappearance of the faces, participants were asked to indicate their choice if they did not do so while the faces were presented (decision), or to confirm their choice if they

23 15 provided a response earlier (decision confirmation). Examining the gaze bias associated with responses occurring after the disappearance of the display, Simion and Shimojo (2007) demonstrated a much stronger gaze bias effect when such responses constituted a decision rather than a decision confirmation. Based on these findings, the authors argued that the gaze bias reflects, at least in part, decision processes rather than post-decision response-related processes. The present body of research further investigates the gaze bias phenomenon demonstrated by Shimojo and colleagues (Shimojo et al., 2003; Simion & Shimojo, 2006, 2007). In Experiment 1 we examined the effects of decision instruction and decision complexity on gaze bias. Contrary to the preference-specificity predicted by the Gaze Cascade model, we found a comparable gaze bias effect under preference and nonpreference decision instructions. We also documented an increase in the magnitude of the gaze bias in more complex decision scenarios (eight alternatives compared to two alternatives). In addition, we introduce a novel analysis of dwells that improves over the gaze likelihood analysis used in prior research by Shimojo and colleagues. We defined a dwell as the cumulative duration of all consecutive fixations from the moment the participant s gaze enters a grid square containing an image, and until it exits that square. The dwell duration measure is similar to the measure of gaze duration used in reading research, which corresponds to the cumulative duration of consecutive fixations on a word during the initial processing (i.e., first pass) of that word (see Rayner, 1998). However, unlike gaze duration, the term dwell does not distinguish between initial and subsequent visits on an item. Our analysis of dwells allowed for a clearer and more

24 16 detailed depiction of the time course of gaze bias, and provided definitive evidence that the gaze bias is dissociable from the decision outcome (response). In Experiment 2 we contrasted decision-related biases in looking behaviour with biases associated with the spatial layout of the stimulus display. Our results replicated the finding from Experiment 1 that as the complexity of a decision increases (by way of an increase in the number of decision alternatives), so does the selectivity with which information is processed (i.e., reflected in an increased gaze bias). Additionally we documented strong spatial gaze biases across the locations in the stimulus display, but these biases did not depend on the task instructions, nor did they influence the decision outcome. In Experiments 3 and 4 we directly tested predictions that were derived from the Gaze Cascade model regarding the effect of stimulus exposure on gaze bias and choice. Due to the central role of exposure in this model, increased stimulus exposure was expected to magnify the gaze cascade effect, and specifically for preference decisions compared to non-preference decisions. We manipulated stimulus exposure in two ways. In Experiment 3 we manipulated stimulus exposure by pre-exposing a subset of the decision alternatives prior to the 8-AFC decision. In Experiment 4 we employed a gaze-contingent methodology in order to manipulate stimulus exposure during ongoing 8-AFC decisions. The specific predictions of the model were not confirmed in either experiment, leading us to conclude that gaze bias is not driven by a gaze cascade mechanism. Instead, we offer an interpretation based on the prior findings of selectivity in information processing during multi-alternative decision making (e.g., Payne et al., 1993). Specifically, the gaze bias is

25 17 likely related to the selective processing of stimulus information according to its relevance to the decision task.

26 18 EXPERIMENT 1: GAZE BIAS AS A FUNCTION OF TASK INSTRUCTIONS AND DECISION COMPLEXITY The main goal of this experiment was to further explore the time course of the gaze bias effect reported by Shimojo et al. (2003), and Simion and Shimojo (2006, 2007). In particular, we attempted to provide unequivocal evidence of a gaze bias that is independent of response-related phenomena. Experiment 1a was designed to replicate the study by Shimojo et al. (2003). In order to examine the generality of their effect, our 2-AFC preference decision task employed a different stimulus set (black & white photographic art) and a different control task (Recency task: participants were required to choose the photograph that they thought was taken more recently). In order to lengthen trial durations, in Experiment 1b we used a gaze-contingent window paradigm similar to Simion and Shimojo (2006), and in Experiment 1c we introduced an eight-alternative forced-choice (8-AFC) version of the preference and control tasks. In addition to the gaze likelihood analysis, the 8-AFC condition permitted more comprehensive analyses of the time course of gaze behaviour. Method Participants. All participants were undergraduate students at the University of Toronto at Mississauga, and each received $10 for their participation. Separate groups of twelve participants took part in Experiments 1a, 1b, and 1c. Apparatus. The eye tracker employed in this research was SR Research Ltd. EyeLink 1000 system. Following calibration, gaze-position error was less than 0.5. Stimulus displays were presented on a 19-inch Viewsonic monitor. In Experiments 1a

27 19 and 1b, the participant s monitor was set to a resolution of 1024 x 768 and a refresh rate of 120 Hz, and in Experiment 1c it was set to a resolution of 1600 x 1200 and a refresh rate of 75 Hz. Participants were seated 60 cm from the display and a chinrest with a head support was used to minimize head movement. Materials and Design. 400 grayscale images were obtained from an archival database of photographic art that was sold in art auctions (Live Auctioneers website). The photographs varied widely in style and subject matter (e.g., portraits, landscapes, social interactions, objects, architecture, etc.; see Figure 2 for an illustration). In all experiments, each participant performed two tasks: Preference and Recency, and the order of tasks was counter-balanced across participants. In the Preference task the participant was instructed to select the image that he/she liked the most. In the Recency task the participant had to select the image that he/she judged to be photographed most recently (i.e., was the most modern in content and/or style). Across images, there was a small but significant correlation between the number of times an image was selected as the preferred image, and the number of times it was selected as the most recent (Pearson s r = 0.20, p < 0.01). Consequently, it appears that the Preference and Recency decisions have very little variance in common (4%). For Experiments 1a and 1b, AFC image pairs were created such that pairs were closely matched in size. Images were scaled such that their longest dimension (height or width) occupied 12.5 of visual angle (400 pixels) (see Figure 2, Panel a). Half of the pairs appeared in the Preference task and half appeared in the Recency task. Within each task, there were four initial practice trials followed by 96 experimental trials. In Experiment 1b, four practice trials and 50

28 20 experimental trials (a subset of the stimuli used in Exp. 1a) were used in each task. The displays were identical to Experiment 1a, with the exception that a 4.7 gaze-contingent circular window (150 pixels in diameter) was continuously centered on the point of gaze (average delay between physical eye movements and display update was 6.67 ms). Images were unchanged inside the gaze contingent window but were replaced with uniform grey field outside (see Figure 2, Panel c). In Experiment 1c, 384 images were used to create the 8-AFC stimulus arrays (192 in each task). Twenty-four trials were created by randomly dividing the 192 images into sets of 8. This was carried out four times so that each stimulus appeared with a random set of seven other stimuli, four times over the 96 trials in each task. Each task began with four practice trials composed of images that were not used elsewhere. The eight stimuli for each trial were presented in a 3x3 array, where each cell measured 8 x 8 degrees of visual angle (400 x 400 pixels) (see Figure 2, Panel e). Procedure. The trial sequence in Experiment 1a began with a side-by-side presentation of a pair of images (a one pixel wide black frame surrounded each image). The distance between image centers was 15.6 degrees of visual angle (500 pixels) (see Figure 2, panel a). A grey dot was located below each image. To respond, the participant fixated the dot below the chosen image. Having fixated the dot for 400 ms, the dot turned green and a chime sounded to indicate that the selection was recorded. The images then remained onscreen for another 400 ms, and were then replaced with grey rectangles of the same size. The next trial started following a 400 ms interval.

29 21 The trial sequence in Experiment 1b was identical to Experiment 1a with the exception of the gaze-contingent viewing mode (see Figure 2, panel c). Participants were instructed that they had to deliberately explore the images and that when they made a decision they were to fixate the grey dot below the chosen image (grey dots were always visible). Given that the trials were longer and more effortful, they were self-paced (initiated by pressing the space bar). In Experiment 1c, the trial sequence began with the presentation of eight images arranged in a 3x3 grid (center cell empty, gridlines shown as one pixel wide black lines) (see Figure 2, panel e). When the participant had reached a decision, they looked at a grey circle located at the center of the screen and pressed a button on a button box. This caused the circle to turn green, which signaled the participant to then fixate the item they had chosen in order to select it. After having gazed at their choice for 500 ms, a chime sounded and the trial ended. The 3x3 grid remained onscreen between trials, and participants advanced to the next trial by fixating the central (empty) grid square and pressing a button on a button box. Results Prior to analyzing gaze bias we contrasted global measures of performance across the three experiments. To compute global performance measures, we defined a dwell as one or more consecutive fixations on a single image (a dwell ended when participants shifted their gaze to another image). For each trial we recorded the number of dwells, average dwell duration, and the total duration (i.e., summed duration across all dwells). Table 1 presents the averages for each of these global performance measures by task and

30 22 experiment. For each variable, a 2 x 3 mixed ANOVA, which crossed Task (Preference, Recency) and Experiment (1a, 1b, 1c), was performed. Table 1: Means and standard errors for Total Duration, Number of Dwells and Mean Dwell Duration, by Task and Experiment. Condition (Exp.) Total Duration (ms) Number of Dwells Mean Dwell Duration (ms) Preference Recency Preference Recency Preference Recency 2-AFC (1a) 3038 (573) 3961 (714) 3.61 (0.27) 3.91 (0.28) 792 (83) 957 (96) 2-AFC gazecontingent (1b) 8481 (939) 8-AFC (1c) 5064 (562) (1988) 7250 (762) 2.68 (0.09) 10.5 (0.64) 3.33 (0.17) 12.9 (0.63) 3144 (274) 468 (28) 3822 (355) 556 (46) For all three experiments, the Recency task produced longer decision times (F(1,33) = 14.58, MSE = 1.24x10 8, p < 0.001), more dwells (F(1,33) = 16.65, MSE = p < 0.001), and longer dwells (F(1,33) = 4.55, MSE = 1.73x10 6, p < 0.05). In addition, experiments varied substantially in terms of total duration (F(2,33) = 19.56, MSE = 3.34x10 8, p < 0.001), number of dwells (F(2,33) = , MSE = , p < 0.001) and average dwell duration (F(2,33) = , MSE = 6.30x10 7, p < 0.001). Specifically, the free viewing 2-AFC condition in Experiment 1a produced comparable

31 23 decision times to the ones reported by Shimojo et al. (2003) (average of 3 to 4 seconds). As expected, the gaze contingent condition in Experiment 1b and the 8-AFC condition in Experiment 1c produced substantially longer decision times than those obtained in Experiment 1a (both Fs > 8.23, MSE = 9.70x10 7, ps < 0.01). However, the manner in which decision times were lengthened differed dramatically across these two conditions. Compared to the free viewing 2-AFC condition, the gaze contingent condition elicited fewer dwells (F(1,22) = 6.92, MSE = 6.90, p < 0.05), and much longer dwells (F(1,22) = , MSE = 8.16x10 7, p < 0.001), while the 8-AFC condition produced more dwells (F(1,22) = , MSE = , p < 0.001), and shorter dwells (F(1,22) = 15.45, MSE = 1.58x10 6, p < 0.01). We used several convergent analysis methods in order to study the gaze bias effect. Following Shimojo et al. (2003), we performed a gaze likelihood analysis. This analysis plots the proportion of time that participants gaze was directed at the chosen item over the period of time just prior to the decision. The analysis window spanned 2 seconds for Experiment 1a, and 5 seconds for Experiments 1b, and 1c. The gaze likelihood curves averaged across participants are shown in Figure 2. In addition, we applied a bootstrap re-sampling procedure (Efron & Tibshirani, 1994) that re-sampled (with replacement) the data for individual subjects in order to obtain 95% confidence intervals about each time bin.

32 24 Figure 2: Stimulus displays for Experiments 1a (panel a), 1b (panel c) and 1c (panel e), and gaze likelihood plots for Experiments 1a (panels b), 1b (panel d) and 1c (panel f). Gaze likelihood curves plot the proportion of time spent on the chosen item, for each 50

33 25 millisecond time bin in the interval prior to the response. Dotted lines represent 95% confidence intervals about each time bin derived from bootstrapping. Consistent with the findings of Shimojo et al. (2003) and Simion and Shimojo (2006), a substantial gaze bias was documented in all three experiments. In the freeviewing 2-AFC paradigm (Exp. 1a) the bias was evident at 700 ms before the response in the Preference task and at 800 ms prior to response in the Recency task. In the gaze contingent 2-AFC paradigm (Exp. 1b), the bias towards the chosen item deviated from chance at about 2 seconds prior to the response in the Preference task and at about 3 seconds prior to the response in the Recency task. The duration of these effects is shorter than the 7 second effect reported by Shimojo et al. (2006). It is likely that differences between studies in either the size of the gaze-contingent window or the nature of stimulus materials used might account for the differences in the duration of the gaze bias. In the 8- AFC paradigm (Exp. 1c) a gaze bias was present throughout the 5 second window in both tasks. However, unlike prior findings by Shimojo, Simion and colleagues the differences in the gaze likelihood curves between the Preference task and the control task were fairly subtle. In Experiment 1a and 1b, Preference produced slightly steeper curves and higher final values than Recency, while the reverse was true for Experiment 1c. However, the interpretation of the gaze bias observed in the gaze likelihood curves is complex, as it may reflect either longer dwells on the chosen item, more frequent dwells on the chosen item, or a mixture of both. Accordingly, for the 2-AFC conditions (Exp. 1a, 1b) for each item type (i.e., chosen vs. other), we computed total

34 26 duration, average dwell time, and number of dwells. As can be clearly seen in Table 2, for each task in each experiment, there was a significant overall tendency to look longer at the chosen item (i.e., a gaze bias in total dwell time) (all Fs > 12.39, MSE = 1.45x10 6, ps < 0.01). It is also clear that this overall gaze bias was not due to longer dwells on the chosen item (no chosen vs. other difference in Exp. 1a, F(1,11) < 1; significant difference in the wrong direction in Exp. 1b, F(1,11) = 40.56, MSE = 2.97x10 6, ps < 0.01). It is unclear why in the gaze contingent viewing condition dwell durations on the chosen items were on average shorter than dwell durations on non-chosen items. The gaze contingent condition involves laborious and inefficient extraction of task-related information and consequently the termination of a given dwell (i.e., dwell duration) might vary in part depending on the speed with which decision-related diagnostic information is accumulated. If this is the case, the present finding might indicate that the gaze contingent viewing mode produces a tendency to choose items from which taskrelated information is more easily obtained. Rather than being due to differences in mean dwell duration, the overall gaze bias appears to be exclusively driven by a greater number of dwells on the chosen item (both F(1,11) > 45.74, MSE = 0.96, ps < 0.001). In addition, it is important to note that in both experiments, the last dwell was directed at the chosen item in a very high proportion of trials (chance = 0.50; Exp. 1a: Preference = 0.81, Recency = 0.77; Exp. 1b: Preference = 0.84, Recency = 0.79; all ts > 6.13, all ps < 0.001). Thus it appears that in the 2-AFC conditions the bias seen in the gaze likelihood curves toward the end of the trial is strongly influenced by the bias in the placement of the last dwell.

35 27 Compared to the 2-AFC conditions, the 8-AFC condition in Experiment 1c produced a much larger number of dwells overall (see Table 1), and a substantial decrease in the proportion of trials in which the last dwell was directed at the chosen item (Chance = 0.125; Preference: 0.48, Recency: 0.48, both ts > 11.44, ps < 0.001). These two factors permitted a much more detailed examination of the time course of biases in dwell duration and dwell frequency in the 8-AFC condition. Accordingly, we introduced a dwell sequence analysis that compared dwell time and dwell frequency by item type (chosen vs. other), at each of eight dwell positions prior to the response. We also computed dwell time and frequency by item type for the first dwell in the trial.

36 2 8

37 29 Figure 3: Dwell sequence analysis in Experiment 1c: 1) Dwell duration for the chosen and other items for each of the last eight dwells prior to response, and for the first dwell (Preference in panel a, Recency in panel b). Above each sequence position, in parentheses, is the proportion of trials in which the chosen item occupied that serial position. 2) Dwell duration analysis for trials with one, two, or three or more visits to the chosen item (Preference in panel c, Recency in panel d). The percentage of trials falling into each category is printed in parentheses. To create a baseline, chosen items were matched with other items at the same serial position. For each chosen-other pair, the average distance from response is printed below (d = distance in dwells).

38 30 As shown in Figure 3, the dwell sequence analysis reveals a striking difference in the duration of dwells on the chosen item compared to other items, at every point along the dwell sequence. To explore this we conducted a 2 x 2 x 8 repeated measures ANOVA which crossed Task (Preference, Recency), Item type (Chosen, Other) and dwell Position (last eight positions). Dwells were longer in Recency decisions than in Preference decisions (F(1,11) = 8.43, MSE = 1.43x10 6, p < 0.05). More importantly, the chosen item had greater dwell duration than other items at each sequence position (all ts > 2.89, ps < 0.05), and that difference increased toward the end of the trial producing a significant interaction between Item type and Position (F(7,77)=5.16, MSE = 7.11x10 4, p < 0.001). For both tasks, this difference in dwell duration was already present in the first dwell in the trial (F(1,11) = 32.74, MSE = 1.68x10 5, p < 0.001). Unlike the difference in dwell duration that is present at each position, the frequency of dwells on the chosen item (see Figure 3) deviated from chance (0.125) only for the last few dwells in the sequence (Preference: last three dwell positions, all ts > 2.97, ps < 0.05; Recency: last four positions, all ts > 2.33, all ps < 0.05). Thus, in marked contrast to the 2-AFC conditions, in the 8-AFC condition dwell duration appears to be a much more sensitive indicator of gaze bias than dwell frequency. To further explore the difference in dwell duration between the chosen and other items, we divided the trials based on the number of dwells on the chosen item. Given that on average there were approximately two dwells per trial on the chosen item (Preference = 1.86; Recency = 2.18), we divided the trials into three groups, with one, two, and three or more dwells on the chosen item (for trials with more than three dwells

39 31 the last three dwells were analyzed). For each dwell on the chosen item we computed its position in the dwell sequence counting back from the response. We contrasted each dwell duration on the chosen item with the average duration of all dwells in the same dwell sequence position that were directed at non-chosen items. Figure 3 displays mean dwell duration by item type (chosen, other) for trials with one, two, or three or more visits to the chosen item. As shown in the figure, for both the Preference and Recency tasks, trials with one or two dwells on the chosen item demonstrated consistent gaze bias (all ts > 3.43, ps < 0.01). In contrast, for trials with three or more dwells on the chosen item, gaze bias was only evident in the last two dwells on the chosen item (all ts > 2.62, all ps < 0.05). In addition, for trials with two or three dwells, the difference in dwell duration between the chosen and other items increases significantly across visits to the chosen item (both Fs > 21.23, MSE = 2.39x10 5, ps < 0.001). Finally, the duration of the last dwell (or only dwell) on the chosen item did not vary significantly as a function of trial type (both Fs < 1) possibly indicating that the final dwell on the chosen item reflects the operation of obligatory decision processes. Discussion The present experiments replicated the gaze bias effect in preference decisions reported by Shimojo, Simion and colleagues (Shimojo et al., 2003; Simion & Shimojo, 2006, 2007) and provided convergent evidence that this effect represents a very robust phenomenon. Specifically, preference decisions in our free-viewing 2-AFC condition in Experiment 1a produced a pattern of findings that closely matched Shimojo et al. (2003), and in preference decisions in the gaze-contingent condition in Experiment 1b, findings

40 32 were obtained that strongly resembled those reported by Simion and Shimojo (2006). In addition, in Experiment 1c we extended these prior findings by demonstrating a dramatic gaze bias effect for preference decisions in multi-element arrays (i.e., 8-AFC condition). However, one of the important assumptions underlying the Gaze Cascade model proposed by Shimojo et al. (2003) concerns the task specificity of the gaze bias effect. Shimojo et al. (2003) and Simion and Shimojo (2006) examined preference decisions with face stimuli and employed a control task in which participants were asked to judge the roundness of the face. Similarly, in the present experiments, we compared the Preference and Recency tasks. In contrast to Shimojo, Simion and colleagues (Shimojo et al., 2003; Simion & Shimojo, 2006) who documented a qualitative difference between Preference decisions and the control task, across our three experiments the Recency and Preference tasks produced very similar findings. Future research is required to investigate whether, as argued by Shimojo, Simion and colleagues, gaze bias has a unique role in preference decisions. Our findings suggest that the gaze bias effect might be a more general phenomenon characteristic of a variety of visual decisions. One possible difference between the Recency task employed in the present study and the Roundness task used by Shimojo, Simion and colleagues concerns the greater extent of semantic encoding required by the former and the relatively shallow processing required by the latter. Clearly more research is required to investigate the relevance of such task differences to the gaze bias effect. Regardless of the issue of task specificity, the primary goal for the present investigation, and an important goal for prior research by Shimojo, Simion and

41 33 colleagues (Shimojo et al., 2003; Simion & Shimojo, 2006, 2007), was the attempt to show that the gaze bias cannot be accounted for by post-decision response-related explanations. Similar to Simion and Shimojo (2006), we employed manipulations that lengthened the duration of preference decisions, and we looked for an early gaze bias effect that is sufficiently removed from the response and therefore less likely to be influenced by response-related processes. Specifically, Experiment 1b was patterned after the gaze contingent window paradigm introduced by Simion and Shimojo (2006) and led to substantially lengthened preference decisions times. Surprisingly, although preference decision times were almost three times longer in the gaze-contingent condition than in the free viewing 2-AFC condition in Experiment 1a, the former condition produced fewer dwells than the latter condition. Further analysis of the 2-AFC conditions (Exp. 1a, 1b) revealed that while more time was spent on the chosen item overall, individual dwells on the chosen item were not longer than dwells on the other item. Rather, the gaze bias effect was driven primarily by an increase in the frequency of visits to the chosen item and a marked tendency for the final dwell to be directed at the chosen item. Taken together, the small number of dwells produced (3-4 dwells), the absence of a gaze bias in individual dwell duration, and the strong bias in the placement of the final dwell make it difficult to rule out response-related explanations of the gaze bias effects that were demonstrated in the 2-AFC tasks. It is in this context that the 8-AFC task introduced in Experiment 1c provides a unique contribution. Compared to the 2-AFC conditions, the findings from the 8-AFC condition revealed qualitative differences in the pattern of gaze behaviour. Specifically,

42 34 the 8-AFC task produced many more dwells, shorter dwells, and a pattern of increasing bias in individual dwell durations from the very first dwell and throughout the trial. In addition, the gaze bias in dwell duration was evident regardless of the number of visits to the chosen item in the trial. Furthermore, in marked contrast to the 2-AFC conditions, in the 8-AFC condition dwell duration appears to be a much more sensitive indicator of gaze bias than dwell frequency. This is because, in the 8-AFC condition, while the bias in dwell duration is evident from the very first dwell, the dwell frequency bias occurs only in the last few dwells in the trial. Most importantly, the early gaze bias in dwell duration documented in the 8-AFC task cannot be accounted for by response-related explanations. There are several possible differences that may explain these differences in gaze behaviour between the 2-AFC and 8-AFC conditions. The 2-AFC tasks might involve greater reliance on a comparison between the memory representation of the non-fixated alternative with the fixated one. In the case of multi-element arrays, the options being considered may be too numerous to adequately represent in visual working memory, and often non-adjacent, making peripheral processing less efficient. Thus, multi-element arrays might encourage a strategy that relies on rapid and repeated visual inspections of alternatives resulting in shorter and more numerous dwells. This is also consistent with prior studies in decision making (see Payne et al., 1993 for a review) that have found that increase in decision complexity may cause participants to be more selective in their processing of those alternatives. Due to limitations in information processing capacity, participants may adopt an encoding strategy whereby promising alternatives are encoded

43 35 to a greater depth, and weak alternatives are processed to a lesser degree or are excluded altogether from further processing. This possibility is investigated further in Experiment 2. On the whole, we would argue that the present experiments clearly demonstrated the potential usefulness of multi-element arrays for the study of visual decision making. Most importantly, we provided conclusive evidence for an early gaze bias effect that cannot be accounted for by response-related factors. A particularly dramatic illustration of this early bias was obtained by considering the very first dwell in the trial. Specifically, dwell durations were substantially longer when the first dwell was directed at the item that was later chosen as compared to the duration of first dwells on other items. It is important to note that participants did not demonstrate an increased tendency to direct the first dwell to the chosen item, indicating that initial parafoveal or peripheral processing of the stimulus array was ineffective in determining task relevance. Instead, this early bias might indicate that from the very first dwell the task relevance of the fixated item is evaluated, and this evaluation partially determines the first dwell duration. In addition, the magnitude of this differentiation increased in the last few dwells prior to response. Finally, although our findings of gaze bias in the multi-element array conform to the accelerated differentiation pattern predicted by the Gaze Cascade model (Shimojo et al., 2003; Simion & Shimojo, 2006, 2007), our failure to find the task specificity predicted by this model led us to hypothesize that gaze bias might be a more general phenomenon characteristic of a variety of visual decisions.

44 36 EXPERIMENT 2: CONTRASTING SPATIAL AND DECISION-RELATED GAZE BIASES In Experiment 1 we suggested that the finding of larger gaze biases in the 8-AFC decision task compared to the 2-AFC decision task might be the result of an increase in the complexity of the decision task. Prior studies have found that in complex decision scenarios, decision makers tend to become more selective in their encoding of decisionrelevant information (Payne, 1976; for a review see Payne et al., 1993). In particular, increasing the number of alternatives in a decision task is known to increase the complexity of the decision, and to produce greater selectivity in the processing of the decision alternatives and their attributes. When the number of alternatives is small (e.g., 2-AFC), the decision maker is more likely to encode each of the alternatives in depth, and compare them along many of their attributes. In contrast, in multi-alternative decisions which define a larger decision space (e.g., 8-AFC), deep encoding of the alternatives may not always be possible given the limited information processing capacity of the decision maker. As a result, the decision maker might engage in a screening process where weak alternatives are subject to shallow processing and may be excluded from further processing while promising alternatives are processed to a greater extent (Beach, 1993; Russo & Leclerc, 1994; Senter & Wedell, 1997; Wedell & Senter, 1999). The increase in gaze selectivity that we observed in 8-AFC decision tasks compared to 2-AFC decision tasks (Experiment 1, Glaholt & Reingold, 2009a) is consistent with the operation of such a screening process.

45 37 However, it is important to note that in our previous experiment, the 2-AFC and 8-AFC decision tasks involved very different stimulus displays (see Figure 2). Indeed, differences in display characteristics were universally present in all prior experimental manipulations of the number of decision alternatives variable. Furthermore such differences are especially important to consider in the context of visual decision making because it is well established that both visual attention and eye movements are not distributed uniformly over visual displays. For example, several findings have pointed to a rudimentary bias in the eye movement system toward the upper visual field (Durgin, Doyle, & Egan, 2008; Heywood & Churcher, 1980; Honda & Findlay, 1992; Pomplun, Reingold, & Shen, 2001; Previc, 1996; Williams & Reingold, 2001), and to a lesser extent toward the right visual field (Efron & Yund, 1996; Hutton & Palet, 1986). Hence it is possible that differences in eye movement patterns observed in our previous study (Experiment 1, Glaholt & Reingold, 2009a) between the 8-AFC and the 2-AFC tasks were partly due to differences in stimulus displays. Thus, in order to further investigate the influence of the number of decision alternatives on the pattern of choice related gaze bias, the present investigation contrasted looking behaviour in 2-AFC and 6-AFC tasks while holding the stimulus display constant. To accomplish this we contrasted a 2-AFC set selection task in which participants chose between two sets of three items and a 6-AFC item selection task in which participants chose a single item out of six. The stimulus displays and response modes were identical across tasks, which allowed for a direct comparison of biases associated with the spatial layout of the display and biases associated with the decision

46 38 task instructions. The results confirmed our previous finding (Experiment 1, Glaholt & Reingold, 2009a) that an increase in the number of stimulus alternatives causes an increase in gaze selectivity. In addition to this choice-related bias, we also documented a bias in looking behaviour associated with the spatial layout of the display, but this bias was largely insensitive to task instructions and did not significantly affect choice. Method Participants. All participants were undergraduate students at the University of Toronto at Mississauga, and each received $10 for their participation. Separate groups of twenty-four participants took part in the 2-AFC set selection task and the 6-AFC item selection task. Apparatus. The eye-tracker employed in this research was SR Research Ltd. EyeLink 1000 system. Following calibration, gaze-position error was less than 0.5. Stimulus displays were presented on a 19-inch Viewsonic monitor. In Experiments 1a and 1b, the participant s monitor was set to a resolution of 1600 x 1200 and a refresh rate of 85 Hz. Participants were seated 60 cm from the display and a chinrest with a head support was used to minimize head movement. Materials and Design. Stimuli were constructed using an image database containing 144 exemplars from each of 4 categories of everyday object (belts, sunglasses, shirts, shoes) for a total of 576 images. Several online shopping websites were used to extract these images. Each image displayed a product on white background and all images subtended 7.2 x 7.2 degrees of visual angle (360 x 360 pixels). For each of the 4 product categories, the 144 images were divided into 24 sets of 6 items, each of which

47 39 appeared in a single trial, for a total of 96 experimental trials. An additional 6 images from each category were used to create 4 practice trials that familiarized the participant with the procedure. The stimulus displays in the set selection and item selection tasks were identical. In each trial the display consisted of two columns of three cells (each cell subtending 400 x 400 pixels or 8 x 8 degrees with a 1-pixel black border) with one of the columns on the left side of the screen and one on the right side of the screen (see Figure 4). The distance between the centers of each column of images was 18.7 degrees of visual angle (600 pixels). Procedure. In both the set selection task and the item selection task, the participant initiated the trial by fixating at the center of the screen and pressing a button on a button box. The stimulus display was then presented on the screen (see Figure 4). In the set selection task, participants decided whether the set of three items on the left side of the screen or the set of three items on the right side of the screen was more expensive (choose a set). Having reached a decision, the participant selected either the left set or the right set by pressing the left or right button on the button box, respectively. In contrast, in the item selection task, participants were required to choose the single most expensive item out of all six items in the display (choose an item). Having reached a decision, the participant first indicated which side of the screen the most expensive item appeared on (by pressing the left or right button on the button box), and then further identified which of the items in that set of three was their choice (by pressing the top, middle, or bottom button on the button box). Following the participant s final response,

48 40 the screen was blanked for 500 ms and the participant was prompted to initiate the next trial. Results First we examined the spatial distribution of gaze over the stimulus display. Throughout our analyses, we included fixations that occurred from the onset of the stimulus display and until the participant made a response. For each trial we identified a series of dwells, where a dwell is a consecutive run of one or more fixations on a single stimulus area defined as the area of the square containing one of the six display items. For each of the six stimulus areas in the display (see Figure 4) we computed the number of dwells, the mean dwell duration, and the total duration (i.e., the summed duration of all dwells). Each of these measures was analyzed in a 2 x 3 x 2 mixed ANOVA that crossed Horizontal Position (left vs. right) and Vertical Position (top, middle, bottom) as withinparticipant variables, and Decision Task (2-AFC set selection vs. 6-AFC item selection) as a between-participants variable.

49 41 Figure 4: Example of a stimulus display used in both the 2-AFC set selection and the 6- AFC item selection tasks. Consistent with the prediction of greater complexity in the 6-AFC item selection task than the 2-AFC set selection task, total duration was longer for the former rather than the latter (6-AFC total duration over all locations = 5.4 seconds; 2-AFC total duration

50 42 over all locations = 4.1 seconds; F(1,46) = 4.73, MSE = 7.05x10 5, p < 0.05). This was due to longer mean dwell duration (F(1,46) = 24.56, MSE = 4.86x10 4, p < 0.001) but not more dwells (F < 1) in the 6-AFC item selection task. As can be seen in Figure 5, there were strong spatial biases in looking behaviour over the stimulus display. Total duration was biased across the vertical positions in the display (F(2,92) = 61.56, MSE = 1.40x10 4, p < 0.001), with greater total duration on the middle locations than on the top locations, and greater total duration on the top locations than the bottom locations (all t s > 2.48, all p s < 0.05). In addition, there was a tendency for total duration to be biased towards the items on the left side of the display (F(1,46) = 18.31, MSE = 8.20x10 3, p < 0.001). As shown in Figure 5 (panels c and d) these biases in total duration were largely driven by differences in the number of dwells directed to different areas of the display. There was a significant effect of Vertical Position (F(2,92) = , MSE = 0.07, p < 0.001), with a greater number of dwells in the middle locations than on the top locations, and a greater number of dwells in the top locations than in the bottom locations. There were also more dwells directed to locations on the left side of the display than on the right (F(1,46) = 38.95, MSE = 0.03, p < 0.001), and the effect of Vertical Position was greater on the left side of the display than on the right (F(2,92) = 25.53, MSE = 0.01, p < 0.001). The pattern of spatial biases was somewhat different for mean dwell duration (Figure 5, panels e and f). There was a significant effect of Vertical Position (F(2,92) = 64.18, MSE = 1.43x10 3, p < 0.001), but this was actually due to shorter dwells in the middle locations than in the top or bottom locations. The reduction in mean dwell duration for the middle

51 43 location was more pronounced on the left side of the display than on the right (F(2,92) = 12.43, MSE = 9.68x10 2, p < 0.001).

52 44 Figure 5: Measures of looking behaviour as a function of Vertical Position (top, middle, bottom) and Horizontal Position (left, right) in the stimulus display, for each Decision

53 45 Task (2-AFC set selection vs. 6-AFC item selection). Total Duration (panels a and b), Number of dwells (panels c and d) and Mean Dwell Duration (panels e and f). On inspection of Figure 5 it is apparent that the pattern of spatial biases was extremely similar across tasks. There was only one significant interaction involving task, where the magnitude of the bias in total duration toward the left side of the display was slightly larger for the 6-AFC item selection task (F(1,46) = 5.76, MSE = 8.20x10 3, p < 0.05). Based on this insensitivity to differences in the task instructions, it is likely that the spatial biases observed here reflect a visual scanning strategy related to the spatial layout of the display. Furthermore, there was only a marginally significant effect of location on choice in the 2-AFC set selection task (probability of choosing left set = 0.48, probability of choosing right set = 0.52; t(23) = 2.02, p = 0.06). In the 6-AFC item selection task there was no influence of location on choice (top left = 0.17; middle left = 0.18, bottom left = 0.16; top right = 0.17; middle right = 0.17; bottom right = 0.15; all t s < 1.52, all p s > 0.14). Thus despite the presence of strong spatial biases in the allocation of attention to different stimulus locations, stimulus location had no substantial influence the outcome of the decision in either task. Next we tested the hypothesis that choice-related gaze selectivity would be greater in the 6-AFC item selection task than in the 2-AFC set selection task. We characterized the degree of choice-related selectivity in each task in two ways. First, we contrasted gaze behaviour on the chosen side of the display (or the side containing the chosen item in the item selection task) and the other side of the display. Second, we

54 46 examined the degree of differentiation between stimulus items for each side of the display (chosen and other) by comparing the item with the maximum total duration (max item) and the item with the minimum total duration (min item). As in our previous analysis, in each case we computed the total duration, the mean dwell duration, and the number of dwells. Each of these variables was analyzed in a 2 x 2 x 2 mixed ANOVA crossing Side (Chosen vs. Other) and Item (Max vs. Min) as within-participant variables, and Decision Task (2-AFC set selection task vs. 6-AFC item selection task) as a betweenparticipants variable. For the purpose of this analysis, trials in which less than two of the items were viewed on either side of the display were excluded (< 1%).

55 47 Figure 6: Measures of choice-related biases in looking behaviour. Max and min items, on the chosen and other side of the display, for the 2-AFC set selection and the 6-AFC

56 48 item selection tasks. Total Duration (panel a), Number of Dwells (panel b) and Mean Dwell Duration (panel c). As can be seen in Figure 6, gaze was biased towards the chosen side of the display under both task instructions. However, the bias was much larger in the 6-AFC item selection task than in the 2-AFC set selection task, in total duration (F(1,46) = 45.77, MSE = 3.60x10 4, p < 0.001), mean dwell duration (F(1,46) = 62.32, MSE = 2.27x10 3, p < 0.001), and in number of dwells (F(1,46) = 26.49, MSE = 0.02, p < 0.001). Note that for the 6-AFC item selection task, the chosen side of the display contains items that were not chosen, which should tend to reduce the difference between the chosen and other sides and thereby underestimate the degree of selectivity in this task. Nevertheless, this analysis still demonstrated a greater selectivity in processing the two sides of the display in the 6-AFC task. A more precise indication of greater differentiation in the 6-AFC item selection task than in the 2-AFC set selection task may be seen in the difference between the max and min items across tasks in either the chosen side or the other side of the display. For the chosen side, the difference between the max and min items was larger for the 6-AFC item selection task than the 2-AFC set selection task, in total duration (F(1,46) = 40.35, MSE = 1.06x10 5, p < 0.001), number of dwells (F(1,46) = 23.28, MSE = 0.03, p < 0.001), and mean dwell duration (F(1,46) = 77.92, MSE = 5.16 x 10 3, p < 0.001). Interestingly, even for the side that was not chosen, there was greater differentiation between the max and min items for the 6-AFC item selection task than for the 2-AFC set selection task in

57 49 terms of mean dwell duration (F(1,46) = 16.14, MSE = 2.57x10 3, p < 0.001) and total duration (F(1,46) = 6.41, MSE = 3.27x10 4, p < 0.05), but not in number of dwells (F < 1). The pattern of max-min differences across tasks were stronger for the chosen side than for the other side, resulting in significant three-way interactions for total duration (F(1,46) = 57.36, MSE = 2.25x10 4, p < 0.001), number of dwells (F(1,46) = 53.10, MSE = 0.01, p < 0.001), and mean dwell duration (F(1,46) = 60.17, MSE = 1.54x10 3, p < 0.001). Taken together, the present findings provide strong evidence that the 6-AFC item selection instructions produce a greater degree of differentiation in processing of stimulus items than the 2-AFC set selection instructions. Importantly, the present findings show that this increase in selectivity as a function of task requirements occurs even when the amount of stimulus information in the display is held constant. Discussion A body of prior research in decision making suggests that as the complexity of a decision increases (e.g., through an increase in the number of decision alternatives), decision makers become more selective in their sampling of decision information (for a review see Payne et al., 1993). This idea was supported by our prior finding of a greater selectivity in an 8-AFC task than in a 2-AFC (Experiment 1, Glaholt & Reingold, 2009a). However, prior manipulations of the number of decision alternatives involved changing the amount and arrangement of stimulus information that was presented. Given that the characteristics of stimulus displays are known to influence the way observers sample information with their eyes, an increase in gaze selectivity with a more complex decision task might be due to differences in the stimulus displays. In the present study we

58 50 compared a 2-AFC set selection task with a 6-AFC item selection task. Importantly, the stimulus displays were identical for each of these decision tasks, allowing for a direct comparison of the pattern of gaze selectivity across tasks. Our analysis of the distribution of gaze across the stimulus display confirmed the presence of strong spatial biases in both decision tasks. Consistent with prior findings (Durgin et al., 2008; Heywood & Churcher, 1980; Honda & Findlay, 1992; Pomplun et al., 2001; Previc, 1996; Williams & Reingold, 2001), we found evidence for greater allocation of spatial attention to top locations relative to bottom locations in the display. In addition, gaze was strongly biased towards the middle locations relative to the top and bottom locations, and we also observed a slight bias towards the left side of the display. These biases did not have a substantial influence on choice, and they were extremely similar under the 2-AFC set selection and 6-AFC item selection instructions. Hence, they are likely to reflect a visual scanning strategy related to the layout of the stimulus display, but that does not interact with the specific requirements of the decision task. In contrast, we found robust choice-related biases in looking behaviour that were very sensitive to the task instructions. In particular, the 6-AFC item selection task exhibited a greater degree of differentiation between the chosen side of the display and the other side of the display, in terms of total duration, number of dwells, and mean dwell duration. The 6-AFC task also showed greater differentiation between individual stimulus items within each side of the display. These findings are consistent with the findings from Experiment 1, where we observed an increase in gaze selectivity in eight alternative decisions compared to two alternative decisions. However, we acknowledge

59 51 that in addition to the difference in the number of alternatives, there might be differences in the processing requirements of the set selection and item selection tasks. Specifically, the 2-AFC set selection task required the decision maker to integrate the value of three items in each of two sets (i.e., how expensive the items are as a group) and then compare those two sets. Conversely, the 6-AFC item selection task required the decision maker to differentiate and compare the values of six individual stimulus items. Nevertheless, the comparison of the 2-AFC set selection and 6-AFC item selection tasks was necessary in order to rule out differences in eye movement behaviour associated with differences in the stimulus displays. When the present results are taken together with the findings of Experiment 1, we find consistent evidence that as the number of decision alternatives increases, the selectivity with which information is processed also increases. We speculate that this is because decisions with more alternatives place greater demands on working memory, which may cause decision makers to employ heuristic strategies (e.g. a screening process), which is in turn reflected in an increase in the degree of selectivity in the processing of stimulus information, where relevant information is processed in greater depth and less relevant information is processed to a more limited extent or excluded from further processing. In summary, the present experiment provides additional support for the general hypothesis that as decision complexity increases, so does the selectivity with which decision makers process stimulus information. Importantly, we have shown that this occurs even under conditions where stimulus information was held constant across conditions.

60 52 EXPERIMENT 3: GAZE BIAS AND PRIOR STIMULUS EXPOSURE The Gaze Cascade model assumes that two component processes related to looking behaviour interact to produce the observed gaze bias in preference decisions. One process is the mere exposure effect, where looking at a stimulus increases preference for that stimulus (Kunst-Wilson & Zajonc, 1980; Moreland & Zajonc, 1977, 1982; Zajonc, 1968). The other process is preferential looking, where one tends to look longer at the stimulus that one likes (Birch et al., 1985). Shimojo and colleagues suggested that these two processes can combine to create a positive feedback loop (i.e., a Gaze Cascade) that progressively increases the activation of one of the decision options until it exceeds the threshold for response (for the most recent description of this model, see Simion & Shimojo, 2007; p. 591). Shimojo and colleagues acknowledge that the Gaze Cascade model is by no means the only determinant of preference decisions and that other factors such as pre-existing preferences influence the outcome of decision processes. However, all other factors being equal, the positive feedback loop postulated by the Gaze Cascade model (see Figure 1) is expected to produce a substantial influence on preference decisions. Due to its reliance on preference-specific mechanisms, the Gaze Cascade model predicts that the gaze bias should be uniquely present in preference decisions. To test this prediction, Shimojo et al. (2003) contrasted preference decisions with a control task in which participants judged which of two faces was more round (roundness decision). The gaze bias was significantly more pronounced in preference decisions than in the roundness decision. In a second experiment, Shimojo et al. (2003) demonstrated that

61 53 longer exposure to a face stimulus increases the likelihood of choosing that face in a 2- AFC preference decision, but not in a 2-AFC roundness decision. To test the generality of the findings reported by Shimojo and colleagues, we introduced an eight-alternative forced choice (8-AFC) paradigm where participants made preference decisions among eight photographs (Experiment 1c, Glaholt & Reingold, 2009a). A robust gaze bias was observed in the 8-AFC decisions. In addition, an analysis of the dwell sequence during the trial (a dwell being a series of one or more consecutive fixations on a single alternative) demonstrated a bias in both dwell duration on the chosen item, and dwell frequency on the chosen item. Specifically, from the very first dwell and throughout the trial, dwells on the chosen item were longer than dwells on other items, and dwell frequency on the chosen item was significantly greater than chance in the last few dwells prior to response. However, in contrast to the prediction of the Gaze Cascade model, a comparison of a preference decision task and a control task in which participants decided which photograph was taken most recently (the Recency decision) revealed a very similar pattern of gaze bias across tasks. Accordingly, we argued that the observed gaze bias pattern might reflect a general phenomenon in visual decision making rather than being restricted to preference decisions as suggested by the Gaze Cascade model. The main goal of this experiment was to provide a further test of the Gaze Cascade model by manipulating stimulus exposure. On each trial, differential exposure across alternatives was produced by providing a preview of four of the eight alternatives prior to the 8-AFC preference decision. The Gaze Cascade model predicts that

62 54 differential stimulus exposure across alternatives would result in a stronger gaze bias effect for items that were pre-exposed. In addition, pre-exposed items would be predicted to have a greater likelihood of being chosen in the preference decision task (Shimojo et al., 2003; see Figure 1). To examine the task specificity of any observed effects of pre-exposure, we contrasted the preference decision task with a control task in which participants were asked to choose the most unusual photograph out of eight alternatives (Typicality decision). Method Participants. All participants were undergraduate students at the University of Toronto at Mississauga, and each received $10 for their participation. Decision was manipulated between participants, with 16 participants performing the Preference decision, and 16 participants performing the Typicality decision. Apparatus. The eye tracker employed in this research was SR Research Ltd. EyeLink 1000 system. Following calibration, gaze-position error was less than 0.5. Stimulus displays were presented on a 19-inch monitor. The participant s monitor was set to a resolution of 1600 x 1200 and a refresh rate of 85 Hz. Participants were seated 60 cm from the display and a chinrest with a head support was used to minimize head movement. Materials and Design. Stimuli were drawn from a set of 384 grayscale images of photographic art (from Experiment 1). The photographs varied widely in style and subject matter (e.g., portraits, landscapes, social interactions, objects, architecture, etc.). These photographs were used to create 48 8-AFC experimental trials. Additional stimuli

63 55 that were not used in experimental trials appeared in 4 practice trials, which served to familiarize the participant with the task. Decision task was a between-participants variable. In the Preference task the participant was instructed to select, from the eight alternatives, the image that he/she liked the most. In the Typicality task the participant had to select the image that he/she judged to be most unusual (i.e., most out of the ordinary, least typical). Across images, there was no significant correlation between the number of times an image was selected as the preferred image, and the number of times it was selected as the most unusual (Pearson s r = 0.08, n.s.). The eight stimuli for the 8-AFC decision in each trial were presented in a 3x3 array, where each cell measured 8 x 8 degrees of visual angle (400 x 400 pixels). The middle cell was empty except for a fixation circle (see Figure 7).

64 56 Figure 7: Stimulus display for the 8-AFC task. Prior to each 8-AFC display, four of the eight images were shown to the participant, one after another. When the 8-AFC display was presented, participants chose the photograph they preferred (Preference) or the photograph that was the most unusual (Typicality).