CORRESPONDENCE MATCHING AND ACTION PLANNING IN CYCLOPEAN VERSUS LUMINANCE APPARENT MOTION PERCEPTION

Transcription

1 CORRESPONDENCE MATCHING AND ACTION PLANNING IN CYCLOPEAN VERSUS LUMINANCE APPARENT MOTION PERCEPTION By ALAN SCOTT BOYDSTUN A dissertation submitted in partial fulfillment of The requirements for the degree of Doctor of Philosophy WASHINGTON STATE UNIVERSITY Department of Psychology MAY 2009

2 To the Faculty of Washington State University: The members of the Committee appointed to examine the dissertation of ALAN SCOTT BOYDSTUN find it satisfactory and recommend that it be accepted. Robert Patterson, Ph.D., Chair Lisa Fournier, Ph.D. Paul Whitney, Ph.D. John Wright, Ph.D.

3 ACKNOWLEDGMENT I would like to take this opportunity to thank those that have helped me with this project, especially, Dr. Robert Patterson, and the participating Committee members, Drs. Lisa Fournier, Paul Whitney, and John Wright. Thank you for your instruction, guidance, and support regarding this project. I would also like to acknowledge Mercedes LaVoy for her help with editing and additional support. ii

4 CORRESPONDENCE MATCHING AND ACTION PLANNING IN CYCLOPEAN VERSUS LUMINANCE APPARENT MOTION PERCEPTION Abstract by Alan Scott Boydstun, Ph.D. Washington State University May 2009 Chair: Robert Patterson A debate exists regarding the relative processing level of cyclopean-defined motion in the visual system. Some authors have attributed the processing of cyclopean-defined motion to stages in the visual system that enact similar algorithms of motion computation to luminance-defined motion; whereas, other authors attribute the processing of cyclopeandefined motion to cognitive processes that track stimuli features. Thus, the purpose of this study was to provide evidence that bears upon the debate regarding the relative processing stage of cyclopean motion. To explore this debate we utilized an action capture paradigm. Action capture refers to the possible influence of bodily movements on visual perception. In this study, we used bi-directional hand movements as a possible bias for the perception of motion direction in an ambiguous apparent motion display. Thus, observers (N = 60) executed (Experiments 1 and 2) or planned (Experiment 3) directional (i.e. horizontal, vertical, or none) hand movements while viewing either luminance- or cyclopean-defined apparent motion. It was predicted that if cyclopean motion is processed at a relatively higher stage iii

5 in the visual stream, then cyclopean motion processing would be more susceptible to the possible influence of action capture than luminance motion processing. However, if cyclopean and luminance motion processing are both processed at similar stages in the visual stream, then cyclopean and luminance motion processing would be equally susceptible to the possible influence of action capture. The results of this investigation showed that directional hand movements could significantly bias the perceived direction of motion in apparent motion displays; however, this bias was no different for luminance and cyclopean defined apparent motion. Thus, we found no evidence that suggests cyclopean motion processing occurs at relatively higher levels of the visual stream where feature tracking may occur. iv

6 TABLE OF CONTENTS Page ACKNOWLEDGEMENTS...iii ABSTRACT...iv LIST OF FIGURES...vii CHAPTER 1: INTRODUCTION...1 CHAPTER 2: GENERAL METHODS...11 Stimuli...11 Apparatus...11 General Procedure...13 CHAPTER 3: EXPERIMENTATION Experiment Experiment Experiment CHAPTER 4: GENERAL DISCUSSION...27 REFERENCES...33 FIGURE CAPTIONS...38 APPENDIX IRB Documentation...48 END NOTES..53 v

7 LIST OF FIGURES 1. Attention diverted versus attention directed Two-possible motion paths Motion Sequence Experiment 1 Results Experiment 2 Results Experiment 3 Results Distributed Channels...47 vi

8 CHAPTER ONE INTRODUCTION One of the key advantages for possessing normal binocular vision is the ability to process binocular disparity, the cue for stereoscopic depth perception. This ability seems to be conferred to predators of the animal kingdom (Fox, 1978). When an animal or individual with binocular vision moves through his or her environment and executes eye movements, whether reflexive or voluntary, or when objects themselves move, the disparity information that the individual receives continuously changes. When disparity information continuously changes, it is called dynamic disparity. Furthermore, when the dynamic disparity is moving it is called cyclopean motion, where the term cyclopean refers to visual processing at and beyond levels of binocular integration. Interestingly, the way in which the visual system processes dynamic-disparity information or cyclopean motion has been controversial (Patterson, 1999). The controversy rests on the question as to whether cyclopean motion is processed by a binocular motion systemi or by a more general-purpose cognitive-type mechanism (Anstis, 1980; Braddick, 1980). By cognitive-type mechanism, we are referring to a system that tracks features, possibly via attentional mechanisms, by comparing the relative position of a given visual stimulus rather than actually computing some type of motion algorithm. For example, such a feature-tracking system would be responsible for an observer s ability to detect the change in the position of an hour hand on an analogue clock (Nakayama, 1985). The hour hand would move too slowly to engage our motion system 7

9 and thus such an ability would rest on detecting position information only, regardless of time. Thus, detecting a change in position is qualitatively different than detecting the objects actual motion. The question, then, is whether cyclopean motion is processed by a true motion system or rather by an attentionally-mediated positional-feature-tracking mechanism, the latter of which would not be considered motion processing, as mentioned above. Much of this debate stems from a body of literature that shows an inability to demonstrate motionprocessing characteristics with cyclopean-defined stimuli, an inability that is due, in part, to researchers incorrectly interpreting null results from studies of insufficient methodological design (Patterson, 1999). For example, Papert (1964) and Steinbach and Anstis (1976, as cited in Anstis, 1980) attempted to demonstrate a cyclopean motion aftereffectii, which refers to a visual illusion of motion created by adapting to a given motion direction, and which is considered a signature of early motion processing (Pantle, 1974; Cavanagh & Mather, 1989). Both of these studies showed that participants did not perceive a cyclopean-motion aftereffect. Thus, the authors concluded that there was no motion-processing system at binocular levels of vision (i.e. their conclusions amounted to committing the denial-of-the-antecedent logical fallacy; Patterson, 1999). However, Patterson et al. (1994) and Bowd et al. (1996) showed that enduring cyclopean motion aftereffects could be obtained if adaptation durations are sufficiently long. Obscuring the nature of cyclopean-motion processing further, Braddick (1974) postulated a dichotomy between types of motion processing, called short- and long-range motion processing. Short-range motion processing was assumed to operate on a point-by- 8

10 point correspondence process, showed adaptation effects, and did not involve feature tracking. Long-range motion processing, however, operated on a feature-identification process, involved feature tracking and attention, and did not show adaptation effects. During the short-range/ long-range tenure, cyclopean-motion processing was associated with long-range motion processing because it did not seem to show adaptation effects, as previously stated (Steinbach & Anstis, 1976, as cited in Anstis, 1980; Papert, 1964). Another consequence of this conceptualization was that cyclopean motion was assumed to be processed with an attentionally-mediated feature-tracking system (i.e. presumably a critical aspect of long-range motion) because it seemed logical that relative depth percepts that are generated by this type of stimulus attract visual attention. Cavanagh and Mather (1989) critiqued the short-/long-range motion-processing theory, by explaining that motion processing, in general, may involve the same motion algorithm. Thus, according to these authors, the short- and long-range distinction only reflected differences in front-end filtering. Front-end filtering, in this context, refers to the way in which visual information is brought into the motion system. For example, luminance-defined edges are characterized by unipole statistics and detected both monocularly and binocularly, whereas cyclopean-defined edges are characterized by dipole statistics and detected only binocularly. These differences in stimulus properties, however, likely play no important role in motion processing because they are filtered out of the information that is brought in to the motion system. These authors contend that once the information projects to the motion system, the pattern of edge movement becomes the critical aspect of stimulus motion, regardless of the stimuli s initial physical 9

11 characterization. Thus, it is likely that a similar motion-analysis framework can explain the visual processing of moving luminance-defined and disparity-defined (i.e. cyclopean) edges. Although the same actual motion algorithm is likely utilized when the brain analyzes different types of motion, the type and extent of front-end filtering, and the relative placement of the motion computation along the motion-processing hierarchy, is still debated. Furthermore, authors debate as to what degree the placement of the motion computation along the motion-processing hierarchy affects its susceptibility to cognitive influences. With respect to cyclopean-motion processing, this issue has led to a current disagreement about the degree to which visual attention, and possible other top-down cognitive influences, plays a role within the cyclopean motion-processing system. In such processing, the role that attention and other cognitive influences may play is usually discussed in the context of theories of multiple stages of motion processing. Thus, currently in the literature, authors debate the existence of a two-stage processing model (i.e. first- and second-order motion processing; Nishida & Sato, 1995, 1996, 1997; Patterson, 1999) or a three-stage processing model (i.e. first, second, and third-order motion processing; Lu & Sperling, 1995, 2001). The two-stage processing model consists of first- and second-order motion processing. First-order motion processing is applied to first-order stimuli, whose boundaries or borders are defined by differences in mean luminance. This type of processing does not require any additional front-end processing. Second-order motion processing is applied to second-order stimuli, whose boundaries are defined by differences in contrast, texture, or disparity (i.e. including 10

12 cyclopean motion) without any differences in mean luminance. This type of processing is thought to entail a front-end rectification operation, followed by a motion energy computation. According to the three-stage processing model (Lu & Sperling, 1995, 2001), the first two stages of processing are equivalent to the descriptions of the two-stage model, except second-order motion does not include disparity-defined motion processing. Thus, in the three-stage model, the third level of processing reflects an attentional-featuretracking mechanism that acts upon disparity-defined information. In this framework, visual information is represented by an attentional-salience map. Before cyclopean motion is processed, attentional mechanisms act to improve the figure-to-ground saliency of disparity-defined stimuli in the map. Note that, regardless of the debate of how many levels of motion processing comprise the visual system, the real issue of contention is the relationship of high-level cognitive processing to cyclopean-motion processing, or in other words, whether such motion processing occurs early or late in the motion stream. One approach to addressing this issue of whether cyclopean-motion processing occurs early or late in the visual stream would be to consider whether manipulations of high-level cognitive processing, like the ability to allocate attention, differentially affects motion processing of one or the other type of stimulus. However, the results from this type of study have been inconclusive. For example, studies that have examined the affects of cognitive processing on texture-defined motion, which is considered another visual cue for motion processing, have been mixed (Allen & Ledgeway, 2003). Allen and Ledgeway (2003) found evidence that suggested that luminance-defined motion processing was 11

13 affected less by diverted attention than texture-defined motion processing. This result suggested that texture-defined motion was processed at a higher stage than luminancedefined motion. However, Allen and Derrington (2001) found that contrast-defined and luminance-defined motion processing were equally affected by diverted attention tasks. Although interesting, these studies relied on attention-diverting paradigms that take the participants attention away from the stimuli and/ or visual task. However, less conflicted results may be obtained in such studies if a paradigm was used that involved having observers direct their attention toward a given task. This is because directing attention away from, versus focusing attention towards, a given task may produce a different pattern of results if the attentional system functions non-linearly. Consider, for example, a set of hypothetical systems, with each system composed of three parts, A, B, and C, such that parts A and B have independent connections to part C (see Figure 1). In this example, activated components are depicted by capital letters; wherein component A (or a) represents activation due to stimulation, component B (or b) represents activation due to attention, and component C (or c) represents activation of an undifferentiated subsystem. Furthermore, assume that part C is dependant upon the dual activation of parts A and B. Therefore, activating part A or B by itself would yield a lesser system response than activating parts A and B together (and thus activating part C), where response magnitude is represented by the pooled area of each component. Thus, if two such hypothetical systems differ regarding the potential magnitude of the activation of component C, then this difference could not be determined when components A and B are not activated at the same time. Now, in the context of a diverted attention paradigm, 12

14 restricting such a resource to only partial activation may obscure the potential of the system to yield a full and differential response that may be obtained when all the parts are engaged. Thus, observing a visual stimulus with attention diverted would not evoke a differential response across conditions in a collaborative system if it were organized analogously with our hypothetical example. However, coupling a visual stimulus with attention directed would in turn maximally active the collaborative system, and thus evoke a differential response across conditions. Thus, another possible way to examine the differential effects of top-down influences on motion-processing levels would be to use an action planning (Prinz, 1997) paradigm, where attention is not diverted or nulled, but rather directed towards a visualprocessing task. Action planning refers to the idea that motor plans can influence visual perceptions (Prinz, 1997; Wohlschläger, 2000, 2001). For example, Hecht, Vogt, Prinz (2001) found that when subjects learned velocity discriminations using a hand lever (i.e. out of visual range), this learning transferred to subsequent visual discrimination tasks. This effect occurred presumably because visual and motor codes share a common neural representation in the cortex (Prinz, 1997). The purpose of the present study is to investigate whether executing or holding an action plan in memory differentially affects correspondence matching in either cyclopeandefined or luminance-defined apparent motion (such an effect is termed action capture). In doing so, we will attempt to provide evidence that will bear on the issue of whether the two types of apparent motion are differentially affected by top-down cognitive processing. Thus, if action planning affects cyclopean-motion processing to a greater extent than it 13

15 effects luminance-motion processing, then that will suggest that the former is more susceptible to higher level cognitive influences than the latter. This, in turn, would suggest that cyclopean motion is processed at a functionally later stage of the motion stream than luminance motion. Alternatively, if action planning affects cyclopean-motion processing to the same extent as it effects luminance-motion processing, then that will suggest that the former is no more susceptible to cognitive influences than the latter. This, in turn, would suggest that the two types of motion are processed at functionally equivalent stages of the motion streamiii. The approach taken will utilize cyclopean-defined or luminance-defined apparentmotion stimuli within a motion competition (i.e. multistable) arrangement. Apparent motion refers to the perception of motion derived from the viewing of a series of stationary stimuli exposed briefly in succession. One necessary requirement for apparent motion to be perceived is that the visual system must establish perceptual correspondence between individual stimuli across different positions and times in order for a unitary perception of motion to be perceived (Dawson, 1991). Such correspondence relies fundamentally upon a particular combination of spatial and temporal proximity (Korte, 1915). Recent explanations of the motion correspondence process utilized a motion energy framework (see Watson & Ahumada, 1983; van Santen & Sperling, 1984; Adelson & Bergan, 1985). Motion energy refers to the output from pairs of Reichardt-type operators in quadraturephase relationship, which, when squared and summed together, generate a directional motion signal. With respect to the correspondence issue, it is believed that perceptual correspondence is established in ways that maximize the motion energy computation. 14

16 Now consider the problem presented to the visual system when stimuli are arranged in various multistable configurations. In these types of configurations, multiple stimuli are typically presented on any given frame of the motion sequence. This type of motion sequence creates the possibility for multiple paths of apparent motion to be perceived, thus rendering the correspondence between stimuli ambiguous (see Figure 2). Thus, when two or more motion pathways are equally likely, the situation is said to be multistable (Attneave, 1971). For multistable configurations, additional stimulus properties may be used by the visual system to solve the correspondence problem, such as stimulus luminance, depth, texture and occlusion (Anstis & Mather, 1985; Ramachandran & Anstis, 1986). Moreover, it has been shown that relatively higher-order manipulations like attention, action planning (Wohlschläger, 2000), and bodily movements (Ishimura, 1995; Ishimura & Shimojo, 1994), can be used also to solve the correspondence problem. For example, vertical and horizontal hand movements, whether planned or executed, can be used to influence the perceived direction of motion in a circular multistable apparent motion display (Wohlschläger, 2000). Thus, in the present study we utilized an action-planning paradigm to differentiate levels of motion processing, where the action planning involved hand movements. This paradigm involved drawing comparisons across three types of strategy, namely executed movements, planned movements or non-movementsiv. We hypothesized that hand movements, whether executed or planned, would influence motion correspondence in an apparent motion display. Furthermore, we hypothesized that this strategy of action planning should differentially affect cyclopean motion more than luminance motion if 15

17 cyclopean motion perception is mediated by a relatively higher stage of processing. This idea is based on the assumption that a higher-level substrate will have greater access to other processing streams, in particular those involved in motor planning. 16

18 CHAPTER TWO GENERAL METHODS Stimuli The visual stimuli consisted of three luminance-defined or cyclopean defined discs, which are labeled as A, B, and C (diameter of each disc = 2.5 deg). These discs appeared grouped in the upper left corner of the display screen (see Figure 2). Two discs (A & B) appeared located on an imaginary horizontal line across the top of the display. Of these two discs, the leftmost disc (A) had a fixed position while the rightmost disc (B) had a variable position. The third disc (C) appeared slightly below the uppermost left disc at a fixed distance. When the stimuli were presented with an oscillation rate of 2 Hz, horizontal or vertical apparent motion could be perceived (see Figure 2). Apparatus The cyclopean discs were created with a dynamic random-dot stereogram generation system (Shetty, Brodersen & Fox, 1979). The display device was a 19-inch Sharp color monitor (refresh rate = 60 Hz; overall display luminance with 50% dot density = 25.2 cd/m2) upon which matrices of red and green random dots were displayed (approximately 5,000 dots per matrix). At a viewing distance of 150 cm, the display subtended approximately 14 x 11 deg. The observers wore glasses containing red (Wratten 29) and green (Wratten 58) filters, which segregated the two eyes' information. The mean luminance of the red and green half images through their respective filters was 3-4 cd/m2. 17

19 To display the red and green dot matrices, a stereogram generator, a hard-wired device, controlled the red and green guns of the Sharp monitor. The stereogram generator produced disparity between the red and green dot matrices by laterally shifting a subset of dots in one eye's view and leaving unshifted the corresponding dots in the other eye's view; the shift occurred in integer multiples of the width of each column of dots. The gap created by the shift was filled with randomly positioned dots of the same brightness and density so that no monocular cues were visible. The observer perceives the shifted subset of dots as a cyclopean form standing out in depth in front of the background dots, the later of which were seen in the plane of the display screen. The dots were replaced dynamically at a rate of 60 Hz, which allowed the cyclopean stimuli to be moved without the introduction of monocular cues. Signals from black and white video cameras provided input to the stereogram generator, which determined where disparity was inserted into the stereogram. The cameras scanned the oscillating discs displayed on an 18-inch LCD monitor (ViewSonic, model VS100049), which was controlled by a PCV. For every place the camera encountered a white disc the camera signaled the stereogram generator to introduce disparity at a corresponding place in the stereogram. General Procedure At the beginning of each trial, the participant viewed the visual display upon which the three discs were shown in an apparent motion sequence. To create this apparent motion sequence, disc A was presented on frame 1, and discs B and C were presented on frame 2. Frames 1 and 2 oscillated for three cycles, at which time disc B changed position 18

20 relative to disc A, with disc B being displaced farther from, or closer to, disc A. The distance between disc A and disc B either increased or decreased depending upon whether the trial was an ascending-limit trial or a descending-limit trial, respectively (see Figure 3). In doing so, disc B sequentially moved through 18 adjacent disc positions (separation between disc centers = 1.75 deg). For each condition for each observer, six ascending limit trials and six descending limit trials were collected for each condition. The twelve trials were averaged together for each condition in order to provide an estimate of an observers performance. To determine the displacement limit of bistability for the ascending sequence, the initial horizontal separation between discs A and B was smaller than the vertical separation between discs A and C, which created a bias for perceiving horizontal motion. For the ascending sequence, the distance between discs A and B increased. As the separation between discs A and B increased, at some point a bias for perceiving vertical motion was created. Thus, as disc B reached the limit of bistability, the perception of vertical motion occurred. The position of disc B at the point in the sequence where perception changed was called an ascending limit. To determine the displacement limit of bistability for the descending sequence, the initial horizontal separation between discs A and B was larger than the vertical separation between discs A and C, which created a bias for perceiving vertical motion. For the descending sequence, the distance between discs A and B decreased. As the separation between discs A and B decreased, at some point a bias for perceiving horizontal motion was created. Thus, as disc B reached the limit of bistability, the perception of horizontal 19

21 motion occurred. The position of disc B at the point in the sequence where perception changed was called a descending limit. Therefore, for each trial, the participant was asked to verbally report the point at which the directional axis of apparent motion switched to the orthogonal perception (i.e. when spatial distances were sufficient for creating a bistable display). Furthermore, the participants executed hand movements (Experiments 1 & 2) or planned to execute hand movements (Experiment 3), or performed no movement, depending upon condition. When horizontal and vertical movements were performed, the angular extent of the movements was on the order of degrees in angle (i.e. approximately 1-foot in arc-length), and occurred at the same temporal rate as the apparent motion display (i.e. 2 Hz). All hand movements were made under the edge of a table so that they were out of sight. Executed hand movements commenced 10 seconds prior to each trial, and continued during each trial until the participant indicated a perceived shift of the direction of apparent motion. For planned hand movements the directional axis was given to each participant 10 seconds prior to the start of each trial, but the actual movement was not executed during the trial; rather the hand movement was executed for a duration of 10 seconds directly after the participant indicated a perceived shift of motion direction and the trial had ended. A 10-minute training session was conducted before data collection, which allowed the participant to learn the required hand movements to be made in conjunction with the apparent motion display. In addition, an imagined strategy group was added to the design, which had observers imagine vertical or horizontal motions of a moving disc. This was done in order 20

22 to control for other cognitive (non-motor related) processes that might also affect motion perception. To induce imagination, a double-sided arrow appeared 10 seconds prior to the start of each trial and remained exposed for the duration of the trial in order to cue the individual for the imagined motion taskvi. 21

23 CHAPTER THREE EXPERIMENT 1 In any study of action capture involving apparent motion displays, the dependant variable is necessarily subjective (i.e. in what direction is the apparent motion perceived?). This, in turn, raises the important issue of demand characteristics: is the observer actually reporting what he or she perceives or what he or she assumes the experimenter wants them to perceive? Therefore, the purpose of Experiment 1 was to determine to what degree demand characteristics play a role in the action-capture paradigm. To do so, two groups of observers were given different experimental instructions. One group, the congruent-instruction condition, was told that horizontal-hand movements lead to horizontal-motion perception, whereas vertical-hand movements led to verticalmotion perception. The other group, the incongruent-instruction condition, was told that horizontal-hand movements led to vertical-motion perception, whereas vertical-hand movements led to horizontal-motion perception. This created a 2 (congruent versus incongruent instructions) by 3 (vertical versus horizontal versus no hand movements) by 2 (imagined versus executed strategies) design. All trials were conducted with a luminancedefined stimulus. Participants Twenty observers (12 females and 8 males) participated (two other individuals were eliminated from participation due to a failed vision test or an inability to understand the experimental instructions because English was not their native language). The twenty 22

24 participants possessed normal or corrected-to-normal acuity, phoria and binocular vision (as tested with a Bausch & Lomb Ortho-rater). Results For each participant, the spatial displacement limits obtained across the 12 trials were averaged together to provide an estimate of displacement limit for each condition. These estimates were then averaged together across observers to provide group means, which are depicted in Figure 4. Figure 4 shows the displacement limits (ordinate) for the three hand movement axes (abscissa) and for the imagined versus executed hand movement strategies (legend), for congruent (top panel) and incongruent (bottom panel) instructions. It can be seen that the participants executed hand movements influenced their displacement limit for perceiving horizontal and vertical motion. When executing horizontal hand movements, observers perceived more horizontal motion (i.e. greater displacement limit), and when executing vertical hand movements they perceived more vertical motion (i.e. lesser displacement limit). However, the figure also reveals that simply imagining the movement did not affect the displacement limit. This pattern of results was the same for congruent versus incongruent instructions; horizontal hand movements lead to a greater frequency of horizontal perceptions of apparent motion, while vertical hand movements lead to a greater frequency of perceived vertical motion, regardless of what the subject was led to believe. The data shown in Figure 4 were analyzed by computing a 2 x 3 x 2 mixed design analysis of variance (ANOVA). This analysis showed that there was a main effect of hand movement axis, F(2, 36) = , p < 0.001, partial η2 = This analysis also 23

25 revealed that there was a significant interaction of hand movement axis and strategy F(2, 36) = , p < 0.001, partial η2 = Tests of simple main effects revealed that the main effect of hand movement axis existed at the executed level F(2, 36) = , p < 0.001, partial η2 = 0.697, but not at the imagined level F(2, 36) = 0.452, p > 0.05, of the strategy variable. Furthermore, under the executed level, Tukey s HSD post-hoc test showed that displacement limits with horizontal hand movements were significantly different from displacement limits with vertical hand movements and with no-movements (both p < 0.05); also displacement limits with vertical hand movements were significantly different from displacement limits with no movement (p < 0.05). The ANOVA revealed two non-significant main effects, which were strategy (F(1, 18) = 1.088, p > 0.05), and congruent/ incongruent instructions (F(1, 18) = 1.431, p > 0.05). The ANOVA also revealed that the factors of hand movement axis and congruent/ incongruent instructions did not interact (F(2, 36) = 0.851, p > 0.05), nor did the factors of strategy and congruent/ incongruent instructions (F(1, 18) = 0.903, p > 0.05), nor hand movement axis, strategy, and congruent/ incongruent instructions (F(2, 36) = 2.288, p > 0.05). Summary Experiment 1 showed that we could reliably demonstrate action capture of a luminance-defined apparent motion stimulus, and that this effect depended upon executing hand movements yet it was unrelated to demand characteristics. Merely imagining motion occurring along a particular axis did not have an effect. 24

26 In the subsequent experiments, we turn to the issue of action capture of cyclopean motion perception. 25

27 EXPERIMENT 2 The purpose of Experiment 2 was to determine whether executed hand movements would capture the perception of cyclopean motion as it did for luminance motion (Experiment 1). To do so, two types of stimuli, luminance and cyclopean, were factorially combined with congruent versus incongruent instructions, vertical versus horizontal versus no hand movements, and imagined versus executed strategies, to create a 2 x 3 x 2 x 2 design. Participants Twenty observers (12 females and 8 males) participated (three other individuals were eliminated from participation due to failed vision tests). The twenty participants possessed normal or corrected-to-normal acuity, phoria and binocular vision (as tested with a Bausch & Lomb Ortho-rater). Results For each participant, the spatial displacement limits obtained across the 12 trials were averaged together to provide an estimate of displacement limit for each condition. These estimates were averaged together across observers to provide group means, which are depicted in Figure 5. Figure 5 shows the displacement limits (ordinate) for the three hand movement axes (abscissa), for cyclopean versus luminance stimuli (legend), and with executed (top panel) versus imagined (bottom panel) strategies. It can be seen that the participants executed hand movements influenced their displacement limit for perceiving horizontal and vertical motion. When executing horizontal hand movements, observers perceived 26

28 more horizontal motion (i.e. greater displacement limit), and when executing vertical hand movements they perceived more vertical motion (i.e. lesser displacement limit), a pattern that occurred for both the cyclopean and luminance stimuli (although there was a trend for the cyclopean stimuli to produce slightly more horizontal motion). Again, the figure also reveals that simply imagining the movement did not affect the displacement limit. Finally, the pattern of results was the same for congruent versus incongruent instructions (figure not shown). The data shown in Figure 5 were analyzed by computing a 2 x 3 x 2 x 2 mixed design ANOVA. This analysis showed that there was a main effect of hand movement axis F(2, 36) = , p < 0.001, partial η2 = 0.869, a main effect of strategy F(1, 18) = , p < 0.001, partial η2 = 0.669, and a main effect of stimulus domain, F(1, 18) = 5.193, p < 0.04, partial η2 = This analysis also revealed that there was a significant interaction of hand movement axis and strategy F(1.387, ) = , p < 0.001, partial η2 = (here, the Greenhouse-Geisser correction was used due to a significant test of sphericity: W (2) = 0.558, p < 0.01). Tests of simple main effects revealed that the main effect of hand movement axis existed at the executed level F(2, 36) = , p < 0.001, partial η2 = 0.878, but not at the imagined level F(2, 36) = 1.285, p > 0.05, of the strategy variable. Furthermore, at the executed level, Tukey s HSD post-hoc test showed that displacement limits with horizontal hand movements were significantly different from displacement limits with vertical hand movements and with no movement (both p < 0.05); also displacement limits with vertical hand movements were significantly different from displacement limits with no hand movements (p < 0.05). 27

29 The ANOVA revealed one non-significant main effect, which was congruent versus incongruent instructions (F(1, 18) = 0.213, p > 0.05). The ANOVA also revealed that the factors of hand movement axis and congruent/ incongruent instructions did not interact (F(2, 36) = 0.880, p > 0.05), nor did strategy and congruent/ incongruent instructions (F(1, 18) = 0.842, p > 0.05), stimulus domain and congruent/ incongruent instructions (F(1, 18) = 0.603, p > 0.05), hand axis and domain (F(2, 36) = 1.363, p > 0.05), nor strategy and stimulus domain (F(1, 18) = 0.500, p > 0.05). The ANOVA also revealed that the factors of hand movement axis, strategy, and congruent/ incongruent instructions did not interact (F(2, 36) = 0.095, p > 0.05), nor did the factors of hand movement axis, stimulus domain, and congruent/ incongruent instructions (F(2, 36) = 0.309, p > 0.05); strategy, stimulus domain, and congruent/ incongruent instructions (F(1, 18) = 1.385, p > 0.05); nor hand movement axis, strategy, and stimulus domain (F(2, 36) = 1.341, p > 0.05). Finally, the ANOVA revealed that the factors of hand movement axis, strategy, stimulus domain, and congruent/ incongruent instructions did not interact (F(2, 36) = 1.604, p > 0.05). Summary Experiment 2 showed that we could reliably demonstrate action capture of both cyclopean and luminance motion, and that the effect was again unrelated to demand characteristics. Perceptions of both types of motion were significantly influenced by executed hand movements but not by merely imagining the movement. 28

30 EXPERIMENT 3 Experiment 2 showed that executing hand movements can influence the perception of cyclopean and luminance motion. Although interesting, a much larger question can be posed: what affect, if any, would the intention, or planning, to execute hand movements have on the perception of motion? This raises the issue of whether intention, goal-directed planning and decision-making can influence motion perception without the actual execution of a movement, an issue that is directly relevant to the question of whether cyclopean motion processing is uniquely influenced by high-level cognition. If it was, then it would be expected that intention and goal-directed planning could affect cyclopean motion processing more than luminance motion processing, the latter of which is known to be processed at relatively low levels of the motion stream (e.g. Cavanagh & Mather, 1989). The purpose of Experiment 3 was to determine whether planning to execute hand movements would capture the perception of cyclopean motion or that of luminance motion. As with the previous experiment, two types of stimuli, cyclopean and luminance, were factorially combined with congruent versus incongruent instructions, vertical versus horizontal versus no hand movements, and imagined versus planned strategies, to create a 2 x 3 x 2 x 2 design. In the present experiment, participants held an action plan in memory but did not execute any hand movements prior to making their verbal report on their perception of motion. Specifically, the participant was told that they would be making a hand movement once they had already given their verbal response indicating their motion perception, so that an actual action plan would be held in memory. Once the verbal response was given, 29

31 the observer then made the required hand movements for 10 seconds. This was done in order to ensure that the participant was keeping a valid plan for movement in memory. Participants Twenty participants (11 females and 9 males) served (two other individuals were eliminated from participation due to failed vision tests). The twenty participants possessed normal or corrected-to-normal acuity, phoria and binocular vision (as tested with a Bausch & Lomb Ortho-rater). Results For each participant, the spatial displacement limits obtained across the 12 trials were averaged together to provide an estimate of displacement for each condition. These estimates were then averaged together across observers to provide group means, which are depicted in Figure 6. Figure 6 shows the displacement limit (ordinate) for three hand movement axes (abscissa) and for cyclopean versus luminance motion (legend), with planned (top panel) versus imagined (bottom panel) strategies. It can be seen that the participants planned hand movements influenced their limits for perceiving horizontal and vertical motion; however, this effect was not as robust as the effect of executed movements in the previous two experiments. Nonetheless, when planning horizontal hand movements, observers perceived more horizontal motion (i.e. greater displacement limit), and when planning vertical hand movements they perceived more vertical motion (i.e. lesser displacement limit). However, the figure also reveals that simply imagining motion did not affect the 30

32 displacement limit. Again, the pattern of results was the same for congruent versus incongruent instructions (figure not shown). The data shown in Figure 6 were analyzed by computing a 2 x 3 x 2 x 2 mixed design ANOVA. This analysis showed that there was a main effect of hand movement axis F(2, 36) = , p < 0.001, partial η2 = 0.425, and a main effect of strategy, F(1, 18) = , p < 0.002, partial η2 = This analysis also revealed that there was a significant interaction of hand movement axis and strategy F(2, 36) = , p < 0.001, partial η2 = Tests of simple main effects revealed that the main effect of hand movement axis existed at the planned level F(2, 36) = , p < 0.001, partial η2 = 0.539, but not at the imagined level F(2, 36) = 2.373, p > 0.05, of the strategy variable. Furthermore, at the planned level, Tukey s HSD post-hoc tests showed that the displacement limit with horizontal hand movements was significantly different from the displacement limit with vertical hand movements and the displacement limit with no movement (both p < 0.05); however, the displacement limit with vertical hand movements was not significantly different from the displacement limit with no movement (p > 0.05). The ANOVA also revealed the existence of two non-significant main effects, which were: congruent/ incongruent instructions (F(1, 18) = 2.378, p > 0.05), and stimulus domain (F(1, 18) = 3.460, p > 0.05). The ANOVA also revealed that the factors of hand movement axis and congruent/ incongruent instructions did not interact (F(2, 36) = 1.621, p > 0.05), nor did strategy and congruent/ incongruent instructions (F(1, 18) = 0.832, p > 0.05), stimulus domain and congruent/ incongruent instructions (F(1, 18) = 0.165, p > 0.05), hand movement axis and stimulus domain (F(2, 36) = 0.127, p > 0.05), nor did 31

33 strategy and stimulus domain (F(1, 18) = 1.032, p > 0.05). The ANOVA also revealed that the factors of hand movement axis, strategy, and congruent/ incongruent instructions did not interact (F(2, 36) = 1.884, p > 0.05), nor did hand movement axis, stimulus domain, and congruent/ incongruent instructions (F(2, 36) = 0.066, p > 0.05), strategy, stimulus domain, and congruent/ incongruent instructions (F(1, 18) = 1.064, p > 0.05), hand movement axis, strategy, and stimulus domain (F(2, 36) = 0.165, p > 0.05), nor hand movement axis, strategy, stimulus domain, and congruent/ incongruent instructions (F(2, 36) = 1.382, p > 0.05). Summary Experiment 3 showed that action capture could result from the mere planning of hand movements, an effect that occurred equally for luminance and cyclopean motion stimuli and which was not related to demand characteristics. The participants motion perception was significantly influenced by the planning of goal-directed movements, not just by their execution. Merely imagining motion in a particular direction did not have the same effect as planned movements, which suggests that planning and imagination are different cognitive processes. 32

34 CHAPTER FOUR GENERAL DISCUSSION The principle results of this study show that action capture occurs only when movements are executed or planned, but not when motion is imagined. Moreover, the action capture phenomenon occurred equally for both the cyclopean and luminance domains. Thus, when an individual planned or executed vertical hand movements, his or her perception of cyclopean or luminance apparent motion was biased for vertical motion, and the same was true for horizontal hand movements and horizontal motion. This pattern was the same when subjects received either congruent or incongruent instructions, thus ruling out demand characteristics as playing a significant role in the phenomenon. Although the execution of hand movements also involves their planning, and thus it might seem reasonable to interpret the action capture phenomenon as being due to movement planning only, we note that the action capture effect was weaker in Experiment 3 that involved only planning than in Experiments 1 and 2 that involved execution and planning. The stronger effect produced by both planning and execution suggests that movement execution also plays a role in the phenomenon. Thus, we interpret the results across all three experiments as indicating that both planning and execution of movements play a role in action capture. Hommel, Müsseler, Aschersleben, and Prinz (2001) have explained the planning or execution of bodily movements by evoking the concept of an event-code. According to Hommel and colleagues, motor events, like planning a hand movement, require the utilization of event codes that are bound to a particular action. Event codes are thought to 33

35 represent characteristics or components of the action, like movement direction, speed and location. Moreover, Hommel and colleagues posit that visual perception, like perceiving the movement of an object, requires the utilization of object codes that are bound to particular objects. Object codes are thought to represent the characteristics or components of a given perception, like direction, speed, and location. Hommel and colleagues further argue that event codes and object codes can be linked in a perceptual-motor synergism. For example, the perception of the direction of a moving object can involve a code that is analogous to that of a planned bodily movement. Thus, in this example, direction would be a common component of both an event code for action and an object code for perception. This commonality between the codes for direction would enable the establishment of a functional coupling between action and perception, whereby the perception of a particular motion direction would have the ability to prime muscular movements executed in the same direction (Hommel, Müsseler, Aschersleben, & Prinz, 2001). Moreover, Hommel and colleagues have proposed that muscular movements and their plans also have the ability to prime perception in an analogous fashion. These situations have been termed perception-to-action transfer, and action-to-perception transfer, respectively (Prinz, 1997). This coupling between action and perception can be experimentally demonstrated by creating situations in which the two entities share the same codes which produces perceptual priming by action (e.g. Wohlschläger, 2001; and this study) or motor priming by perception (e.g. Kornblum & Lee, 1995). 34

36 Although interesting and conceptually useful, the concept of event codes and object codes by Hommel and colleagues seems vague in the way that it is expressed by these authors. The issue, then, is to retain the concept of bound codes (between perception and action), but placing that concept in a physiological framework. To do so, we evoke the well-known concept of distributed channels (e.g. Blake & Sekuler, 2005; Grunewald & Lankeet, 1996). First consider the idea that various sensory dimensions are each processed by a collection of specialized sensory channels, with each channel devoted to signaling the presence of a subrange of values along a given dimension. Next consider such a system of distributed channels for motion direction, where a channel represents a collection of cells that are activated by the same subrange of motion directions. For example, Figure 7 (top panel) depicts the activity profile of three hypothetical directional channels, with one channel maximally responsive to motion in the 0º direction (i.e. upward), a second channel maximally responsive to motion in the 45º direction (i.e. diagonally upward and to the right), and a third channel maximally responsive to motion in the 90º direction (i.e. rightward). (Note that an actual network for representing motion direction likely contains many more channels which cover all directions of motion; see Nawrot & Sekular, 1990.) Thus, each of these three channels responds maximally to a particular direction of motion, with the adjacent channels activated to a lesser degree to that particular motion direction. Now, a stimulus moving in the direction labeled X in Figure 7 would activate the most the directional channel selective for motion in the direction of 0º, followed by the channel selective for motion in the direction of 45º. A stimulus moving in the direction of 35

37 X would active the least the channel selective for motion in the direction of 90º. The activation across all channels would represent a population code for the perception of motion direction. Thus, for a stimulus moving in the direction labeled X, the population code of responding would be: 0º cells >> 45º cells > 90º cells, which would represent motion in the direction of X. Evidence exists that suggests that this kind of neural code represents directional coding for perceiving cyclopean motion (e.g. Phinney, Bowd, & Patterson, 1997) as well as for executing actions (e.g. Georgopoulos, Kalaka, Caminiti, & Massey, 1982). Consider now that the motor system for action, shown in the bottom panel of Figure 7, is similarly organized into a network of three hypothetical channels, 0º, 45º, and 90º, where each channel represents a pool of motor neurons that initiate an action (Gazzaniga, Ivry, & Mangun, 2002). Thus, a movement in the direction labeled Y in the bottom panel of Figure 7 would activate the most the pool of motor neurons that selectively move a limb in the direction of 0º, followed by the pool of motor neurons that selectively move a limb in the direction of 45º. A movement in the direction labeled Y would active the least the pool of motor neurons that selectively move a limb in the direction of 90º. Here, the population code of activation for making a movement in the direction labeled Y would be: 0º cells >> 45º cells > 90º cells, which would represent an executed or planned movement in the direction of Y. In the present study, the tendency for action to influence perception may be the result of linked population codes across the perceptual-motor substrate. Thus, in Figure 7 we assume that direction X (top panel) is functionally linked to direction Y (bottom panel), 36