Oxford Compendium of Visual Illusions Arthur Shapiro and Dejan Todorovic, Editors Chapter xx Illusory Color Spread from Apparent Motion Carol M. Cicerone 1, Professor Emeritus, and Donald D. Hoffman, Professor Department of Cognitive Sciences University of California, Irvine Irvine, CA 92697 1 Corresponding author s address: 2931 University Terrace, NW Washington, DC 20016 USA carolcicerone@nas.edu
Abstract and Keywords Color from motion describes the perception of subjective color that spreads over physically achromatic regions that are seen in apparent motion. Multiple frames are shown in quick succession, each frame composed of a random placement of differently colored dots on an achromatic background. From frame to frame, the locations of all dots are fixed, whereas the color assignments of dots in the test region change. Subjective color can be measured by color matches to and cancellation by real lights; can be seen with chromaticity differences alone in test and surround dots; and is independent of contour formation. In stereoscopic view, the perception of depth, as well as color and form, can be recovered in tandem with seeing motion. We suggest that in natural scenes, mechanisms triggered by motion may reconstruct the depth, color, and form of partially obscured objects so that they can be seen as if in plain view. Keywords: color from motion, subjective color, apparent motion, depth perception, camouflage xx.1 Introduction It is known that the visual system is capable of constructing illusory colors and contours that may be absent in the physical stimulus (e.g., Grossberg, 1994; Kanisza, 1979; Michotte, Thines, & Crabbe, 1964; Nakayama & Shimojo, 1990, 1992; Nakayama, Shimojo, & Ramachandran, 1990; Peterhans & von der Heydt, 1991; van Tuijl, 1975; Varin, 1971; Yamada. Fujita, & Masuda, 1993). In particular, motion is effective in allowing the visual system to use multiple fragmented views of an object over time to reconstruct its shape as a whole (e.g., Anderson & Braunstein, 1983; Andersen & Cortese, 1989; Gibson, 1979; Kaplan, 1969; Lappin, Doner, & Kottas, 1980; Shipley & Kellman, 1983, 1984; Stappers, 1989; Wallach & O Connell, 1953; Wertheimer, 1923; Yonas, Craton, & Thompson, 1987). We introduced (Cicerone & Hoffman, 1992) an effect 2
called color from motion for which, the perception of apparent motion is accompanied by the perception of illusory color that is seen in physically achromatic regions of the stimulus. We (Cicerone, Hoffman, Gowdy, & Kim. 1995; Cicerone & Hoffman, 1997; Miyahara & Cicerone, 1997; Chen & Cicerone, 2002a, b) explored this phenomenon to define the conditions under which it occurs, to understand how it might be useful to organize the visual scene, and to link it to the perception of motion, contour, color, and depth. A typical display of color from motion is shown in Figure 1. Each frame (8 deg of visual angle on a side) consisted of an achromatic background field (CIE x = 0.276, y = 0.286, luminance 73 cd/m 2 ) over which was randomly arrayed 800-1200 dots (each 3.5 min of arc in diameter). Within a circular test region (typically 1 to 2 deg of visual angle in diameter), the dots were colored green (CIE x = 0.280, y = 0.610). All other dots were red (CIE x = 0.621, y = 0.344). To create successive frames, no dots changed their locations within the frame; only the color assignments of the dots were changed, according to a uniform vertical displacement (0.12 deg of visual angle) of the test region in successive frames. When frames are cycled, typically with an effective displacement rate of the test region equivalent to 7 deg/sec, up and down over a vertical region spanning 5 deg of visual angle, an illusory green disk moving up and down pops into view and a spread of illusory green color is seen in the physically achromatic test region. This effect and others described below can be viewed in an on-line publication of the Journal of Vision (Chen & Cicerone, 2002a). < Figure 1 here > 3
The rating methods used in the early studies (Cicerone & Hoffman, 1992; Cicerone et al., 1995) established the salience of the color spreading effect and its link to the perception of apparent motion. Certain aspects of the illusory color, however, such as its saturation, require more sensitive methodologies. Miyahara & Cicerone (1997) used a side-by-side matching method and found that the hue of the subjective color spread approximates that of the test dots. In their methodology, the matching stimulus was stationary and homogeneously colored, whereas the color from motion stimulus was perceived as moving and included test and surround dots. Chen & Cicerone (2002a) used real lights to cancel the subjective color spread in color from motion, while keeping luminance constant, to measure the hue of the illusory color spread. An increase in the luminance of the test dots produces an increase in the saturation of the physical lights required to match the subjective color spread as measured by the side-by-side method (Miyahara & Cicerone, 1997) and the cancellation method (Chen & Cicerone, 2002a). xx.2 Perception of motion is essential in color from motion Illusory color spread as measured by a rating method is linked to apparent motion of the test region. Therefore, we asked if the salience of the color spread as measured with cancellation was linked to the salience of the apparent motion (Chen & Cicerone, 2002a). We varied the effective rate of vertical translation of the test region between zero and 12 deg of visual angle per sec. The results for two observers (Figure 2) can be well described by two linear functions with a steeply sloping first branch (from zero to 1 deg/sec) and a second branch with a much reduced, near zero slope (from 1 deg/sec to 12 deg/sec). The general profile of the results suggests an allor-none relationship between apparent motion and illusory color spread. Observers report that as the translation speed of the test region increases, the perception of the illusory, green-colored 4
disk as a separate form that moves over the field of dots is enhanced. This perception of separation is reported reliably for speeds exceeding 1 deg/sec. Concurrently, the test dots appear to assume the color of the surround dots so that all dots appear to have the same color, in this case red, even if the test dots are physically colored green. Thus, it appears that for speeds of effective translation of the test region that are greater than 1 deg/sec there is little or no enhancement of the illusory color spread whereas the salience of the perceived separation of the figure defined by the illusory color spread from the array of dots is enhanced. <Figure 2 here> xx.3 Color from motion without contour formation Is color from motion linked to contour formation as well as to the perception of motion? Luminance contrast is known to be necessary for the formation of static subjective contours (e.g., Frisby & Clatworthy), for the perception of apparent motion in achromatic stimuli (e.g., Ramachandran & Gregory, 1978; Cavanagh, Boegelin, & Favreau, 1985), and for the perception of achromatic neon spreading (e.g., Bressan, 1983). If, near equiluminance, color spread occurs without the formation of a subjective contour, then color from motion is likely to be regulated by mechanisms distinct from those regulating contour formation. Miyahara and Cicerone (1997) tested this idea with green (test) and red (surround) dots that were matched in luminance as determined for each observer individually by means of heterochromatic flicker photometry. Chromaticity differences between the test dots and the surround dots in the absence of luminance differences produced the perception of spread of color from motion. As the luminances of the test and surround dots approach equality, the strength of the subjective contour surrounding the 5
test region is reduced while the subjective color spread is still perceived. Observers reported that in such equiluminant conditions, there is no clear contour bounding the region of the subjective color spread and that apparent motion is not smooth and slower than the conditions in which the test and surround dots differ in luminance (Chen & Cicerone, 2002a). These results are consistent with findings that suggest that the neural mechanisms responsible for contour formation rely on luminance information (e.g., Kanizsa, 1979; Marr, 1982; von der Heydt et al., 1984). These findings support the idea that the mechanisms regulating color from motion are separable from those regulating contour formation and that color spread can occur without contour formation. Another way to study the role of luminance differences is to use red-green dichromatic observers who are incapable of making red-green discriminations on the basis of chromaticity discriminations alone. Miyahara and Cicerone (1997) presented these stimuli to a deuteranope. The deuteranope lacks the middle-wavelength-sensitive pigment and is therefore insensitive to chromaticity differences in the middle- to long-wavelength range of the visible spectrum. In this range, where our red and green stimuli lie, the deuteranope sees only luminosity differences, based on the activity of the long-wavelength-sensitive cones. Thus, the results of a red-green dichromat should allow us to assess the impact of luminosity differences alone for these stimuli. When the luminosity of the green dots in the test region was high relative to that of the dots in the surround, the deuteranope saw a bright disk moving over the test region. As expected, near equiluminance between the test and surround dots, the deuteranope in our study saw no apparent motion nor did he see brightness spread in the test region. This is in contrast to color normal 6
observers who saw apparent motion and color spread without a clearly defined contour when test and surround dots were matched in luminance. xx.4 Modal versus amodal completion in color from motion There are two modes in which color from motion is perceived, either (1) as a localized change of illumination, a colored spotlight or shadow, moving over a textured surface (Cicerone & Hoffman, 1992; Cicerone et al., 1995; Miyahara & Cicerone, 1997) or (2) as a moving, colored object seen through apertures in an occluding surface (Cicerone & Hoffman, 1997; Chen & Cicerone, 2002a). The mode in which color from motion is seen depends on figural cues and regional differences in the luminance contrast between the chromatic elements and the achromatic background. Regions with figural cues tend to be seen as moving. These regions are seen in the first mode (modal completion, Figure 3, left) if their defining figural elements, dots in this case, are of lower luminance as compared to the background and in the second mode (amodal completion, Figure 3, right) if the defining figural elements are of higher luminance as compared to the background. As compared with color from motion seen in modal completion, the perceived color in amodal completion is markedly higher in saturation and the organization of the scene is different in that objects are perceived to lie behind a partially occluding screen. Nonetheless, in both cases, the perception of motion is linked to a spread of color over regions defined by motion. Hence, the mechanisms driving the color spread, whether a desaturated veiling color (modal completion) appearing to glide over the display or a highly saturated disk (amodal completion) moving behind a partially-occluding screen, are likely to be the same. <Figure 3 here> 7
xx.5 Luminance relationships shape visual scene organization in color from motion In still view of our standard stimulus, the small test region of green dots is clearly seen as the figure, and the surrounding region of red dots is seen as the ground. Regardless of the luminance of the achromatic background, the test region is seen to move and color spread is linked to this moving region. To explore the importance of figural cues, Chen and Cicerone (2002a) used stimuli composed of alternating bands of equal widths of red and green dots whose luminance contrasts, as compared to the achromatic background, could be manipulated. For such stimuli without clear figure/ground configurations, we asked whether color from motion remains a salient effect and how luminance relationships help to organize the visual scene. Regardless of the luminance of the achromatic background, apparent motion and color spread are associated with regions of lower luminance contrast. A switch in perception of seeing red bands moving or green bands moving occurs at each observer s point of equiluminance between the red and green stimuli. The relative widths of the bands of red and green dots were varied to test whether figure/ground cues could supersede luminance cues. Indeed, for bands that are thin enough (roughly an 8 to 1 ratio for our observers), the narrower bands (more figure-like) are seen as moving regardless of luminance contrast relationships (Chen & Cicerone, 2002a). xx.6 Color from motion is regulated at a point beyond binocular combination The dependence of color spread on the perception of apparent motion in this phenomenon and the spread of color in the absence of contour formation suggest that the locus of the mechanisms underlying color from motion may be at a point beyond binocular combination. Evidence supporting this idea was obtained by the dichoptic presentation of every other frame of the full 8
stimulus sequence to one eye and, out of phase, the alternate frames to the other eye. To overcome binocular rivalry, the eye that did not receive the standard stimulus was presented with a stimulus that was identical in every way, except that the test dots were absent. A compelling perception of color from motion, as measured by a rating method, is seen that is equal to that obtained when the full stimulus sequence is presented to each eye alone (Cicerone & Hoffman, 1997). This is consistent with the regulation of color from motion at sites beyond the convergence of monocular pathways. Perhaps even more persuasively, Chen and Cicerone (2002b) showed that depth, as well as form and color, is recovered from apparent motion. Stimuli were presented dichoptically with left and right eye views identical as to the locations of all dots. Binocular disparity was introduced by means of translations in the color assignments for corresponding image elements in the two eyes. Horizontal crossed or uncrossed disparities of 0.5 deg of visual angle were created by differences in color assignments alone. In random presentation of the crossed or uncrossed horizontal displacements, observers were asked to judge if the illusory figure defined by the color spread was behind or in front of the field of dots. In still view of the stimulus, binocular rivalry occurs and neither apparent motion, nor color spread, nor depth is seen. In this case, observers performed at chance level when required to judge depth. When the frames are cycled at an effective rate of 7 deg/sec over a vertical distance upward and downward spanning 10 deg of visual angle, apparent motion, color spread, and depth are perceived. In random presentations of stimuli for crossed and uncrossed disparities in motion view, observers performed well above chance (95% confidence interval) with some observers performing with 100 per cent accuracy. 9
xx.7 General discussion The distinctive features of color from motion The effect we call color from motion is distinctive in a number of ways. First, neither contour formation nor color spread is seen in still view of our stimuli. In this way it is distinct from static neon color spreading, an effect that is well known, as we have reviewed above. Furthermore, illusory color spread as seen in color from motion is not a general feature in motion stimuli; for example, it is not reported in kinetic occlusion. Second, in color from motion, there are no spatial displacements of the dots; only the color assignments of the dots change from frame to frame. Apparent motion and the attendant illusory color spread are generated only by the change in chromaticity or luminance of the dots. To buttress this second point, we created stimuli in which the test region remains fixed in space and the test dots were set in motion either 1) independently and randomly; 2) in unison along the same trajectory; or 3) in unison along a random trajectory. Naïve observers were tested with all of these stimuli, and none reported seeing color spread (Chen & Cicerone, 2002a). Third, although the saturation of the illusory color spread increases with increases in the luminance of the test dots, the luminance of the dots in the surround region has no impact on the illusory color spread when the chromaticities of the test and surround dots differ (Chen & Cicerone, 2002a). This differentiates color from motion from color contrast, wherein the luminance of surround elements has considerable impact. Fourth, subjective color spread is seen without the perception of a subjective contour near the point of equiluminance between test and surround dots, as long as there is a chromaticity difference between the dots (Miyahara & Cicerone, 1997). This result suggests that the spread of illusory color in color from motion does not require the prior formation of a contour and that color, independent of contour, can be recovered in tandem with seeing motion. 10
When color from motion is seen in modal completion, the low saturation and neon-like quality of the illusory color spread is reminiscent of the quality of the perception in displays of transparency. Is color from motion the same as transparency? We argue that it is not for the following reasons. First, the perception of transparency occurs in displays due to both figural and luminance cues that are already present in the stimulus (e.g., Adelson, 1983; Beck, 1978; da Pos, 1989; D Zmura, Colantani, Knoblauch, & Laget, 1997; Metelli, 1974), whereas in color from motion a new colored surface, with or without a border, is created by the visual system in physically achromatic regions. In other words, when transparency is perceived, physically present but differentiated regions are unified into a single perceptual layer, whereas in color from motion, an entirely new, and illusory layer is constructed by the visual system. Second, motion is not required for transparency to be perceived, whereas color from motion requires the perception of apparent motion and is never seen in still view. Third, color from motion can be seen in amodal completion (Cicerone & Hoffman, 1997; Chen & Cicerone, 2002a), a perception that differs markedly from any of the characteristics of transparency. Our results indicate that color from motion is regulated at a point beyond binocular combination (Cicerone and Hoffman, 1997), that it requires the concurrent, if not prior, perception of motion (Cicerone et al., 1997; Chen & Cicerone, 2002a); that in addition to form and color, depth can be recovered in color from motion (Chen & Cicerone 2002b); and that figure/ground configuration can override luminance relationships as the determinant of which areas appear to move and to be filled with illusory color (Chen & Cicerone, 2002a). Considering our current understanding of 11
visual processing in the primate brain, these findings suggest that the mechanisms supporting the perception of color from motion include neural processing at higher levels. The functional significance of color from motion Can our results be related to the visual system s ability to break visual camouflage? In natural scenes, objects or surfaces may not be perceived because of the color, luminance, or texture of nearby surfaces. We reasoned that if color from motion is a robust camouflage-breaking mechanism, then it should be able to render the test object visible even when color is not an obvious cue. The stimulus was modified so that a proportion (0 to 0.8) of the dots in the surround region were green instead of all red. In still view, the test region with green dots was not reliably seen; thus, in still view, the test region was effectively camouflaged. Nonetheless, when the stimulus sequence was cycled as before and apparent motion was perceived, a moving, illusory green disk was seen (Cicerone & Hoffman, 1997). In other natural scenes, objects may be hidden from full view by occlusion. To mimic this situation, we reduced the illumination of the achromatic background in our stimuli to show that color from motion can be seen in amodal completion as a highly saturated green disk that moves over a highly saturated red background, all seen through random perforations in a dark screen (Cicerone & Hoffman, 1997; Chen & Cicerone, 2002a). The mode in which color from motion is seen depends on figural cues and on regional differences in luminance contrast between the chromatic elements and the achromatic background (Chen & Cicerone 2002a). In still view, the physical representation of the scene may give an equivocal interpretation of objects and surfaces. When the test is seen in apparent motion, subjective color spread helps to 12
reveal the hidden object in modal or in amodal completion. Furthermore, not only form and color, but also depth can be recovered in tandem with seeing motion. We propose that the neural mechanisms that support perceptions in color from motion may be the same as those that work in natural scenes to reveal form, color, and depth to the visual system, even when it is confronted with fragmented physical information, as occurs in camouflage. References Adelson, E. H. (1993). Perceptual organization and the judgment of brightness. Science, 262, 2042-2044. Andersen, G. J., & Braunstein, M. L. (1983). Dynamic occlusion in the perception of rotation in depth. Perception & Psychophysics, 34, 356-362. Andersen, G. J., & Cortese, J. M. (1989). 2-D contour perception resulting from kinetic occlusion. Perception & Psychophysics, 46, 49-55. Beck, J. (1978). Additive and subtractive color mixture in color transparency. Perception & Psychophysics, 23, 265-267. Bressan, P. (1993Revisitation of the luminance conditions for the occurrence of the achromatic neon color spreading illusion. Perception & Psychophysics, 54, 55-64 Cavanagh, P., Boeglin, J., & Favreau, O. E. (1985). Perception of motion in equiluminous kinematograms. Perception, 14, 151-162. Chen, V. J. & Cicerone, C. M. (2002a). Subjective color from apparent motion. Journal of Vision, 2, 424-437. Chen, V. J. & Cicerone, C.M. (2002b). Depth from subjective color and apparent motion. Vision Research, 42, 2131-2135. Chen, V. J., & D Zmura, M. (1998). Test of a convergence model for color transparency perception. Perception, 27, 595-608. Cicerone, C. M., & Hoffman, D. D. (1992). Dynamic neon colors: Perceptual evidence for parallel visual pathways. Advances in Color Vision, Technical Digest, 4, 66-68. Cicerone, C. M., & Hoffman, D. D. (1997). Color from motion: Dichoptic activation and a possible role in breaking camouflage. Perception, 26, 1367-1380. Cicerone, C. M., Hoffman, D. D., Gowdy, P. D., & Kim, J. S. (1995). The perception of color from motion. Perception & Psychophysics, 57, 761-777. D Zmura, M., Colantoni, P., Knoblauch, K., & Laget, P. (1997) Color transparency. Perception, 26, 471-492. da Pos, O. (1989). Trasparenze. Padua: Icone. 13
Frisby, J. P. & Clayworthy, J. L. (1975). Illusory contour: Curious cases of simultaneous brightness contrast? Perception, 4, 349-357. Gibson, J. J. (1979). The Ecological Approach to Visual Perception. Boston: Houghton Mifflin. Grosof, D. H., Shapley, R. M., & Hawken, M. J. (1993). Macaque V1 neurons can signal illusory contours. Nature, 365, 550-552. Kanizsa, G. (1979). Organization in Vision. New York: Praeger. Kaplan, G. A. (1969). Kinetic disruption of optical texture: The perception of depth at an edge. Perception & Psychophysics, 6, 193-198. Lappin, J. S., Doner, J. F., & Kottas, B. (1980). Minimal conditions for the visual detection of structure and motion in three dimensions. Science, 209, 717-719. Marr, D. (1982). Vision. New York: Freeman. Metelli, F. (1974). The perception of transparency. Scientific American, 230, 90-98. Michotte, A., Thines, G., & Crabbe, G. (1964). Amodal completion of perceptual structures. Louvain, France: Publications Universitaires de Louvain. Miyahara, E., & Cicerone, C. M. (1997). Color from motion: separate contributions of chromaticity and luminance. Perception, 26, 1381-1396. Nakayama, K., & Shimojo, S. (1990). Toward a neural understanding of visual surface representation. Cold Spring Harbor Symposia on Quantitative Biology, 55, 911-924. Nakayama, K., & Shimojo, S. (1992). Experiencing and perceiving visual surfaces. Science, 257, 1357-1363. Nakayama, K., Shimojo, S., & Ramachandran, V. S. (1990). Transparency: Relation to depth, subjective contours, luminance and neon color spreading. Perception, 19, 497-513. Peterhans, E., & von der Heydt, R. (1991). Subjective contours: Bridging the gap between psychophysics and physiology. Trends in Neurosciences, 14, 112-119. Shipley, T. F., & Kellman, P. J. (1994). Spatiotemporal boundary formation: Boundary, form, and motion perception from transformations of surface elements. Journal of Experimental Psychology: General, 123, 3-20. Stappers, P. J. (1989). Forms can be recognized from dynamic occlusion alone. Perceptual and Motor Skills, 68, 243-251. van Tuijl, H. F. (1975). A new visual illusion: Neon-like color spreading and complementary color induction between subjective contours. Acta Psychologica, 39, 441-445. Varin, D. (1971). On phenomena relating to contrast and chromatic diffusion in the spatial organization of perceptual space. Rivista di Psicologia, 65, 101-128. von Bezold, W. (1874). Die Farbenlehre. Braunschweig: Viewig. von der Heydt, R., & Peterhans, E. (1989). Mechanisms of contour perception in monkey visual cortex. I. Lines of pattern discontinuity. Journal of Neuroscience, 9, 1731-1748. Wertheimer, M. (1923). Investigations on the gestalt of shape. Psychologische Forschung, 4, 301-350. 14
Wallach, H. & O Connell, D. N. (1953) The kinetic depth effect. Journal of Experimental Psychology, 45, 427-430. Yamada, W., Fujita, N., & Masuda, N. (1993). Amodal completion as another perception of color spreading stimuli. Perception and Motor Skills, 76, 1027-1033. Yonas, A., Craton, L. G., & Thompson, W. B. (1987). Relative motion: Kinetic information for the order of depth at an edge. Perception & Psychophysics, 41, 53-59. Figure Captions Figure 1. A typical display of color from motion is shown. Each frame (8 deg of visual angle on a side) consists of an achromatic background field (CIE x = 0.276, y = 0.286, luminance 73 cd/m 2 ) over which was randomly arrayed 800-1200 dots (each 3.5 min of arc in diameter). Within a circular test region (typically 1 to 2 deg of visual angle in diameter), the dots were colored green (CIE x = 0.280, y = 0.610). All other dots were red (CIE x = 0.621, y = 0.344). The luminance of the red and green dots could be independently varied. To create successive frames, no dots changed their locations within the frame; only the color assignments of the dots were changed, according to a uniform vertical displacement (0.12 deg of visual angle) of the test region in successive frames. On the left: Still views of successive frames are depicted. On the right: When frames are cycled, typically with an effective displacement rate of the test region equivalent to 7 deg/sec, up and down over a vertical region spanning 5 deg of visual angle, an illusory green disk moving up and down pops into view and a spread of illusory green color is seen in the physically achromatic test region. Figure 2. (Adapted from Chen & Cicerone, 2002a) The salience of the color spread as measured with cancellation is linked to the salience of the apparent motion. We varied the effective rate of vertical translation of the test region between zero and 12 deg of visual angle per sec. The 15
results for two observers can be well described by two linear functions with a steeply sloping first branch (from an effective speed of translation from zero to 1 deg/sec) and a second branch of much reduced or zero slope (from 1 deg/sec to 12 deg/sec). Figure 3. There are two modes in which color from motion can be perceived, either in modal completion (left) as a localized change of illumination, a colored spotlight or shadow, moving over a textured surface or in amodal completion (right) as a moving, colored object seen through apertures in an occluding surface. The mode in which color from motion is seen depends on figural cues and regional differences in the luminance contrast between the chromatic elements and the achromatic background. Regions with figural cues tend to be seen as moving. These regions are seen in modal completion if their defining figural elements (dots in this case) are of lower luminance as compared to the background and in amodal completion if the defining figural elements are of higher luminance as compared to the background. 16
Still View of Single Frames Apparent Motion
0.6 Cancellation Value (cd/m 2 ) 0.4 0.2 0 5 10 15 Speed (deg/sec)
Modal Apparent Motion Amodal
Modal Apparent Motion Amodal