Scotopic hue percepts in natural scenes

Transcription

1 Journal of Vision (2013) 13(13):15, Scotopic hue percepts in natural scenes Sarah L. Elliott Department of Psychology, Roosevelt University, IL, USA # $ Dingcai Cao Department of Ophthalmology and Visual Sciences, University of Illinois at Chicago, IL, USA # $ Traditional trichromatic theories of color vision conclude that color perception is not possible under scotopic illumination in which only one type of photoreceptor, rods, is active. The current study demonstrates the existence of scotopic color perception and indicates that perceived hue is influenced by spatial context and topdown processes of color perception. Experiment 1 required observers to report the perceived hue in various natural scene images under purely rod-mediated vision. The results showed that when the test patch had low variation in the luminance distribution and was a decrement in luminance compared to the surrounding area, reddish or orangish percepts were more likely to be reported compared to all other percepts. In contrast, when the test patch had a high variation and was an increment in luminance, the probability of perceiving blue, green, or yellow hues increased. In addition, when observers had a strong, but singular, daylight hue association for the test patch, color percepts were reported more often and hues appeared more saturated compared to patches with no daylight hue association. This suggests that experience in daylight conditions modulates the bottom-up processing for rod-mediated color perception. In Experiment 2, observers reported changes in hue percepts for a test ring surrounded by inducing rings that varied in spatial context. In sum, the results challenge the classic view that rod vision is achromatic and suggest that scotopic hue perception is mediated by cortical mechanisms. Introduction It is well known that the color appearance of a light in photopic lighting conditions is not determined solely by cone excitations from the light. Instead, color appearance depends strongly on the context in which the light is seen. This dependence is often exemplified with chromatic contrast or assimilation effects effects that are accounted for by retinal or cortical receptive field properties (see Shevell & Kingdom, 2008). Recent studies also point to strong contributions from high-level processes such as perceptual organization (e.g., Schirillo & Shevell, 2000; Wollschlager & Anderson, 2009; Xian & Shevell, 2004), or more cognitive processes of visual memory (Hansen, Olkkonen, Walter, & Gegenfurtner, 2006; Olkkonen, Hansen, & Gegenfurtner, 2008). Rod and cone signals interact through three major pathways within the retina (see Buck, 2004 for review), and physiological as well as psychophysical studies suggest signals between photoreceptor types are confounded within the magnocellular, parvocellular, and koniocellular pathways (Cao, Pokorny, & Smith, 2005; Cao, Pokorny, Smith, & Zele, 2008; Field et al., 2009; Lee, Smith, Pokorny, & Kremers, 1997; Purpura, Kaplan, & Shapley, 1988). Percepts under rod-mediated vision, therefore, are exposed to similar contextual influences as percepts under daytime illuminance. This study examined whether contextual and high-level cognitive processes influence color perception under rod-mediated vision as well. Color perception under photopic light levels begins with the postreceptoral comparison of activity within the L, M, and S cone types. As illumination decreases into the mesopic range, rod signals are known to contribute to color percepts, causing a shift in hue sensations most frequently toward blue or green (or other less frequent hue percepts, reviewed in Buck, 2014), a perceived reduction in saturation of spectral lights (Buck, Knight, Fowler, & Hunt, 1998; Nerger, Volbrecht, & Haase, 2003) and an enhancement of perceived brightness (Benimoff, Schneider, & Hood, 1982; Ikeda & Shimozono, 1981). When illumination decreases to levels below cone thresholds, wavelength discrimination is theoretically impossible (i.e., there is no comparison of spectral signals available with only one active receptor type). Under scotopic conditions, however, the world does not appear completely devoid of color and many individuals feel that the should-be achromatic world has a bluish quality. Nagel (1924) notes that stimuli presented below cone thresholds appear blue or bluish green. Color Citation: Elliott, S. L., & Cao, D. (2013). Scotopic hue percepts in natural scenes. Journal of Vision, 13(13):15, 1 13, journalofvision.org/content /13/13/15, doi: / doi: / Received May 21, 2013; published November 14, 2013 ISSN Ó 2013 ARVO

2 Journal of Vision (2013) 13(13):15, 1 13 Elliott & Cao 2 naming (Ishida, 2002; Pokorny, Lutze, Cao, & Zele, 2006) and color matching (Shin, Yaguchi, & Shioiri, 2004) experiments using color order samples (e.g., Munsell or OSA Uniform Color Scale) presented under scotopic illuminance support this observation, but also indicate that other hues, such as red or orange, are perceived when the lightness of the sample is low. Although color percepts under scotopic conditions are less saturated and reduced in gamut compared to photopic conditions, some color experience clearly remains in very dim illumination. In fact, previous studies suggest color percepts under scotopic conditions may change depending on the context surrounding a target object. When more than one color sample is in the field of view, hue percepts can fall into more than one or two categories (Pokorny et al., 2006; Schneider & von Campenhausen, 1998). For example, Pokorny et al. (2006) found that at scotopic light levels, reported hues for the OSA-UCS gray samples could be primarily classified into two perceptual hue categories that were associated with sample scotopic luminances: (a) blue or green for gray samples having the highest scotopic luminance, and (b) red or orange for gray samples having an intermediate scotopic luminance in the scene. The same trends were observed for the other OSA-UCS color sample categories in the same experiment. Therefore, scotopic hue percepts depend on how the scotopic luminance of the target sample relates to the scotopic luminance of other samples in the scene. In this sense, scotopic hue percepts are relational and potentially involve natural visual experience (Pokorny et al., 2006). These studies, however, used a limited number of large samples that varied in spectral reflectance and were freely viewed in space and time (Pokorny et al., 2006; Schneider & von Campenhausen, 1998). Therefore, it is not clear if relational scotopic hues (a) exist under conditions with little variability in long-wavelength reflectance (i.e., where rod and cone spectral sensitivities are similar; Wald, 1945), and (b) occur in a more natural context. The current study used natural images presented under scotopic illumination to first confirm the existence of rod-mediated relational hue percepts and then to evaluate how the percepts relate to the spatial pattern of light at the retina as well as the presence of a daylight hue association. Methods Observers Participants included eight naïve observers recruited from the University of Illinois at Chicago as well as the two authors. Observers ranged in age from years (mean age: 26.6 years, two male) and were normal trichromats based on testing with the Neitz anomaloscope (Neitz Instruments Co., Ltd., Tokyo, Japan). Observers wore their habitual corrective lenses if necessary. Written informed consent was obtained following the Tenets of Helsinki and with approval of the Institutional Board of the University of Illinois. Apparatus Visual stimuli were presented on a 45-cm ViewSonic color CRT (P95fþ, model VCDTS M, View- Sonic North America, Walnut, CA) controlled by an imac (O.S , Apple, Inc., Cupertino, CA). The CRT had a refresh rate of 60 Hz noninterlaced and a pixel resolution of A luminance look-up table was created from the light output of each CRT phosphor by measuring to find 990 equal steps (0.1% increments) between 1% and 100% of the phosphor s maximum using a photometer (International Light 1700, InternationalLight Technologies, Inc., Peabody, MA). Only the blue phosphor, however, was used during stimulus presentation, since the blue phosphor provides the highest ratio of rod/cone excitation. Four layers of calibrated neutral density filters were sandwiched (for a total log units) and completely covered the CRT s face through the whole study. This reduced the luminance of the monitor so stimuli emitting 2 photopic cd/m 2 (11.44 scotopic cd/m 2 ) from the blue phosphor corresponded to a luminance of approximately photopic cd/m 2 ( scotopic cd/m 2 ) when viewed through the filters. A game pad was used to record observer responses. The experimental software was written in Cocoa Objective-C (Macintosh Xcode environment). Stimuli Fifty-four natural images representing a variety of scenes including mountains, meadows, textures, closeups of fruit and flowers, and manmade objects were chosen from the McGill Calibrated Colour Image Database (Olmos & Kingdom, 2004). Images were cropped (if necessary) to pixels, corresponding to a visual angle of 248 on the display. Images were first calibrated for camera nonlinearity following the procedures of Olmos and Kingdom. Scotopic luminance was then computed at each pixel using the scotopic luminosity function and the spectral sensitivity curves for the three camera sensors (obtained from Fred Kingdom, personal correspondence). Finally, adjusted RGB values were summed to obtain a gray value for each pixel, which was then converted to 256 gray-levels using the calibrated luminance lookup table for the blue phosphor of the CRT. Examples of original

3 Journal of Vision (2013) 13(13):15, 1 13 Elliott & Cao 3 Figure 1. (Top panel) Six out of the 54 original images, showing (from left) manmade objects; closeups of fruit, flowers, and foliage; mountains; and textures. (Bottom panel) calibrated versions (see text for details) of images in the top panel with the test patch area highlighted by the white box. and calibrated images are shown in Figure 1. When viewed through the neutral density filters, the average mean luminance across the 54 images was 3.79 log photopic ( 3.03 log scotopic) cd/m 2, but varied between 3.48 and 4.32 log photopic ( 2.72 and 3.56 log scotopic) cd/m 2. The maximum luminance within each image was no higher than 3.17 log photopic cd/m 2. Note cone detection threshold is about 1 log Troland or 2.7 log photopic cd/m 2 for an 8 mm pupil (Stabell & Stabell, 1976). At this maximal luminance, chromatic sensitivity is significantly degraded if not absent in the S-cone pathway (Brown, 1951; Walkey, Barbur, Harlow, & Makous, 2001). Although activity within the L- and M-cone pathways cannot be entirely ruled out, rod sensitivity increases up to 2 3 log unit greater than cone sensitivity in the midto short-wavelengths regions of the spectrum (Wald, 1945), and the use of the blue phosphor alone decreased the probability and variability of L-cone contributions to hue percepts. Therefore, the detection of the natural images was largely mediated by rods. Procedure Sessions began with 25 minutes of dark adaptation. The end of the adaptation period was signaled by 3 beeps of the computer. Images were presented on a black background sequentially in a random order. Each image was initially displayed for 10 s. For the first 5 s, observers were allowed to move their gaze freely over the whole image. The second 5-s period highlighted a single 48 test patch of the image. To maintain constant adaptation, the highlight was achieved by setting all pixels outside the 48 area to the pixel value corresponding to the mean luminance of the whole image. The patch was typically selected to target specific objects or uniform regions within an object (e.g., the head of a dandelion) or a textured region if a focal object was not present (see Figure 1 for examples). Two arrows, one black and one at 3.17 log photopic cd/m 2 (each 18 28), were overlaid to touch to the bottom and top border of each test patch so the patch was clearly visible (in some cases the luminance of the 48 patch was similar to the mean image luminance and therefore difficult to distinguish). Following the second 5-s period, the whole image was displayed and observers reported hue percepts for only the area within the image corresponding to the location of the 48 test patch. Observers could respond only when the whole image reappeared, but had no time constraints in this phase. A 2-s black screen separated individual trials. One daily session contained 108 trials (two repeats of each unique image), and observers completed five daily sessions for a total of 10 hue judgments per image. Color percepts were recorded using a hue scaling technique (Gordon & Abramov, 1988). Observers were instructed to indicate the perceived hue and saturation of the test patch by pressing buttons corresponding to the four basic hues and one button corresponding to percent saturation. Observers were permitted to use any combination of the four basic hues (e.g., orange requires the observer to push the button for red and the button for yellow ). Perceived saturation was reported for the total hue percept (i.e., not split into saturation for each hue part, as was done in Gordon & Abramov, 1988) in 5% increments. For example, a 15% perceived saturation corresponded to pressing the saturation button three times. After observers made their hue and saturation ratings based on the appearance within the test patch, they were also instructed to indicate whether the object in the test patch was associated with a strong, but singular, daylight hue by pressing a separate button. Importantly, however, they were not asked to name

4 Journal of Vision (2013) 13(13):15, 1 13 Elliott & Cao 4 which color was associated with the test patch area. This of course did not prevent them from applying a color name to their association, but the hope was to minimize specific semantic cues that might influence hue appearance. Finally, a complete button was pressed to finish the current trial. If the observer did not perceive hue on a specific trial, they were instructed to press the complete button without making hue or saturation responses (although they were allowed to indicate whether the test patch had a daylight hue association). Natural image statistics The spatial pattern of luminance at the retina is a fundamental stimulus dimension for all visual perception. In daylight conditions, the color appearance of an object can change due to differences in contrast (Chevreul, 1839/ 1967; Ware & Cowan, 1982) or texture (Jenness & Shevell, 1995; Shevell & Wei, 1998) of the surrounding light. If relational hue percepts exist in scotopic conditions, the spatial pattern of light on the retina is likely a critical factor. To quantify the spatial pattern of luminance for each image, the pixel-luminance distribution was first characterized for both the whole image and the test patch alone using the mean, variance, skew, and kurtosis (first, second, third, and fourth-order moments, respectively). The second, third and fourth-order moments of the distribution are measures used to describe the texture, or structure, of a discrete image (Chubb & Yelliott, 2000; Julesz, Gilbert, Shepp, & Frisch, 1973). Next, the Fourier amplitude spectrum was determined as the line fitted to the log amplitude versus rotationally averaged log spatial frequency. Humans are insensitive to frequencies. 2 cpd under scotopic conditions (Savage & Banks, 1992), therefore, the total Fourier amplitude for frequencies,2 cpd was computed for the whole image as it may better reflect the information available for neural processing. To quantify the luminance relationship between the test patch and whole image directly, the Weber contrast of the test patch was calculated using the mean luminance within the test patch compared to the mean luminance of the whole image. In sum, there was a total of 11 statistics to describe each image. Factor analysis with a principal component method was used to extract common factors from nine of the 11 natural image statistics for the 54 images. The mean luminance of the test patch was not included in the analysis as the Weber contrast is a similar measure when compared to the whole image. In addition, the slope of the Fourier amplitude spectrum did not significantly load with any other statistics when it was originally included, so was therefore left out of the final statistical analysis. With a range of 1.58 to 0.96 (average of 1.30), the slope of the amplitude spectra Image statistic across the set of images used here likely did not provide enough variance to load with other image statistics. As shown in Table 1, the analysis revealed that three main factors had eigenvalues larger than 1 (3.70, 2.85, and 1.62, respectively), which accounted for 97% of item variance cumulatively for the images. Following an orthogonal varimax rotation, factor loadings aligned well with raw luminance versus textural properties of each image. Factor 1 loaded the second, third, and fourth order moments of the test patch luminance distribution as well as the Weber contrast and explained 36% of the variance. The loadings indicate that when the test patch was low in variance, skew, and kurtosis, it was often a decrement in contrast as well. This factor was referred to as patch texture. Factor 2 loaded the second, third, and fourth order moments of the whole image luminance distribution, explaining an additional 31% of the variance. This factor was referred to as whole image texture. Factor 3 accounted for 30% of the variance and loaded firstorder moments of mean luminance as well as the total Fourier amplitude for frequencies,2 cpd. This factor was referred to as whole image luminance. For each image, factor scores were computed and then used for further statistical analysis. Statistical analysis Rotated Factor 1 Factor 2 Factor 3 Test patch contrast Variance (test patch) Skew (test patch) Kurtosis (test patch) Variance (whole image) Skew (whole image) Kurtosis (whole image) Mean image luminance Fourier amplitude (, 2 cpd) Table 1. Rotated factor loadings. Hue percepts were grouped into six categories for each image and observer (Pokorny et al., 2006): achromatic (no reported hue), reds (including red and orange), yellows (including yellow and yellow-green), blues (including green, blue-green, and blue), purple, and opponent combinations (blue-yellow and redgreen). A multinomial logistic regression model was used to evaluate how hue percepts changed compared to achromatic percepts in relation to natural image statistics (by factor scores) and daylight hue associations.

5 Journal of Vision (2013) 13(13):15, 1 13 Elliott & Cao 5 Results are reported as relative risk ratios (RRR), which are the ratios of the probability of reporting a specific hue category over the probability of reporting an achromatic percept. Therefore, a RRR value, 1 indicates reporting a specific hue category was less likely than reporting an achromatic percept, whereas a value. 1 indicates reporting that hue category was more likely than reporting an achromatic percept. In addition, Z-scores were calculated for pair-wise comparison of odds between any two hue categories for an increasing value of an independent variable. For these comparisons, a positive (negative) z-score indicates that the relative probability of reporting a specific hue category increased (decreased) compared to the probability of reporting the comparison hue category. Changes in perceived saturation were evaluated in relation to factor scores for the three factors and daylight hues using a four-way randomized-block analysis of variance, with blocks defined by individual observers. The mean saturation for each observer by image combination used in the analysis included only trials where observers reported hue percepts (i.e., the image did not appear achromatic). Results Relational hues are perceived under scotopic illumination, with individual observers reporting hue in an average 45% 6 8% SEM of trials. The reported hues were extremely desaturated, however, as on average the reported saturation was between 5% 10% across observers for the 54 images. There was a slight increase in perceived saturation (on average ;1%) if observers had a daylight hue association with the image, F(9, 338) ¼ 19.91, p, 0.01, but no significant changes in perceived saturation were detected in relation to any factors for the natural images. There was a significant association between the reported scotopic hues and the presence of an unspecified daylight hue association. Observers were more likely to report hue, regardless of hue category, when there was a daylight hue association for the test patch area (RRR 1.50, p, , Table 2). For illustration purposes, images were divided into three groups based on the percentage of trials with a reported daylight hue association, and the mean percentage of the hue percepts within each group are plotted in Figure 2. The first criterion for group division was to separate the images based on the number of trials observers reported a daylight hue association. However, only 24% of the images were reported to have a daylight hue association in.40% of the trials. The second criterion for group division therefore was to obtain an approximately equal number of images with Hue category Factor 1 Factor 2 Factor 3 Daylight hue association Achromatic a Reds 0.22*** 1.17** 1.37*** 6.58*** Yellows 2.18*** 0.64*** 0.47*** 8.74*** Blues 1.69*** 0.70*** 0.90** 1.49*** Purples 1.52*** ** 3.61*** Opponent 2.44*** 0.58* 1.45* 12.25*** Table 2. Relative risk ratios (RRRs) for each hue category. a : Reference category. * p, 0.05; ** p, 0.01; *** p, few reported daylight hue associations (Group 1) compared to more numerous daylight hue associations (Group 3). The first, second and third group, therefore, plot the mean hue percepts for images where daylight hue associations were reported in 10%, between 11% 40%, or.40% of trials, corresponding to 16, 25, and 13 images, respectively. This plot illustrates that hue was perceived in 57% of trials on average for images with a reported daylight hue association in more than 40% of the trials. Hue was perceived less often (average of 41% of trials) for images where a daylight hue association was reported in 40% of trials. The odds-comparison z-scores across color categories revealed the largest increase in percepts of reds, yellows, and opponent combinations, which were significantly more likely to be reported compared to both blues and purple with a daylight hue association. This is shown in Table 3A, which lists the z-scores and p-values of the odds comparisons across hue categories. In Table 3, a positive (negative) z-score indicates that the relative probability of reporting the hue category in Figure 2. Natural images were separated into three groups based on how many trials out of 100 each image was reported to have a daylight hue association: 10 trials (16 images), between 11 and 40 trials (25 images), or. 40 trials (13 images). Bars plot the mean hue percepts for each group as a percentage of the total number of trials. Colored sections correspond to hue categories of achromatic (gray), blues (aqua), red (orange), yellows (yellow), purple (purple), and opponent combinations (dark gray).

6 Journal of Vision (2013) 13(13):15, 1 13 Elliott & Cao 6 Yellows Blues Purple Opponent Achromatic A. Daylight hue association Reds ** 3.00** *** Yellows 10.61*** 4.00*** *** Blues 4.88*** 4.94*** 4.04*** Purples 2.72** 7.43*** Opponent 5.91*** B. Patch texture Yellows Blues Purple Opponent Achromatic Reds 10.22*** 9.19*** 8.44*** 9.67*** 6.85*** Yellows 5.71*** 5.03*** *** Blues ** 14.17*** Purples 3.70*** 6.15*** Opponent 7.58*** C. Whole image texture Yellows Blues Purple Opponent Achromatic Reds 6.38*** 9.32*** 3.24** 2.86** 3.48** Yellows ** *** Blues 3.18** *** Purples Opponent 2.24* D. Mean image luminance Yellows Blues Purple Opponent Achromatic Reds 11.75*** 7.07*** *** Yellows 8.15*** 9.49*** 5.66*** 9.76*** Blues 4.13*** 2.55* 3.12** Purples ** Opponent 1.97* Table 3. Z-scores for pair-wise odds-comparisons between hue categories with increasing (A) daylight hue associations, (B) patch texture (Factor 1), (C) whole image texture (Factor 2), (D) mean image luminance (Factor 3). A positive Z-score indicates an increase in the probability of percepts listed as the row header compared to percepts in the column header, while a negative number indicates a decrease in the probability of a percept in the row header. * p, 0.05; ** p, 0.01; *** p, the row header increased (decreased) compared to the probability of reporting the hue category in the column header. The luminance and texture properties of the test patch and whole image also contributed to relational hue percepts. When the test patch was more uniform (i.e., low variance, skew and kurtosis) and a decrement in contrast compared to the whole image (low Factor 1 score), the probability of reporting reds compared to achromatic (Table 2) and other hue percepts (Table 3B) was higher. Recall that in Table 2, an RRR value,1 indicates the probability of reporting a specific hue category decreased compared to achromatic percepts, whereas a value.1 indicates the probability of reporting that hue category increased compared to achromatic percepts. In this case, RRR ¼ 0.22 indicates that the probability of perceiving reds decreased as Factor 1 increased compared with achromatic percepts. As patch texture increased, the probability of perceiving yellows, blues, purple, and opponent combinations significantly increased compared to achromatic percepts (Table 2), but yellows and opponent combinations increased significantly more compared to all other hue categories (Table 3B). The shift in hue percepts from reds to yellows and blues with an increase in test patch texture is illustrated Figure 3. Each bar illustrates the distribution of hue percepts across 100 trials (10 observers 10 repeats) for individual images, with colored sections corresponding to the percentage for each respective hue category. Larger texture variations within the whole image (Factor 2), on the other hand, predicted a higher probability of reporting reds compared to achromatic percepts (Table 2) as well as all other hue categories (Table 3C), but a lower probability of reporting yellows, blues and opponent combinations compared to achromatic percepts (Table 2). Lastly, a higher mean luminance for the whole image (Factor 3) predicted a lower probability of reporting yellows and blues, but a higher probability of reporting reds, purple, and opponent combinations compared to achromatic percepts (Table 2). In fact, the probability of reporting yellows decreased significantly more than any other hue category, followed closely by blues (Table 3D). Figure 4 summarizes the results in relation to natural image statistics. For this illustration, the images were separated into tertiles (18 images per tertile) based on the relative scores (low, moderate, and high) within each factor. The mean percentage for each hue category

7 Journal of Vision (2013) 13(13):15, 1 13 Elliott & Cao 7 was calculated for each group, and the top, middle, and bottom panels show the shifts in hue percepts for each of the three factors. This figure first makes clear that hue percepts were more likely to be reported overall when patch texture and contrast are high (top panel) and when the whole image is low in texture (middle panel). Whether hue is perceived at all, however, does not appear to change with changes in mean luminance of the whole image. Second, the figure illustrates how hue percepts change in relation to natural images statistics. Reds were more likely reported when the test patch was low in texture and a decrement in contrast (top panel), or when the whole image was higher in texture (middle panel) or mean luminance (bottom panel). On the contrary, yellows and blues (and to a lesser extent, purples and opponent hue combinations) were more likely reported with low whole image texture (middle panel) and mean luminance (bottom panel), but high patch texture (top panel). While the data from all observers were collapsed and the RRR values and z-scores from multinomial logistic regression capture the general pattern of relational hue percepts across observers, the variability of hue percepts between observers should be noted. Figure 5 is similar to the top panel of Figure 4, with the images split into tertiles based on the relative scores within Factor 1. The separate panels plot the mean percentage of each hue category for individual observers. Of interest also are the percepts with opponent-hue combinations. Opponent-hue combination were reported by four observers and accounted for 0.015% of the total trials with a hue percept. The opponent-hue percepts were not likely the result of incorrect button combinations for at least two observers (including one author) as the percepts were consistent across repeats for specific images (although not the same images for the two observers). Blue-yellow percepts accounted for 94% of the opponent-hue combinations, while the other 6% were a combination of red and green. Although previous evidence suggests opponent-hue percepts are possible under some conditions (e.g., with stabilized images and border melting; Billock, Gleason, & Tsou, 2001; Billock & Tsou, 2010; Crane & Piantanida, 1983), it is difficult to draw any conclusions about opponent-color perception with the limited data in the current study. Figure 3. Distribution of hue percepts for each unique image over 100 trials (10 observers 10 repetitions), organized by increasing score for Factor 1 (patch texture). The position of images from Figure 1 is shown to the right. Colored sections are as in Figure 2. Experiment 2. Scotopic hue percepts with simultaneous contrast The previous experiment showed that perceived hue in scotopic conditions is related to context as well as

8 Journal of Vision (2013) 13(13):15, 1 13 Elliott & Cao 8 percepts depend on the border contrast or on the average light in the surrounding area. Methods Observers Eight naïve observers as well as the two authors participated. The authors and one naïve observer participated in both experiments. Observers ranged in age from years (mean: 26.5 years, five male) and were normal trichromats based on testing with the Neitz anomaloscope. Figure 4. Natural images were separated into three groups of 18 based on the relative scores (low, moderate, and high) within each factor. Bars plot the mean hue percepts for each group as a percentage of the total number of trials, and the top, middle, and bottom panels show the shifts in hue percepts within each of the three factors. Colored sections are as in Figure 2. daylight hue associations. With natural images, it is difficult to separate whether hue percepts arise from daylight hue associations and are then modified by image context, or whether hue percepts can be induced by context alone. The following experiment used a relatively simple design with a test ring surrounded by inducing rings to determine whether hue percepts are influenced by context alone. The previous experiment suggests that luminance differences between the test patch and whole image have a large impact on scotopic hue percepts. This experiment therefore used contrast as the contextual cue, and varied the contrast between the test and inducing rings. Separate conditions varied the number of inducing rings to determine whether scotopic hue Figure 5. Natural images were separated into three groups of 18 based on the relative scores within Factor 1. Bars plot the mean hue percepts for each observer as a percentage of the total number of trials, and each panels corresponds to a separate observer. Colored sections are as in Figure 2.

9 Journal of Vision (2013) 13(13):15, 1 13 Elliott & Cao 9 Figure 6. Unique spatial parameters used for stimuli in Experiment 2. Observers judged the hue appearance of the middle ring only. Differences in Michelson contrast are shown for simple (two inducers, left panel) and complex context (four inducers, right panel). Procedure The test stimulus was a ring 18 in width, placed at a 78 retinal eccentricity (spanning a retinal eccentricity of ). This was to ensure the test ring was placed at a retinal location with a high density of rods without requiring observers to scan the monitor. Pilot data indicated that hue percepts did not differ significantly between a 18 and 28 test ring, but with a 28 ring size, some stimuli could not be displayed completely due to the limited size of the monitor. The luminance of the test ring was held constant at 3.84 log photopic (or 3.24 log scotopic) cd/m 2. For a total of 122 trials (61 unique trials, two repeats) per session, inducer number (no, simple, and complex context) and contrast were varied. No context served as the control, where observers judged the perceived hue of the test ring in isolation. With context, the test ring was surrounded by either two (simple context) or four (complex context) 18 inducting rings, split evenly on each side of the test ring and presented on a black background. The Michelson contrast of the inducing rings differed from the test by 640% or 680%. In simple context trials, the two inducers were presented with the same contrast. In complex context trials, inducer rings flanking the test shared contrast (e.g., þ40%), and inducer rings furthest from the test shared the opposite contrast (e.g., 40%). The average luminance of the complex context was therefore the same luminance as the test ring. If hue percepts depend on the average light in the surround instead of the local contrast, then observers should report achromatic percepts more often in the complex context case. Examples of the different spatial configurations are shown in Figure 6. Within each session, the order of inducer number and contrast was randomized. Observers were asked to judge the hue and saturation appearance of the middle (which was always the test) ring only during a 2-s stimulus presentation. The hue scaling technique as above was used to record hue and saturation percepts, but observers were not asked about daylight hue associations. Following the observer s response, the screen remained black for 2 additional seconds before the next stimulus presentation. Data analysis As was done for the natural images, hue percepts were grouped into five (no opponent combination needed) categories for each image and observer. A multinomial logistic regression model was used to evaluate how hue percepts changed compared to achromatic percepts in relation to inducer number and contrast. Results are reported as relative risk ratios (RRR), indicating the ratio of the probability of reporting a specific hue category over the probability of reporting an achromatic percept. Changes in perceived saturation were evaluated using a four-way randomized-block ANOVA, with blocks defined by individual observers and excluding trials where observers reported achromatic hue percepts. Results This experiment confirms that relational scotopic hues can be experienced due to differences in spatial context alone. Individual observers reported hue percepts in an average of 55% 6 10% SEM of the trials. Perceived saturation remained low, however, with a mean of 9.5% and range of 5% 28%. Consistent with findings using natural images, no significant differences in perceived saturation were found across changes in spatial context. Hue percepts across conditions for all observers are summarized in Figure 7. Separate panels illustrate differences in hue percepts by inducer number (left), inducer contrast (middle left), and the interaction of inducer number by contrast (right two panels). As above, colored sections correspond to the respective hue category. RRR and corresponding p-values for each hue category are listed in Table 4. Hue percepts in the current experiment show some consistency with trends detected with natural image statistics. Reds were more likely perceived when the test was a decrement compared to the flanking inducers (inducer contrast of þ40% or þ80%, RRR¼ 1.01, p, 0.05), regardless of inducer number. This is consistent with findings above, where the test patch

10 Journal of Vision (2013) 13(13):15, 1 13 Elliott & Cao 10 Figure 7. The distribution of hue percepts, presented as the percentage of total trials for each unique spatial parameter defining stimuli in Experiment 2. (Left panel) hue percepts for differences in inducer number. With no context, each bar corresponds to 100 trials (10 observers 10 repetitions). With context, each bar corresponds to 400 trials (10 observers 4 unique contrasts 10 repetitions). (Middle panel) hue percepts for differences in inducer contrast; each bar corresponds to 200 trials (10 observers 2 inducer conditions 10 repetitions). (Right two panels) hue percepts for the interaction between contrast and inducer number; each bar corresponds to 100 trials (10 observers 10 repetitions). Colored sections are as in Figure 2. was more likely to be judged as red or orange if it was a decrement compared to the whole image, and indicates that the hue percepts depend on the border contrast rather than the mean luminance. Yellows were reported more often when the test ring was presentedwithnocontextanddecreasedinfrequency as the context became more complex (RRR ¼ 0.48, p, 0.01). This is also consistent with results using natural image statistics, where yellows were reported more often when the test patch was an increment in contrast and was surrounded by less texture. That is, in the no context stimulus here, the lone test ring was an increment in luminance compared to the black background, and the black background itself was inherently absent of texture information. The relationship between spatial context and hue percepts of blue and purple, however, is not clearly consistent with natural image statistics. Purple was more likely reported with more complex context (RRR ¼ 1.94, p, 0.05), and blue percepts showed a significant interaction with inducer number and contrast (RRR ¼ 0.99, p, 0.05). Hue category Inducer number Contrast Inducer number contrast Achromatic Reds * 1.00 Yellows 0.48** Blues * Purples 1.94* Table 4. Relative risk ratios (RRRs) for each hue category with controlled context. * p, 0.05; ** p, Discussion The current study confirms that desaturated hue percepts under scotopic illumination a) exist, and b) are related to the spatial context in natural scenes (Pokorny et al., 2006) as well as cognitive processes of color perception. Hue percepts were shown to change with differences in the luminance and textural context across scenes, as well increase in frequency and saturation with the number of reported daylight hue associations of objects within the test patch. The presence of an unspecified daylight hue association was the strongest predictor of whether hue was reported (see Table 2), and contributed to hues appearing slightly more saturated. Many manmade and natural objects have a distinctive color under daylight illumination. Recent studies suggest that the memory color associated with certain objects may play an important role in color constancy, maintaining the color appearance of familiar objects despite large changes in illumination (Hansen et al., 2006; Olkkonen et al., 2008). The image set used here included many familiar objects with distinct daylight hue associations within the test patch area, such as a stop sign, a pumpkin, a dandelion, or a Shell oil company sign. Although the current results cannot directly attribute the increase in hue percepts to specific daylight hue associations (i.e., the specific daylight hue associations were not recorded), results are consistent with a role for memory color and with previous literature demonstrating that hues matched from memory (which are distinct from measures of spectral reflectance, Jin & Shevell, 1996) are more saturated (Bartleson, 1960; Newhall, Burnham, & Clark, 1957). Memory color may

11 Journal of Vision (2013) 13(13):15, 1 13 Elliott & Cao 11 therefore provide a strong cue to color appearance under scotopic illumination, but more work needs to be done to elucidate the mechanisms. The current results do indicate, however, that the luminance and texture relations within the image influence hue percepts under scotopic illumination. When the test patch was low in texture and a decrement in contrast (Factor 1), or when the whole image was higher in texture (Factor 2), the likelihood that observers reported hues in the red category increased compared to all other hue categories. Red percepts also increased compared to other hue categories when the mean luminance of the whole image (Factor 3) was high. On the contrary, hue percepts in the yellow and blue category, and to a lesser extent, purple and opponent hue combinations, were significantly more likely to be reported when the test patch was an increment in contrast and contained more texture, as well as when the whole image was low in texture and mean luminance. Results from Experiment 2 confirmed that relational hue percepts can be induced by context alone and are not simply an artifact of daylight hue associations. The probability of perceiving hues in the red category increased if the test ring was a luminance decrement compared to the flanking inducer rings, consistent with findings using natural images (Experiment 1) and previous results using OSA paper samples (Pokorny et al., 2006). The probability of perceiving hues in the yellow category decreased with the number of elements surrounding the test ring, possibly reflecting the change in border contrast (with no context, the ring was an increment in luminance compared a black background, consistent with findings using natural images) and/or a change in the spatial pattern of light (i.e., more texture with more context) in the surround. Previous studies suggesting the presence of relational hue percepts under scotopic illumination used color order systems (i.e., Munsell, HKS, or OSA uniform color scale) with samples that vary in spectral reflectance (Pokorny et al., 2006; Schneider & von Campenhausen, 1998). From these studies, it is not clear if color perception under scotopic conditions is due purely to rod activity, or due to the postreceptoral comparison of activity from different photoreceptor types. In color appearance studies, L-cone activation may account for scotopic shifts in color appearance toward red for Munsell colors reflecting long wavelength light (Shin et al., 2004) due to the equivalent sensitivity of rods and cones at wavelengths.650 nm (Wald, 1945). One strength of the current study is that the blue phosphor maintained the spectral composition of light across stimulus conditions and ruled out significant contributions from the L cone. This study therefore provides strong evidence that some color experience remains in purely scotopic illumination. Potential neural explanations for rod-mediated hue percepts Hue percepts under scotopic illumination may reflect the microcircuitry of postreceptoral mechanisms, which cannot distinguish between rod and cone signals (Cao et al., 2008; Field et al., 2009; Lee et al., 1997; Purpura et al., 1988). A similar explanation has been used to account for multiple color sensations using a stimulus with a diameter that approximates the aperture of a single cone (Hofer, Singer, & Williams, 2005). For instance, one explanation for the preponderance of reported hues in the blue category is that rod signals connect to the small bistratified ganglion cells with the same sign as S cones (Field et al., 2009). Rods also influence postreceptoral mechanisms via gap-junction synapses with L and M cones (Hornstein, Verweij, Li, & Schnapf, 2005). This raises the possibility that hue percepts may be the result of color-opponent centersurround mechanisms in the parvocellular and koniocellular pathway. Previous studies suggest, however, that a retinal illuminance,10 scotopic Td is below threshold for the rod-cone coupling pathway (Sharpe & Stockman, 1999) as well as color opponent responses in the koniocellular pathway (Field et al., 2009). With a maximum luminance approaching 0.1 scotopic Td in the current study, it is unlikely that chromatic opponency can account for the diversity of hue percepts seen here. In addition to a contribution from memory about the color of specific objects, the scotopic hue percepts may arise from cortical mechanisms with knowledge about chromatic appearance relations within the environment. There is some evidence that chromatic structure is rather constant across natural scenes. Differences in luminance are strongly associated with differences in color (Fine, MacLeod, & Boynton, 2003), but the sign and magnitude of these correlations is not as clear. One recent study suggests that there is a slight positive correlation between the luminance and redgreen postreceptoral channels as well as a spatialfrequency dependent correlation between the luminance and blue-yellow channels (Johnson, Kingdom, & Baker, 2005). In their image set (Johnson et al., 2005), red objects (e.g., fruit) were found to have a higher luminance than the surround (e.g., foliage), and finegrained yellowish shadows (e.g., shadows of foliage with a narrow spectrum centered on green) had a lower luminance than the more uniform bluish shadows. There is a relationship also between viewing distance and chromaticity (Burton & Moorhead, 1987; Hendley & Hecht, 1949). Short-range scenes contain more saturated chromaticities composed of middle to long wavelengths, and scenes with increasing viewing distance contain less saturated chromaticities shifted toward the short wavelength region of the spectrum.

12 Journal of Vision (2013) 13(13):15, 1 13 Elliott & Cao 12 Color constancy mechanisms may use these relations to maintain color percepts under scotopic conditions. For instance, relative brightness changes related to the Purkinje shift may provide one cue about color in a scene. That is, objects that reflect long wavelength light may appear bright during the day, but will appear dimmer compared to objects reflecting short- to midwavelengths at night, so dim items may take on a reddish appearance. In addition, images in the current experiment that contain closeups of objects may appear more saturated in yellow and red hues compared to images of distant mountains or meadows. Keywords: hue percepts, scotopic color vision, natural image statistics Acknowledgments This paper is supported by F32 EY (S.E.), R01 EY (D.C.), P30 EY (UIC core grant for vision research), and an Unrestricted Grant from Research to Prevent Blindness. We would like to thank Dr. Michael Webster for his help with the natural image statistics and Dr. Joel Pokorny for his thoughtful discussions throughout the project. Commercial relationships: none. Corresponding author: Dingcai Cao. dcao98@uic.edu. Address: Department of Ophthalmology and Visual Science, University of Illinois at Chicago, Chicago, IL, USA. References Bartleson, C. J. (1960). Memory colors of familiar objects. Journal of the Optical Society, 50, Benimoff, N. I., Schneider, S., & Hood, D. C. (1982). Interactions between rod and cone channels above threshold: a test of various models. Vision Research, 22, Billock, V. A., Gleason, G. A., & Tsou, B. H. (2001). Perception of forbidden colors in retinally stabilized equiluminant images: An indication of softwired cortical color opponency? Journal of the Optical Society of America A, 18, Billock, V. A., & Tsou, B. H. (2010). Seeing forbidden colors. Scientific American, 302, Brown, W. R. J. (1951). The influence of luminance level on visual sensitivity to color differences. Journal of the Optical Society of America, 41, Buck, S. L. (2014). The interaction of rod and cone signals: Pathways and psychophysics. In J. S. Werner & L. M. Chalupa (Eds.), The new visual neuroscience (pp ). Buck, S. L., Knight, R., Fowler, G., & Hunt, B. (1998). Rod influence on hue-scaling functions. Vision Research, 38, Burton, G. J., & Moorhead, I. R. (1987). Color and spatial structure of natural scenes. Applied Optics, 26, Cao, D., Pokorny, J., & Smith, V. C. (2005). Matching rod percepts with cone stimuli. Vision Research, 45, Cao, D., Pokorny, J., Smith, V. C., & Zele, A. J. (2008). Rod contributions to color perception: Linear with rod contrast. Vision Research, 48, Chevreul, M. E. (1967). The principles of harmony and contrast of colors and their applications to the arts. New York: Reinhold. (Original work published 1839) Chubb, C., & Yelliott, J. I. (2000). Every discrete, finite image is uniquely determined by its dipole histrogram. Vision Research, 40, Crane, H. D., & Piantanida, T. P. (1983). On seeing reddish green and bluish yellow. Science, 221, Field, G. D., Greschner, M., Gauthier, J. L., Rangel, C., Shlens, J., Sher, A., et al. (2009). Highsensitivity rod photoreceptor input to the blueyellow color opponent pathways in macaque retina. Nature Neuroscience, 12, Fine, I., MacLeod, D. I. A., & Boynton, G. M. (2003). Surface segmentation based on the luminance and color statistics of natural scenes. Journal of the Optical Society of America A, 20, Gordon, J., & Abramov, I. (1988). Scaling procedures for specifying color appearance. Color Research and Application, 13, Hansen, T., Olkkonen, M., Walter, S., & Gegenfurtner, K. R. (2006). Memory modulates color appearance. Nature Neuroscience, 9, Hendley, C. D., & Hecht, S. (1949). The colors of natural objects and terrains, and their relation to visual color deficiency. Journal of the Optical Society of America, 39, Hofer, H., Singer, B., & Williams, D. R. (2005). Different sensations from cones with the same photopigment. Journal of Vision, 5(5):5, ,