Effectiveness of color in picture recognition memory

Transcription

1 Japanese Psychological Research 1997, Volume 39, No. 1, Effectiveness of color in picture recognition memory KOTARO SUZUKI 1 and RIKA TAKAHASHI 2 Department of Psychology, Faculty of Humanities, Niigata University, Ikarashi, Niigata , Japan Abstract: Investigations were conducted into the effectiveness of color in picture recognition memory. In the study phase, half of the pictures were presented in color and the other half in black and white. In the test phase, half were presented in the same color mode as the study pictures and the other half in the other mode. In immediate and 1-week-delayed tests, the recognition performance was highest when color pictures were used in both the study and test phases. The recall for the color mode of the study pictures, however, was not as good, even with the color pictures. This suggests that the effectiveness of color in picture recognition is not necessarily due to the memory for colors in the pictures themselves, but is probably due to the distinctiveness of features highlighted by the colors. We also found that in the recall performance for the color mode it was more difficult to detect the deletion of colors than to detect their addition. Key words: picture memory, recognition memory, color, distinctiveness, asymmetric confusability effect. Studies on picture memory have shown that people can recognize quite a large number of pictures (e.g., Haber, 1970; Nickerson, 1965; Shepard, 1967). In these studies and many other picture memory studies to date, either colored pictures or black-and-white pictures have often been used, but few have compared the recognition performance between the two. Therefore, it is not known whether or not colors affect recognition performance for pictures. The present study deals with this issue. There are a few studies which indirectly suggest a possible effect of colors on picture recognition. Studies on word memory have shown that words were recalled better when they were presented in color than when presented in black and white (Bousfield, Esterson, & Whitmarsh, 1957; Denis, 1976). Boynton and Dolensky (1979) reported that colors improved the performance of the sorting and recognition of books. Watkins and Schiano (1982) also reported that imagining colors on black-and-white drawings in the study phase improved recognition performance when the imagined colors were the same as those of the colored drawings in the test phase. On the other hand, in an experiment that compared recall performance between words and drawings, Paivio, Rogers, and Smythe (1968) found that colors had no incidental effect on the recall of the drawings. Concerning memory for colors of drawings, Park and Mason (1982) found this to be poorer than that for position, and claimed that the encoding of 1 Thanks are due to Drs. Hitoshi Honda, Yuji Hakoda, James C. Bartlett, and Margaret J. Intons-Peterson for their helpful suggestions on earlier versions of the manuscript. Correspondence concerning this article should be addressed to Kotaro Suzuki. 2 Present name: Rika Kanzaki. She is now at ATR Human Information Processing Research Laboratories, Kyoto Japanese Psychological Association. Published by Blackwell Publishers Ltd, 108 Cowley Road, Oxford OX4 1JF, UK and 350 Main Street, Malden, MA 02148, USA.

2 26 K. Suzuki and R. Takahashi color attributes in the drawings was not an automatic process, but rather one which required effort. These findings suggest the possibility that color information has only a minor effect, if any. Evidence on the effects of details on picture recognition is also equivocal in suggesting an effect of colors on picture recognition memory. Nelson, Metzler, and Reed (1974) presented subjects with three types of picture representing the same scenes: black-and-white photographs; complex, embellished line drawings; simple, unembellished line drawings. They obtained no difference in recognition performance among these three types. They argued that the information of details of the pictures was not stored, and therefore did not facilitate recognition of the scenes. However, Loftus and Bell (1975) reported that details affected the recognition performance of pictures. Performance for photographs was better than for simple and complex line drawings of the same scenes. Loftus and Kallman (1979) also showed that recognition performance for pictures was better when the information of details was encoded at the study and used at the test. In the present study, therefore, colors, which are thought to be a kind of extra, detailed information about the scenes, should facilitate recognition of the pictures. In the present study, we examined how color information from the scenes is used in the encoding and retrieval processes. We used naturalistic scenes as stimulus materials, and minimized the possibility of verbal mediation in recognition memory. In the study phase, subjects memorized pictures of the same scenes presented in either color or black and white. In the test phase, they were presented with pairs of test pictures, one study scene and one distractor scene. Test pictures were in either the same or different color mode from the study ones. Therefore, there were four sets of study/test color mode: color/color, color/blackand-white, black-and-white/color, and blackand-white/black-and-white. If colors are effective only in memorization, the performance in the color/color and color/ black-and-white sets should be better. If color information is effective in both encoding and retrieval, the performance in the color/color set should be the best. If colors have no effect in picture recognition memory, performance should not differ among the four sets. If an advantage of a match between features of the stimuli at the study and the test (Roediger & Blaxton, 1987), which is supposed to be a kind of encoding specificity (Tulving & Thompson, 1973), exists in picture recognition too, better performance should be obtained in the black-and-white/black-and-white set as well as in the color/color set. Additionally, we examined how well the subjects remembered the color mode of the study pictures. In each trial, after answering which picture had been the study one, the subjects also answered whether the picture they chose as the study scene had been colored or black and white in the original study. It was expected that the performance pattern of the recall for the color mode of pictures would be similar to that of the recognition for the pictures. That is, if an effect of color could be found in any of the color mode sets, the recall for the color mode of the pictures also would be better in such sets. An additional prediction may also be made. Pezdek, Maki, Valencia-Laver, Whetstone, Stoeckert, and Dougherty (1988) had their subjects memorize simple and complex drawings in the study phase, and asked them whether or not the test drawing, the complexity of which was either the same as or different from that of the study drawing, had been the study one. As in our experiments, four study/test sets of simple and complex drawings were used: simple/simple, simple/complex, complex/simple, and complex/complex. Performance was worse in the complex/simple sets than in the simple/complex sets. They called this phenomenon the asymmetric confusability effect (which we suggest is not necessarily an appropriate term), and interpreted it in the following way. The drawings were schematically encoded, so the resultant memory representation of both simple and complex drawings was close to the simple version of the drawing. The information about the details in the complex study drawings was easy to recognize in the

3 Picture recognition memory 27 same complex test drawings, but was difficult to retrieve when the complex study drawings were changed to simple test drawings. Therefore, if the colored pictures used in our study can be said to be more detailed than the black-and-white drawings, a similar result to Pezdek et al. s would be obtained: recall performance for color mode would be worse in the color/black-and-white set than in the blackand-white/color set. Also, we introduced two retention interval conditions: the immediate and 1-week-delayed test conditions. We examined how any effects of color on memory performance would vary over the retention interval. Manipulating the retention interval also had another purpose. In the immediate test, recognition performance for pictures would be expected to be high (e.g., Haber, 1970; Nickerson, 1965; Shepard, 1967), so any effect of color might be obscured. A lapse of 1 week between the study and test would lower the overall performance and might make a color effect more pronounced. Method Subjects The subjects were 48 undergraduate students at Niigata University. All had normal color vision and normal visual acuity. Materials and apparatus The 240 natural scenes selected as stimulus materials were presented on 35-mm slides. The scenes had been photographed on color negative films; the films were then printed on both color negative films and black-and-white negative films, to make color and black-andwhite slides. These were scenes of university campuses, rail stations, airports, parks, and city streets, rural landscapes, and so on. All were of locations remote from the area where most of the subjects lived, and therefore were deemed to be unfamiliar. Pictures including proper names were not used, to avoid any verbal mediation in memory, nor did pictures include persons in the foreground, because they would be expected to aid recognition performance. One hundred and twenty scenes were study scenes, and the other 120 were used as distracter scenes during the test. In the test phase, the study pictures were presented paired with the distracter pictures, and a given study picture was paired consistently with the same distracter picture. In order to strengthen the effect of distracter pictures on recognition and to lessen the effect of verbal mediation, we paired pictures by motif. In consequence, the study and test pictures were similar. Examples of the picture pairs used in the experiment are shown in Figure 1. Pictures were rear-projected onto a screen, 90 cm 90 cm, by slide projectors (Kodak Ektagraphic II). The screen was 57.5 cm away from the subjects. The subjects fixed the position of their head with a chin-rest, the center of the screen being at eye level. The image size of a projected picture was in visual angle. During the test, pairs of pictures were presented side by side, 5 cm apart. Presentation time and a dark interval between presentations were regulated by electrical shutter drivers (TKK-270, Takei Scientific Instruments). Design and procedure Retention interval was the between-subjects variable. There were two retention interval groups, i.e., immediate and 1-week-delayed test groups. Half the 48 subjects were assigned to the former, and the other half to the latter group. Color sets were the within-subjects variable. In the study phase, 60 colored and 60 blackand-white pictures were presented. In the test phase, half the test pictures were presented in the same color mode as the study pictures, and the other half in the different one. That is, each subject was provided with four color sets: color picture at both study and test (C/C), color pictures at study and black-and-white pictures at test (C/BW), black-and-white pictures at study and color pictures at test (BW/C), and black-and-white pictures at both study and test (BW/BW). Subjects performed individually in both phases. In the study phase, the subjects were asked to memorize the scenes successively presented on the screen for a later memory test.

4 28 K. Suzuki and R. Takahashi Figure 1. Two examples of picture pairs used in the experiment. One picture in each pair was the study picture, and the other the distracter picture. Which was used for the study picture or which was used for the distracter was changed from subject to subject. The 120 (60 colored, 60 black-and-white) were intermingled and presented in random order, each for 5 s, with a dark interval of 2 s. In the immediate test group, the test was given 5 min after the presentation of the last study picture. In the 1-week-delayed test group, the test was given 7 days later. In the test phase, the subjects were presented with pairs of test pictures, one study and one distracter, shown side by side on the screen. Each test pair was in the same color mode. The subjects were asked to answer which (left or right) was identical to any of the scenes among the study pictures, ignoring color mode. They were told that the color mode of the pictures would sometimes differ, such that color pictures presented in the study phase might be presented as black-andwhite ones in the test phase, and vice versa. Additionally, they were asked whether the test picture they recognized as the study scene had originally been colored or black and white. Pairs of test pictures were presented successively, each for 10 s with a dark interval of 2 s. The subjects were instructed to answer within 12 s. At the test, 120 pairs of test pictures (30 pairs for each color set) were intermingled and presented in random order. The four possible study/test color sets for each test pair were

5 Picture recognition memory Table 1. Percentage of recall for color mode of the pictures correctly recognized Study/test color set Percent recognition Figure 2. Immediate test used equally across the subjects. The left-right position of the study pictures in the test pairs was randomly arranged. Results C/C C/BW BW/BW BW/C 1-week-delayed test Percentage of recognition for pictures in four color sets as a function of retention interval. Figure 2 shows mean percentage of correct recognitions for pictures in each color set plotted as a function of retention interval. The overall performances were 90% and 78% for the immediate and 1-week-delayed recognitions. Performance in the C/C set was the highest among the four, being 5% higher in the immediate test and 10% higher in the 1-weekdelayed test. A three-way analysis of variance, color sets retention intervals subjects, was performed on the arcsine-transformed percent recognition. Differences in performance among the four sets and between the retention intervals were significant, F(3,138) = 15.28, p.001 and F(1,46) = 37.89, p.001, respectively. There was no interaction between the color sets and retention intervals, F(3,138), ns. In the 1-week-delayed test, the recognition performance in the C/C set was significantly C/C C/BW BW/C BW/BW Immediate week-delayed Note. C/C, color/color; C/BW, color/black-and-white; BW/C, black-and-white/color; BW/BW, black-and-white/ black-and-white. higher than that in other sets, p.01, p.05, and p.05, Tukey s test, for the BW/C, C/BW and BW/BW set, respectively, and the differences in performance among the latter three sets were not significant. In the immediate recognition test, the differences among the color sets did not reach a significance. Recognition performance declined significantly over the retention interval in every color set, p.01 in all cases, using Tukey s test. Mean percentage of correct recalls for the color mode of the study pictures that the subjects correctly recognized is shown in Table 1. In the immediate test, the subjects answered the color mode 70% correctly. In the 1-weekdelayed test, the performance was only 55%, on average. Performance in the C/BW set was the lowest among the four: in the immediate test, the performance was 10% lower than those of the other three sets, and it was even 6% below a chance level in the 1-week-delayed test. An analysis of variance revealed that the color sets and retention intervals significantly affected performance, F(3,138) = 5.14, p.01 and F(1,46) = 56.85, p.001, respectively. There was no interaction between color sets and retention intervals, F(3,138) = 1.35, ns. A multiple comparison test was performed on the differences among color sets. No difference was found in the immediate test, whereas in the 1-week-delayed test the performance in the C/BW set was significantly lower than those in the C/C and BW/BW sets, p.01 and p.05, respectively, using Tukey s test. Performance declined significantly over the retention interval in the C/BW, BW/C and BW/BW sets,

6 30 K. Suzuki and R. Takahashi p.01, p.01 and p.05, respectively, using Tukey s test. Discussion Recognition performance for pictures in the C/C set was the highest of the four. Thus, color information was indeed effective in both encoding and retrieval. This tendency was not so evident in the immediate recognition, probably because the overall recognition performance was so high, i.e., because of the so-called ceiling effect. In the 1-week-delayed recognition, in which forgetting occurred, the tendency became pronounced. The other three color sets showed similar performance to each other. As expected from Loftus and Bell s (1975) and Loftus and Kallman s (1979) studies on the effect of details on picture recognition, colors did increase recognition performance. However, there was no effect of color on picture recognition when colors were present either in the study or test picture only. Thus, colors were effective if, and only if, they were present during both encoding and retrieval. What produced such good recognition performance for pictures in the C/C set? If memory of the colors of certain objects in the pictures had directly produced the highest recognition performance for pictures, the subjects should have shown the highest performance also in the recall for the color mode of the study pictures in the C/C set. However, this was not the case: the subjects did not remember the color mode of the study pictures so well, even in the C/C set. The recall performance for the color mode in the C/C set was nearly the same as that for the BW/BW set. Therefore, the highest recognition performance for pictures in the C/C set could not be attributed to the subjects memory for the colors in the pictures themselves. Rather, such good performance might be induced by indirect effects. Colors could highlight certain features in the study and test pictures, so that the distinctiveness of these features enabled the pictures to be encoded and retrieved more easily. By contrast, the features of the black-and-white pictures were not as distinctive, and might therefore have been more difficult to encode and retrieve. This explanation is also suggested in the following studies. Jacoby and Craik (1979) have shown that the distinctiveness of stimuli can produce better performance of recognition and recall if it is present also in the test. Similarly, Loftus and Kallman (1979) showed that recognition performance for pictures was better when information concerning detail was encoded at the study and was retrieved at the test. On the other hand, in the recall of color mode of the study pictures, the overall performance was poorer than expected, being 70% even in the immediate test, and declining to only 55% in the 1-week-delayed test. This suggests that the color mode was hard to remember. This result agrees with Park and Mason s (1982) finding that encoding and retrieving of colors are not automatic processes but rather ones that require effort. Thus, although, as mentioned above, color could facilitate encoding and retrieval, the presence of color was often not remembered. Furthermore, the recall performance pattern for the color mode was quite different from that for recognition of the pictures. In particular, when the test pictures were presented in a different color mode from the study ones, the recall performance for color mode tended to be lower. This result suggests that the subjects were influenced more by the mode of the test pictures presented than by the actual mode of the study pictures. This agrees with Belli s (1988) and Loftus (1977) findings that misleading post-event information about colors led subjects to incorrect recognition and recall for the colors of the objects in the pictures. This suggests that colors in picture memory can be quite labile to post-event information. In the present experiment, presenting the test pictures in a different color mode from the study ones is similar to adding misinformation in Belli s and Loftus experiments. Thus, our results also suggest a lability of color memory. Moreover, the recall performance for the color mode was not symmetric between C/BW and BW/C sets. The subjects made more errors in the C/BW set than in the BW/C set, even in the immediate test. Moreover, in the

7 Picture recognition memory 31 1-week-delayed test, the recall performance in the C/BW set was even below chance level. Thus, in most cases, the subjects were less likely to notice that colors were dropped from the colored study pictures at the test than to notice that colors were added. This result is similar to Pezdek et al. s (1988) asymmetric confusability effect. In their experiments, the information about detail was difficult to retrieve when complex study drawings were changed to simple test drawings. A few studies have reported similar effects. Agostinelli, Sherman, Fazio, and Hearst (1986) found that it was more difficult to detect and identify the deletion of some features from line drawings than to detect and identify their addition. Healy (1981) reported a similar effect of added and missing features of letters on proofreading. Maki (1989) found the same effect in the script memory for action. Thus, it is more difficult to notice deletions than additions. Our results can be similarly interpreted. This asymmetric confusability effect of color suggests the following. In the color pictures, although color could help both encoding and retrieving by highlighting other features, the information of color itself was often unavailable. Therefore, when the color study pictures were changed to black-and-white at the test, the subjects often made errors in remembering the original color mode: they often took the study pictures for black-and-white. In contrast, when colors were added to the black-andwhite pictures at the test, they were more likely to notice the change. In conclusion, it can be said that colors are effective in picture recognition only when they are present in both study and test pictures. The memory for the color mode of the pictures, however, was quite poor and labile, suggesting that colors themselves might not improve the recognition performance for pictures, but might improve it by strengthening the distinctiveness of other features. References Agostinelli, G., Sherman, S. J., Fazio, R. H., & Hearst, E. S. (1986). Detecting and identifying change: additions versus deletions. Journal of Experimental Psychology: Human Perception and Performance, 12, Belli, R. F. (1988). Color blend retrievals: compromise memories or deliberate compromise responses? Memory and Cognition, 16, Bousfield, W. A., Esterson, J., & Whitmarsh, G. A. (1957). The effects of concomitant colored and uncolored pictorial representations on the learning of stimulus words. Journal of Applied Psychology, 41, Boynton, R. M., & Dolensky, S. (1979). On knowing books by their colors. Perceptual and Motor Skills, 48, Denis, M. (1976). Test of the incidental-cues hypothesis. Perceptual and Motor Skills, 43, Haber, R. N. (1970). How we remember what we see. Scientific American, 222, Healy, A. F. (1981). The effects of visual similarity on proofreading for misspelling. Memory and Cognition, 9, Jacoby, J. L., & Craik, F. I. M. (1979). Effects of elaboration of processing at encoding and retrieval: trace distinctiveness of recovery of initial context. In L. S. Cermak & F. I. M. Craik (Eds.), Levels of processing in human memory (pp. 1 21). Hillsdale, NJ: Lawrence Erlbaum and Associates. Loftus, E. F. (1977). Shifting human color memory. Memory and Cognition, 5, Loftus, G. R., & Bell, S. M. (1975). Two types of information in picture memory. Journal of Experimental Psychology: Human Learning and Memory, 1, Loftus, G. R., & Kallman, H. J. (1979). Encoding and use of detail information. Journal of Experimental Psychology: Human Learning and Memory, 5, Maki, R. (1989). Recognition of added and deleted details in scripts. Memory and Cognition, 17, Nelson, T. O., Metzler, J., & Reed, D. A. (1974). Role of details in the long-term recognition of pictures and verbal description. Journal of Experimental Psychology, 102, Nickerson, R. S. (1965). Short-term memory for complex meaningful visual configurations: a demonstration of capacity. Canadian Journal of Psychology, 19, Paivio, A., Rogers, T. B., & Smythe, R. C. (1968). Why are pictures easier to recall than words? Psychonomic Science, 11, Park, D. C., & Mason, D. A. (1982). Is there evidence for automatic processing of spatial and color attributes present in pictures and words? Memory and Cognition, 10,

8 32 K. Suzuki and R. Takahashi Pezdek, K., Maki, R., Valencia-Laver, D., Whetstone, T., Stoeckert, J., & Dougherty, T. (1988). Picture memory: recognizing added and deleted details. Journal of Experimental Psychology: Learning, Memory, and Cognition, 14, Roediger, H. L., III, & Blaxton, T. A. (1987). Effects of varying modality, surface features, and retention interval on word fragment completion. Memory and Cognition, 15, Shepard, R. N. (1967). Recognition memory for words, sentences, and pictures. Journal of Verbal Learning and Verbal Behavior, 6, Tulving, E., & Thompson, D. M. (1973). Encoding specificity and retrieval processes in episodic memory. Psychological Review, 80, Watkins, M. J., & Schiano, D. J. (1982). Chromatic imaging: an effect of mental coloring on recognition memory. Canadian Journal of Psychology, 36, (Received August 31, 1994; accepted July 1, 1995)