Gathering and Repetition of the Elements in an Image Affect the Perception of Order and Disorder

Transcription

1 International Journal of Affective Engineering Vol.13 No.3 pp (2014) ORIGINAL ARTICLE Gathering and Repetition of the Elements in an Image Affect the Perception of Order and Disorder Yusuke MATSUDA and Hirohiko KANEKO Interdisciplinary Graduate School of Science and Engineering, Tokyo Institute of Technology, G2-3, 4259 Nagatsuta, Midori-ku, Yokohama , Japan Abstract: When looking at visual patterns, we receive various impressions such as goodness, complexity, and beauty. This study was conducted to identify the stimulus factors giving rise to impressions of order and disorder. We presented two visual patterns simultaneously with manipulation of the stimulus factors of gathering. Participants reported which pattern was more disorderly. Gathering index was defined to indicate the degree of gathering of elements in the pattern. Results showed that the disorder ranking value calculated from the responses increased as the value of gathering decreased. Results also showed that the disorder ranking value decreased as the number of local repetition in the whole stimulus increased only when the gathering value was small. From the results of present study, we conclude that factors of gathering and repetition affect impression of order and disorder. We suggest that the gathering value indicates whether there are any objects or not in the pattern and the repetition value indicates whether there are any texture or not. Objects and textures seem to be the important concept to analyze or recognize visual patterns for human. Therefore, we can suppose the impression of order and disorder tells us whether there is meaningful information in visual patterns. Keywords: Perception of Order/Disorder, Gathering, Repetition 1. INTRODUCTION We perceive some visual properties such as brightness, color, and size when looking at visual patterns. Similarly, we receive various impressions from the patterns such as goodness, complexity, and disorder. Fundamentally, each visual property has a corresponding physical quality: brightness bases on the physical energy of light from the object, color perception reflects the outputs of three cones on the retina produced by the spectral distribution of light, and the mechanisms for perceived size uses the area occupying visual field by the object as the initial information. In contrast, identifying a physical quality responsible for impressions such as those described above is difficult. Many researchers have sought to identify physical qualities related to some visual impressions. Koffka [1], a Gestalt scientist, claimed that when people see visual patterns, they use some laws depending on the physical features of the patterns, such as proximity, similarity, and good continuity, for extracting figural goodness, called Gestalt lows. Many researchers think the Gestalt laws were the roots of emotion or feeling (designated as impressions herein) for visual patterns. After Koffka s proposal, many scientists conducted experiments to identify the physical qualities responsible for the impression of visual patterns because Koffka s proposal described only the qualitative features of visual patterns. Garner and Clement [2] proposed the Equivalent Set Size (ESS) as the physical quality that determines the goodness of visual pattern. The ESS was defined as the number of different patterns that are producible from an original image using the transformations of mirror reflection and 180-deg rotation. For example, the ESS of a circle is one. That of a pattern comprising irregular random dots is eight. They claimed that if the ESS value of a pattern is small, then the pattern had high goodness. Moreover, they asserted that ESS could account for about 70% of the goodness rating for visual patterns. Yodogawa [3] proposed Symmetropy, defined by calculating the entropy of a pattern using a two-dimensional Walsh function, as an index to predict the goodness of a pattern. If the Symmetropy of a pattern is high, then the pattern has higher redundancy and low goodness. He succeeded in predicting the goodness of visual patterns accurately. Chipman [4] and Ichikawa [5] reported that the physical quantities that had been implicated as indicators responsible for impressions are divisible into two groups: quantitative factors and structural factors. Quantitative factors depend on the total number of elements such as turns. Structural factors depend on the features of patterns, such as symmetry. Especially, Ichikawa showed that a linear combination of the two factors can predict the complexity of visual patterns. Moreover, he concluded that the individual difference in an impression for the same pattern resulted from the difference in weights between two factors among observers. Received Accepted Copyright 2014 Japan Society of Kansei Engineering. All Rights Reserved.

2 International Journal of Affective Engineering Vol.13 No.3 Recently, many scientists have come to examine the impressions processed in a high level of the brain specifically, such as aesthetic preference or beauty. Reportedly, the size and color of an element object are responsible for giving some impression [6-9]. Furthermore, Stevanov et al. showed that visual illusion has valence for aesthetic impressions [10]. Although many scientists have insisted that some stimulus factors were responsible for some impressions as described above, one of the problems in this stream of research is that most of the treated impressions are limited to beauty and goodness. As the results, the proposed relations among them were qualitative in many cases because many difficulties arose in quantifying the stimulus factors for such impressions processed at a higher level. These impressions were affected by many factors such as experience, ethics, culture, and individual differences. Therefore, it seems important to choose a simple impression and to assess the quantitative relation between the impression and some stimulus factors to elucidate the mechanism, which produces impressions in the human visual process. For this study, we chose the impression of order and disorder as a simple impression. We strove to identify a quantitative relation between the factors in the pattern and the impression. The impression of order and disorder seemed important for us because it seems resemble the factor randomness used in information theory that is an index of the information amount [11]. We expected that this impression reflected the amount of useful information for humans in the visual pattern. In other words, we regarded the impression of order and disorder as representing whether important information for humans exists in the pattern or not. repetition, which was defined as the number of repeated patterns in the pattern. 2.1 Methods Participants Nine participants (eight men, one woman) aged years old and nine men aged joined the experiment A and B (explained later), respectively. Two and one of them were authors in these experiments, respectively. The others were naive to the purpose of this study. All had normal or corrected to normal vision Stimulus We used visual stimuli comprising 4 4 square-areas. Half of the areas (eight areas) chosen randomly had white dots on the black background. The other areas were uniformly black (Figure 1). We calculated the gathering and repetition indices described above for the stimulus as stated in the next two sections. Gathering index First, we calculated the square of the difference between the number of dots in each 2 2 area and the average number across all the 2 2 areas in the pattern: the variance of dot number in each 2 2 area (Figure 2(a)). The average of dots in 2 2 area was always two because all stimuli had eight dots in 16 unit squares. This calculation of variance was performed for all possible 2 2 areas including the cases containing at least one unit square (Figure 2(b) and (c)). For the case in which 2 2 area contained the area outside of the pattern, we regarded the area as having 0.5 white dots (Figure 2(c)). In consequence, we calculated the variance in 25 areas for each stimulus. Nine of the areas contained all of the four unit squares in the 4 4 pattern (Figure 2(b)), 12 of 2. EXPERIMENT This study was conducted to identify the factors in visual pattern determining the impression of order and disorder. We focused on two factors related to the positional arrangement of elements in the pattern. One of the factors was the degree of gathering of elements in the pattern. We quantified this factor using an index called gathering, which was defined as the averaged deviations of element number in small unit areas over the whole pattern. The factor is similar to the proximity used in the Gestalt law [1], which meant that elements were regarded as one group when they were mutually proximate. Another factor was regularity of element arrangement in the pattern. We quantified this factor using an index called Figure 1: Examples of stimulus patterns (a) of all possible value of gathering, (b) in experiment A and (c) in experiment B. The value shows gathering index for each stimulus. 168

3 Gathering and Repetition of the Elements in an Image Affect the Perception of Order and Disorder International Journal of Affective Engineering Vol.13 No.3 them contained two in the pattern and two in the outside, and four of them were at the corners and contained just one unit square in the pattern and three in the outside (Figure 2(c)). Finally, we calculated the sum of variances for all 2 2 areas, which is the gathering index of one stimulus. Repetition index We define the repetition index as the sum of the numbers of different patterns when whole pattern is divided into parts. Figure 3 shows, schematically, the way to define the repetition index. For example, when the 4 4 pattern presented in Figure 3(a) is divided into two vertical areas of 2 4, the patterns in the two areas are the same (one Figure 2: Schematic representation of the definition of gathering. See text for the details. Figure 3: Schematic representation of definition of repetition. (a) Example of the pattern. (b) The way to calculate the repetition index. See text for the details. kind of pattern) (Figure. 3(b) top). In this case, we define the repetition index of this whole pattern with this mode of division as one. We regarded the rotated patterns and mirror symmetric patterns as the same pattern. When the same 4 4 pattern is divided into two horizontal areas of 4 2, the patterns in the two areas differ (two kinds) so that the repetition index with this division is two (Figure 3(b) middle). Moreover, when the pattern is divided into four squares of 2 2, the patterns in the four areas are two kinds, so that the repetition index with this division is two (Figure 3(b) bottom). We define the weighted sum of the repetition indices for the ways of division described above as the index of the whole pattern. We assigned the weight to the index proportional to the size of the unit area because we regarded the effect of repetition in a larger area as greater. Specifically, we assigned weight of two to the repetition index for 4 2 and 2 4 divisions and assigned weight of one to the repetition index for 2 2 division. For the pattern depicted in Figure 3(a), the final value of repetition index becomes eight. With this definition of repetition index, the value becomes smaller when the pattern includes many local areas with the same pattern (repetition). Stimulus conditions Figure 1(a) shows the whole range of gathering value possible for the 4 4 pattern used. We divided the whole range of gathering value into two groups and conducted two experiments for each group (experiment A and B) because we wanted to investigate the effect of small change of gathering value on the perception of order and disorder. In addition, the number of stimulus condition in one experiment should be relatively small because we used the method of paired comparison for measuring perceptual responses as stated later. Figure 1(b) shows patterns having the gathering values used in experiment A. We used stimuli of five kinds having different gathering index values of 13, 18, 23, 28, and 30.5 (Figure 1(b)). For each gathering value of the stimulus, there were five different patterns. The stimulus with a gathering value of 30.5 included patterns with the value of 30 and 31 half and half because varieties of the pattern with the gathering value of 30 or 31 could not be made more than 10. We used two sets of patterns, Group 1 and Group 2 (each set consisted of 5 patterns 5 gathering values), to reduce the particular effects for specific pattern. Group 1 had three stimuli with gathering value of 30 and two with gathering value of 31. Group 2 had opposite ratio of stimuli for gathering value of 30 and

4 International Journal of Affective Engineering Vol.13 No.3 Figure 1(c) shows patterns having the gathering values used in experiment B. We used five patterns for each stimulus having a gathering value of 3.5, 5, 7, 10 or 13 (Figure 1(c)). The patterns with a gathering value of 3.5 actually included the patterns with the value of 3 and those with 4 because varieties of the pattern with the gathering value of 3 and 4 were less than 10. Luminance of the white dots and black background were, respectively, 21.7 [cd/m 2 ] and 0.34 [cd/m 2 ]. The visual angle of the whole stimulus pattern (4 4 square) was 15.0 [deg] 15.0 [deg]; that of the white dot was 1.8 [deg] in diameter. Stimuli were produced using self-made software with a PC (Power Book Pro; Apple Computer Inc.) with an editor/compiler (x-code ver ; Apple Computer Inc.) and a library for psychological experiments (Psychlops ver1.5.6) Procedure Participants sat in front of a desk with a chin-rest, put their chin on it and observed a display on the desk. The visual distance was 43 [cm]. A sequence of stimulus presentation in one trial is depicted in Figure 4. First, two patterns placed side by side were presented simultaneously for 2500 [ms]. Then, the mask patterns consisted of randomly placed squares were presented for 500 [ms]. Finally, the blank squares were presented. Participants reported which pattern was more disorderly using a keyboard. They were able to respond whenever the pattern, mask or blank was presented but they were required to respond within 2000 [ms] from the onset of blank pattern. A beep sounded and the trial was terminated if a participant did not respond during that period. The stimulus was presented again in a later trial. The two mutually differing visual patterns presented in a trial were chosen from 25 stimuli. A block comprised Figure 4: Sequence of stimulus presentation in an experimental trial. 300 trials, which included all possible combinations from 25 patterns. Three blocks were used for each group of stimulus set (Figure 1(b) and (c)) for each participant. The participants took six blocks as a whole. Participants also took a preliminary experiment consisting of 45 trials to get used to making response before the main experiment. In the preliminary experiment, the stimuli patterns differed from those used in the main experiment, but the gathering values were the same as those in the main experiment (13, 18, 23, 28 and 30.5 or 3.5, 5, 7, 10 and 13 in experiment A and B, respectively). 2.2 Results and Discussion We considered that in this experiment, understanding the purpose did not matter for making the response and included the data of authors to the results because there was no difference between the trends in the data of authors and those of naive participants. Figure 5 shows the averaged relation between the impression of order and disorder and the gathering index across nine participants. The horizontal axis shows the value of gathering. The vertical axis shows the disorder ranking calculated using the method of Thurstone s Paired Comparison that presented psychological distance of the patterns for the impression of order and disorder. We regarded the three responses for the same stimulus pair in different experimental blocks made by the same participant as independent responses for the Thurstone s Paired Comparison method. The gray symbol represents the result for each pattern. The black symbol shows the average across patterns for each gathering value. Left and right panels respectively show the results for stimulus Group 1 and those for stimulus Group 2. Upper and lower panels respectively show the results for experiment A and B. The responses for each pattern were accumulated in three blocks for each participant. The result presented in Figure 5 in experiment A (upper panels) clearly illustrates that the impression of disorder decreases as gathering value increases and that the relation is apparently more or less linear (correlation coefficient: r = (Group 1), (Group 2), respectively). It is readily apparent that the trends for different stimulus groups were the same. On the other hand, the result of experiment B (lower panels) does not show the same trend as in experiment A. The function is not monotonic and seems to have a peak around the gathering value of five. The disorder ranking decreases as the value of gathering increases when the gathering value is greater than five. This trend is consistent with that in experiment A. However, when the gathering value is less than five, it is 170

5 Gathering and Repetition of the Elements in an Image Affect the Perception of Order and Disorder International Journal of Affective Engineering Vol.13 No.3 clear that the disorder ranking increases as the gathering value increases. This fact implies that another factor is expected to affect the impression of order and disorder for gathering value less than five. From the results of this experiment, we were able to conclude that the gathering is at least one of the important factors used to decide the impressions of order and disorder. If the gathering value is high, namely if the white dots are gathered in one or a few area, then the pattern is perceived as more orderly. Figure 6 shows the relation between the impression of order and disorder and the repetition value of the pattern. Horizontal and vertical axes respectively show the repetition value and the disorder ranking. The left two panels show the results of stimulus Group 1. Right panels show those of stimulus Group 2. Upper two panels show the results in experiment A and lower panels show the results in experiment B. For data in experiment A (upper panels), the correlation coefficient between the disorder ranking and repetition value was low: r = 0.10 (Group 1), (Group 2). For data in experiment B (lower panels), the correlation coefficient between the disorder ranking and repetition value was also low: r = 0.46 (Group 1), 0.26 (Group 2). These results might indicate that repetition does not affect the impression of order and disorder. However, it appears that disorder ranking in the range of small gathering increases as gathering value increases. To show the effect of repetition on the perception of order and disorder in the range of small gathering value more clearly, we re-plotted only the data of gathering value less than six in Figure 7. The correlation coefficients between them were quite high: r = 0.92 (Group 1), 0.93 (Group 2). These results imply that for the patterns with ultimately low gathering value, repetition is an important factor to decide the impression of order and disorder. 3. GENERAL DISCUSSION All participants were able to judge order and disorder of the visual pattern stably and confidently throughout the experiments, which indicates that order and disorder is a stable impression when measured within and among subjects. In experiment A, in which the patterns with intermediate magnitude of gathering value were used, the results showed that the impression of disorder decreased as the value of gathering increased. There was no pattern producing a large deviation of the response from the trend. These Figure 5: Result of experiment as a function of gathering value. Figure 6: Result of experiment as a function of repetition value. Figure 7: Re-plotted results of experiment B as a function of repetition value in the range of gathering value less than six. facts indicate that gathering is one of the important factors to decide the impression of order and disorder. In experiment B, in which the patterns with small magnitude of gathering value were used, the results showed the same trend as in experiment A if the gathering value was relatively large but showed a completely different trend if the gathering value was extremely small. We investigated the effect of another factor, repetition, and found it to be a determinant for the impression of order and disorder. However, the effect is smaller and more attributive than that of gathering. Moreover, for this 171

6 International Journal of Affective Engineering Vol.13 No.3 study, we used only three unit areas of 2 2, 2 4, and 4 2 to calculate the repetition index, so that additional experiments must be conducted to ascertain the best way to define the factor of repetition. The two factors, gathering and repetition, indicated by the present result to be responsible to decide the impression of order and disorder might be corresponding to the quantitative and structural factors proposed by Ichikawa [5]. He showed that the perception of complexity could be quantified by combining quantitative factors and structural factors linearly. The quantitative factors indicate the strength of elements such as number of turns and the structural factors depend on the composition of patterns, such as symmetric structure. The factor of gathering shows the degree of uniformity of the element distribution in this study which is a quantitative index. On the other hand, the factor of repetition shows the similarity of local pattern, which seems to be a structural factor. If this correspondence would be valid, the individual difference in the response of order and disorder could be due to the difference in the weights of the factors as claimed by Ichikawa [5] for the perception of complexity. We suppose that there could be other definitions of gathering. The definition of gathering could be based on the area of an element, although, in this study, we defined gathering based on the number of elements (dots) in a subunit. We suppose both of the definitions of gathering are possible. Some parts can gather to form one uniform area regarded as one object and some similar elements can gather to form one group. In this study, we only consider the latter case, gathering as a group. We need to know the property of the other gathering based on the area of elements to understand fully the perception of order and disorder from the value of gathering. It would be possible to extend the concept of gathering to visual stimuli comprising N N square-areas, although, in this study, we used only 4 4 pattern and 2 2 subunit for defining gathering index. In the case of N N square-areas, however, some modifications must be needed for the definition of gathering. For example, when we see a stimulus comprising square-areas, the subunit to define gathering index could be so many from 2 1 to Then, it becomes difficult to decide the size of subunit. It could depend on the size of stimulus or on the size of visual angle. In addition, we might be able to change the size of subunit intentionally for recognizing the gathering when they observe complex stimuli. We need to investigate the relationship between the perception of order and disorder and the sizes of the stimulus and subunit to define gathering for answering the question. Present results showed that the sensation of order and disorder was affected by, at least, two different factors, gathering and repetition. We speculate that this fact suggests that these factors indicate degree of common information, which is the meaningfulness of pattern. One of the meaningful things in the pattern would be object. When we observe the real world, we usually try to see objects such as tree, car, post, table and so on. Most objects are composed of same or similar luminance so that existence of object in the scene increases the degree of gathering. Another meaningful thing in the pattern would be texture. Texture is important because it tells us the properties of surface, such as the materials. Textures, such as wallpaper, fabric and natural ground surface usually have repeated pattern so that existence of texture increases the degree of repetition. Our visual system might produce the impression of order and disorder to inform us whether the pattern contains meaningful things, object and texture, or not, based on two factors, gathering and repetition. Although this idea is only a speculation at this point, it would be attractive we suppose. Present study shows that people regard the areas with high gathering value or those with high repetition value to be orderly. This relationship can be applied to control quantitatively the impression of real space, picture and so on by manipulating the indices of gathering and repetition. It can also be applied to obtain an objective measure indicating the degree of messiness in the room with these factors. ACKNOWLEDGEMENTS We deeply thank Dr. Kazuho Fukuda and Dr. Makoto Inagami for the help and comments in the early stage of the present study. REFERENCES 1. K. Koffka; Principle of Gestalt Psychology, Harcourt- Brace, New-York (1935). 2. W.R. Garner, D.E. Clement; Goodness of pattern and pattern uncertainty, Journal of Verbal Learning & Verbal Behavior, 2(5-6), pp (1963). 3. E. Yodogawa; Symmetropy, An entropy-like measure of visual symmetry, Perception & Psychophysics, 32(3), pp (1982). 4. S.F. Chipman; Complexity and structure in visual patterns, Journal of Experimental Psychology: General, 106(3), pp (1977). 5. S. Ichikawa; Quantitative and structural factors in the judgment of pattern complexity, Perception & 172

7 Gathering and Repetition of the Elements in an Image Affect the Perception of Order and Disorder International Journal of Affective Engineering Vol.13 No.3 Psychophysics, 38(2), pp (1985). 6. A.C. Hurlbert, Y. Ling; Biological components of sex differences in color preference, Current Biology, 17(16), pp (2007). 7. T. Konkle, A. Oliva; Canonical visual size for realworld objects, Journal of Experimental Psychology: Human Perception and Performance, 37(1), pp (2011). 8. S. Linsen, M.H. Leyssen, J. Sammartino, S.E. Palmer; Aesthetic preferences in the size of images of realworld objects, Perception, 40(3), pp (2011). 9. S.E. Palmer, K.B. Schloss; An ecological valence theory of human color preference, Proceedings of the National Academic of Sciences of the United States of America, 107(19), pp (2010). 10. J. Stevanov, S. Marković, A. Kitaoka; Aesthetic valence of visual illusions, i-perception 3(2), pp (2012). 11. C.E. Shannon; A Mathematical Theory of Communication, The Bell System Technical Journal, 27, pp , pp (1948). Yusuke MATSUDA Yusuke Matsuda is a Ph. D. candidate of Department of Information Processing in graduate school of Tokyo Institute of Technology. He obtained a Bachelor of Engineering from Tokyo University of Science in 2009 and a Master of Engineering from Tokyo Institute of Technology in He is engaged in impression formation and pattern perception. Hirohiko KANEKO Hirohiko Kaneko is an Associate Professor of Department of Information Processing at Tokyo Institute of Technology. He obtained his Bachelor of Science (applied physics) in 1987 and Dr. (Eng.) (information processing) in 1992 from Tokyo Institute of Technology. After obtaining the degree, he worked at York University in Canada and ATR Human Information Processing Research labs. in Kyoto. Then he moved to Tokyo Institute of Technology in His current research interests are binocular vision, stereopsis, eye movement, pattern perception and multisensory information integration. 173