Spatial Distortions in Visual Short-term Memory: Interplay of Intrinsic and Extrinsic Reference Systems

Transcription

1 SPATIAL COGNITION AND COMPUTATION, 4(4), Copyright 2004, Lawrence Erlbaum Associates, Inc. Spatial Distortions in Visual Short-term Memory: Interplay of Intrinsic and Extrinsic Reference Systems Thomas Schmidt University of Göttingen Spatial short-term memory for single target positions is subject to distortions which depend on the spatial layout of visual landmarks. Here, participants had to reproduce the positions of briefly presented targets in the context of three-landmark configurations presented in various orientations. Symmetry properties of distortional patterns were determined by the intrinsic reference system of the landmark configuration as well as by the environment- or body-centered vertical axis. Symmetry was best about the cardinal axes of the landmark system irrespective of their orientation, but symmetry of non-cardinal axes was enhanced when these axes were aligned with the extrinsic vertical. Results are inconsistent with most current models of spatial memory distortions but in line with models explaining distortions in terms of attentional processes in topographical neuronal networks. Keywords: Spatial memory distortions, visual short-term memory, spatial reference systems When we are looking at a visual scene, we seem to effortlessly extract most aspects of spatial layout. But although perception of some spatial primitives, such as the distance between two single dots or the length of a single line, is essentially unbiased (Baird, 1970; Gescheider, 1997), veridicality often breaks down when stimuli are presented within a visual context. For example, many visual illusions involve geometric distortions induced by the presence of suitable flanking stimuli (e.g., Crawford, Huttenlocher, & Engebretson, 2000), giving rise to biased perception of line length, distance, angles, size, parallelism, and other fundamental aspects of spatial layout. Spatial distortions also occur in a Correspondence concerning this article should be addressed to Thomas Schmidt, University of Göttingen, Institute of Psychology, Gosslerstr. 14, D Goettingen, Germany; thomas.schmidt@psych.uni-goettingen.de.

2 314 SCHMIDT large number of paradigms where moving stimuli are systematically mislocated (Bridgeman, Peery, & Anand, 1997; Hubbard, 1995; Müsseler, Stork, & Kerzel, 2002; Nijhawan, 1994). Such findings suggest that the positions of spatial objects are not encoded independently; rather, it seems that objects encoded in the same spatial map mutually influence their positions, e.g., by lateral connections of cells in topographic neuronal networks (Suzuki & Cavanagh, 1997). Viewed in this way, some amount of distortion in perceived space may actually be the rule rather than the exception. For example, Watson (1977) has suggested that each visual stimulus induces some curvature of visual space in the same fashion that mass points induce curvature of physical space in Einsteinian physics. According to this view, every line or dot that enters the visual field is a source of distortion; it might cause nearby lines to bend away from it, or shift the positions of line terminations. If multiple objects in spatial representations mutually influence their encoded positions, systematic distortions should develop over time if the stimulus input is turned off and the representation is left to itself. Spatial short-term memory distortions of this sort could provide interesting insights into mechanisms of spatial representation and organization. As argued below, they might reveal regularities of spatial interactions between visual objects in topographic networks, the temporal dynamics of spatial memory storage, and the overall organization of visual memory representations in different spatial reference systems. Distortions in Visual Short-term Memory. Indeed, rich patterns of spatial memory distortions can be demonstrated in simple tasks of reproducing the position of a briefly presented target stimulus after a retention interval. Huttenlocher, Hedges, and Duncan (1991; see also Laeng, Peters, & McCabe, 1998; Nelson & Chaiklin, 1980) presented a small target dot located somewhere within a large circle. After a short retention interval, participants reproduced the location by placing a dot into a new, empty circle identical to the first one. It was found that reproductions were not scattered uniformly but tended to be bean-shaped and to slightly bend around the circle center. In addition, the horizontal and vertical meridians were systematically avoided. It was concluded that participants used some type of polar coordinate frame centered on the circle such that radial and angular coordinates were distorted independently, and that they spontaneously segmented the circle into four quadrants. Werner and Diedrichsen (2002) showed observers a small target dot in the presence of two horizontally aligned circles (visual landmarks), one to the left and one to the right of the display center, in an otherwise empty display. After a variable retention interval where the display was visually masked, the landmarks reappeared, and the observers had to use a mouse cursor to reproduce the exact location of the target dot relative to the landmarks. It was found that the landmarks induced a systematic pattern of spatial biases or distortional field: Targets near the landmarks were reproduced too far away from them as if repulsed in a radial fashion, and there was an additional bias away from the mid-

3 REFERENCE SYSTEMS OF SPATIAL MEMORY DISTORTIONS 315 point between the landmarks. These effects built up gradually during the retention interval but were already discernible at retention intervals as short as 50 ms (measured in a position change detection task that allowed for such short intervals; see also Werner & Schmidt, 2000). Subsequent work showed that distortional fields critically depend on encoding the landmarks together with the target before the retention interval, whereas landmarks presented only after the retention interval had little effect (Diedrichsen, Werner, Schmidt, & Trommershäuser, 2004), suggesting that the landmarks and target have to be included in the same memory representation for spatial distortions to develop. Reference Systems of Spatial Memory Distortions. Carlson-Radvansky and Irwin (1993) distinguish three types of reference systems in spatial cognition. Object-centered reference systems are locked to intrinsic axes of an object, e.g., its axes of elongation, symmetry, or movement direction. Environment-centered reference systems are anchored to features of the object s surroundings (like salient landmarks in an open landscape, or room geometry) independent of the object s orientation. Finally, viewer-centered reference systems code spatial relations relative to parts of the observer s body, e.g., the eyes, head, trunk, hand, or shoulder. In this paper, object-centered reference systems will be denoted as intrinsic (Levinson, 1996). Because viewer-centered and environment-centered reference systems usually coincide for stimuli in the fronto-parallel plane, and because they cannot be disentangled by the methods reported in this paper, they will be collectively referred to as extrinsic reference systems. In a number of experiments, it has been shown that distortional fields closely follow geometrical transformations of the landmark layout, including translation, rotation, dilation, and compression of the landmark configuration (Diedrichsen, 1998; Schmidt, Werner, & Diedrichsen, 2003; Werner & Schmidt, 2000). Therefore, distortions must obey an intrinsic reference system centered on the landmarks and oriented so as to respect major properties of symmetry or elongation in the landmark configuration. Intrinsic spatial references are not only provided by the landmarks themselves but also consist of the implicit connection lines between landmarks, implicit midpoints between landmarks, the geometrical center of gravity, or further auxiliary points and lines that can be constructed from these primitives, all of which tend to have a repulsory effect on remembered target position (virtual landmarks; Schmidt et al., 2003; see also Psotka, 1978). We have argued that virtual landmarks serve to stabilize spatial representations by providing regions of low positional uncertainty. Although increased certainty of stimulus position comes at the expense of small spatial biases, these biases will tend to check each other in multiple-landmark systems (Schmidt et al., 2003). In addition to the intrinsic reference system provided by physical and virtual landmarks, there are indications that these intrinsic reference systems are modulated by extrinsic ones. For example, the lower half of the distortional field of two landmarks (in monitor coordinates) is often subject to systematic downward distortions independent of whether the landmarks are oriented

4 316 SCHMIDT horizontally or vertically (Schmidt et al., 2003), suggesting an influence of environmental or viewer-centered reference systems, e.g., the monitor edges, the laboratory layout, or the observer s body midline. Existing models of spatial memory distortions differ in their allowance for extrinsic reference systems to affect spatial memory distortions. Nelson and Chaiklin (1980) have proposed a model similar to that of Watson (1977) where the remembered target position is distorted with respect to the position of the landmarks. Remembered distance from the landmarks is underestimated (Gescheider, 1997), resulting in a distortion towards the nearest landmark that increases with increasing distance from the landmark. Multiple landmarks contribute to the overall pattern as a weighted sum, with weights depending on their distance from the target. In this model, distortions are exclusively depending on the layout of the stimuli, i.e., an intrinsic reference system. Huttenlocher et al. (1991) theorize that participants responses are not only influenced by their recollection of the target location but also by the spatial category it is assigned to. They propose two levels of representation in spatial memory, both noisy but spatially unbiased: a fine-grained map of spatial locations, and a coarse scheme of spatial regions where only topological relations are encoded. Distortions arise when knowledge of category membership is used to assist in the reconstruction of an imprecisely remembered target location. What forms a spatial category does not strictly follow from stimulus geometry in this model; it depends on spatial coding strategies of the observers as well, and can only be determined ex post from the scatter of responses. The model expressly allows for individual differences as well as attentional strategies to segment the landmark configuration (Bryant & Subbiah, 1993, 1994; Hollands & Dyre, 2000), which may include the use of intrinsic as well as extrinsic reference information. Separating Intrinsic and Extrinsic Reference Systems in Spatial Memory Distortions. The goal of this paper is to use symmetry properties of distortional fields to study the interplay of intrinsic and extrinsic reference systems in spatial short-term memory. Symmetry should be influenced by spatial reference systems because these systems place constraints on the direction and magnitude of distortions: If, say, reproduced target locations near the cardinal axis of a landmark configuration are biased away from the axis, the resulting distortional field should display some symmetry about that axis, at least for the vector component perpendicular to the axis 1. Symmetry about an axis should be positively related to its saliency, whereas the complete absence of any spatial reference system should yield distortional fields with randomly oriented vectors only. Most experiments on spatial memory distortions have worked with landmark configurations possessing quite narrow symmetry properties. For example, a space containing a single landmark or a large circle (as in Huttenlocher et al., 1 Throughout this paper, the term cardinal axis is used for the single symmetry axis of a landmark configuration possessing only one-fold symmetry.

5 REFERENCE SYSTEMS OF SPATIAL MEMORY DISTORTIONS ; Nelson & Chaiklin, 1980) is radially symmetric as long as extrinsic spatial references like the monitor edges do not come into play. Therefore, the number and direction of possible symmetry axes is indeterminate. In contrast, a space containing two landmarks (as in Diedrichsen et al., 2004; Schmidt et al., 2003; Werner & Diedrichsen, 2002) cannot have more than two symmetry axes, the one connecting the landmarks and the perpendicular one bisecting it. Because most extrinsic reference systems (like monitor frames, room edges, gravity) also consist of perpendicular axes, it is usually not possible to misalign one symmetry axis of the landmark configuration with the extrinsic system without misaligning the other axis too. In order to compare aligned and misaligned intrinsic axes in the same distortional field, one needs a landmark system with nonorthogonal symmetry axes. For example, in an equilateral configuration of three landmarks, there are three axes of symmetry forming angles of 120 with each other. However, only one of these axes can be aligned with any system of perpendicular extrinsic axes; the other two are necessarily misaligned. In the following experiments, triangular configurations of landmarks are used to study spatial interactions of intrinsic and extrinsic reference systems in visual short-term memory. The first two experiments investigate equilateral configurations where only one axis is aligned with the extrinsic vertical (Experiment 1) or where all three axes are misaligned to different degrees (Experiment 2). The third experiment investigates possible interactions of extrinsic and intrinsic reference systems by using elongated, equiscleral triangles where the axis of elongation does or does not coincide with the extrinsic vertical (Experiment 3). Experiment 1 In this experiment, two equilateral configurations of three landmarks were used. Because equilateral triangles have three axes of symmetry, distortions in visual memory might develop with respect to any of these axes. In the present experiment, triangular configurations were displayed in either upright (0 ) or top-down (180 ) orientation so that one of their symmetry axes was always aligned with the extrinsic vertical, as defined by the monitor frame, gravity, or the observer s body midline. If distortional fields were exclusively determined by intrinsic properties of the stimulus, all symmetry axes should be equally strong. In contrast, if extrinsic reference systems also influenced the distortional pattern, an intrinsic axis aligned with an extrinsic axis should display enhanced symmetry. Method Participants. Eight students (age 15 to 40, one male, all right-handed) of the Institute of Psychology at the University of Göttingen participated for course credits or for a payment of 15 DM (Deutsche Mark) per hour. Their vision was normal or corrected-to-normal. Informed consent was obtained according to institutional guidelines.

6 318 SCHMIDT Apparatus and Stimuli. Participants were seated at the wider end of a large funnel, with the monitor situated at the narrow end such that the laboratory environment was blocked from view and the monitor edge was circular rather than rectangular. The experiment was performed in darkness so that the monitor edge was clearly visible only during the masking interval and almost invisible during the presentation of landmarks and target. The funnel was made of a homogenously textured fabric and left only a circular central portion (17 cm, 8.8 ) of the screen where stimuli were presented at a viewing distance of 110 cm. Head position was not fixed. The experiment was controlled by a Personal Computer with an AMD K-2 processor (300 MHz). Stimuli were presented on a 14" VGA color monitor (640 by 480 pixel (px)). Stimulus presentation was synchronized with the monitor retrace rate of 60 Hz. All stimuli were presented against a black background (0.01 cd/m²). Landmarks were three green unfilled circles (28.70 cd/m²), 9 px (0.26 ) in diameter, that were arranged in an equilateral triangle with a side length of 180 px (5.20 ), with its center of gravity located at the screen center. This configuration could appear either upright (0 orientation) or top-down (180 orientation). The target was a small white dot (43.0 cd/m²) with a diameter of 3 px (0.09 ), presented on a regular grid of 40 locations shown in Figure 1. A dynamic pattern similar to static interference on a television screen was used as a visual mask. It filled the entire visual portion of the screen and consisted of randomly chosen black and white elements (2 2 px), with one quarter of the elements white at any given time. Four different random patterns were presented in succession for 33 ms each, after which the sequence repeated. Procedure. A trial started with the presentation of the target together with the three landmarks for 500 ms. The target was then replaced by the dynamic mask for 500 ms while the landmarks remained visible. Immediately after the masking interval, a mouse cursor looking exactly like the target appeared randomly in one of the landmark locations. The participant s task was to reproduce the exact target position with the mouse cursor, pressing the left mouse button when she thought she had reached the correct position. A 1000 Hz, 100 ms tone was sounded for feedback that the response had been registered. After an intertrial interval of 500 ms, a new trial began. Participants were instructed to work quickly, but accuracy was emphasized over speed. They were allowed to move their eyes freely. Stimulus conditions were counterbalanced such that each combination of landmark orientation (0, 180 ), starting position of the mouse cursor, and target position occurred quasi-randomly and equiprobably, with each combination appearing once in 10 consecutive blocks. After each block, participants received summary feedback about their average Euclidean deviation from the true target position, rounded to the nearest pixel. Participants performed one session of 20 blocks with 24 trials each. Each session started with an additional practice block of 24 trials with stimuli drawn randomly from the experimental blocks. Practice trials were not analyzed. After the final session, participants were debriefed and received an explanation of the purpose of the experiment.

7 REFERENCE SYSTEMS OF SPATIAL MEMORY DISTORTIONS 319 Figure 3. Symmetry calculations in Experiment 1. a) Normalization procedure. Distortional fields from the top-down condition were first rotated by 180. Then, distortional fields were rotated so that the symmetry axis of current interest (1) was upright (2). Distortional vectors on the left side of the axis were then mirror-imaged and subtracted from those on the right side (3). Large black arrows show the original vectors of distortion; large gray arrows the mirror images from the left side; small gray arrows the vector difference of left side minus right side (symmetry residuals). b) Correlations between vectors of distortion on both sides of the symmetry axis of interest, separately for the error components perpendicular ( ) and parallel ( ) to that axis. Insets show the symmetry axis of interest prior to the normalization procedure described in a). c) Average length of symmetry residuals after normalization. Filled symbols are for the upright condition; open symbols for the top-down condition. Insets on the abscissa show the symmetry axis of interest in the respective condition prior to normalization.

8 320 SCHMIDT Data treatment. Trials with response times faster than 100 ms or slower than 10,000 ms were excluded because these trials likely reflected anticipatory or guessing behavior. Trials with a Euclidean target deviation larger than 30 px were also excluded. The entire procedure eliminated 1.35% of the raw data. Results Overall Effects. Vector fields of memory distortions were rotated into an upright orientation. Trimmed raw data were analyzed in a MANOVA design with target position and triangle orientation as fixed factors, using the x (horizontal) and y (vertical) components of distortion (in monitor coordinates) as dependent variables. To take repeated measures into account, a random participant factor was included whose effects are not reported here. Landmarks induced a complex but regular pattern of distortions, which was similar for both display orientations (Figure 2). Reproduced target positions near the landmarks were distorted away from the landmarks in a radial fashion; this repulsion effect tended to turn into attraction at larger distances from the landmarks. There was some distortion away from the geometrical center of gravity of the entire figure, which was more pronounced in the upright than in the top-down configuration. Distortional effects along the virtual sides of the triangles were similar to those observed in two-landmark configurations (Werner & Diedrichsen, 2002; Schmidt et al., 2003), with distortion away from the landmarks and from the midpoints of the triangles sides. Overall, the pattern of distortions seems to follow major lines and axes in the three-landmark figure: There is little departure from the virtual sides of the triangle or from the vertical mid-axis. Figure 2. Spatial distortions in Experiment 1. Vectors of distortion point from the true to the reproduced target location. Throughout this paper, arrow length is augmented by a factor of 2.5 for better readability.

9 REFERENCE SYSTEMS OF SPATIAL MEMORY DISTORTIONS 321 MANOVA confirmed that the vectors of distortion depended on target position, Wilk s Λ(78, 544) = 0.307, p <.001. There was no main effect of landmark orientation, Λ(2, 6) = 0.723, p =.379, indicating that the net vectors of distortion (averaged over all target positions) were comparable in the 0 and 180 conditions. Importantly, there was no interaction of both factors, Λ(78, 544) = 0.753, p =.343, showing that both vector fields did not differ systematically after rotation into an upright position. Two Measures of Symmetry. Symmetry in vector patterns was evaluated by examining the correlations between opposite halves of the vector fields. The field was rotated so that the symmetry axis of current interest was in an upright position (Figure 3a). First, targets directly on the symmetry axis were excluded. Then, coordinates of corresponding targets from the left and right halves of the vector field were correlated separately for horizontal coordinates (perpendicular to the symmetry axis of interest, denoted by symbol " ") and vertical coordinates (parallel to it, " "), yielding one correlation coefficient for each coordinate. Symmetry about other axes was evaluated in the same way after appropriate rotation of the pattern. With this measure, perfect symmetry would result in correlations of r( ) = 1 (for error components perpendicular to the symmetry axis) and r( ) = 1 (for components parallel to the symmetry axis). For statistical tests, correlations were compared using Fisher s Z transformation at a Bonferroni-corrected significance level of α/3 =.017. Note that this correlational measure is invariant with respect to changes in scale between the two halves: If all vectors, say, in the right half were twice as long as those in the left half but had the same direction, correlations would not be affected. To aid in the interpretation of the correlational measure, a Euclidean measure of the degree of asymmetry was computed from the pooled vector fields (i.e., vector fields averaged across participants). Again, vectors lying on the symmetry axis were discarded. Next, vectors on one side of the symmetry axis were reflected upon corresponding vectors on the other side of the axis, so that the differences between corresponding vectors yielded symmetry residuals (Figure 3a, rightmost column). The length of these residuals, averaged across all vector pairs, is a Euclidean measure of symmetry that equals 0 if symmetry is perfect and has some finite value if there is any asymmetry 2. 2 Like all Euclidean error measures, its value depends on random noise as well as on more systematic sources of asymmetry, and can reach zero only at zero variance. Since random noise is stronger in individual participants than in the pooled data, the individual values would generally be larger and less discriminative than those from the pooled data. Because data from individual participants were too noisy to yield useful values on their own, the Euclidean symmetry measure was computed for pooled vector fields as a purely descriptive measure.

10 322 SCHMIDT Results from the correlational approach are shown in Figure 3b. For both orientations, there was symmetry about all possible axes 3, all p.021. In the upright condition, symmetry was significantly better for the vertical symmetry axis than about the 120 axis, z( ) = 2.239, p =.013, z( ) = 1.019, p =.154, and also significantly better than about the 120 axis z( ) = 2.295, p =.011, z( ) =.931, p =.176. The two latter axes did not differ significantly. In the top-down condition, although symmetry about the vertical axis seems slightly better than along the oblique axes, there were no significant differences between axes, z 0.787, all p.216. The Euclidean index gave a similar picture: In both landmark configurations, the vertical axis displayed smaller symmetry residuals than the remaining axes, indicating less asymmetry (Figure 3c). Again, this difference was more pronounced for the upright configuration. Discussion Three landmarks arranged in an equilateral triangle induce a complex but systematic pattern of distortions in visual short-term memory that is consistent with the distortional effects observed in the more simple two-landmark situation. There is repulsion away from the landmarks, from the midpoints of the virtual sides, and from the center of gravity of the landmark configuration. All this suggests a straightforward generalization of the patterns reported before (Schmidt et al., 2003; Werner & Diedrichsen, 2002). Symmetries of distortions about all possible axes of the landmark configuration suggest that an intrinsic, stimulus-based reference system is the main determinant of the distortional fields here. However, symmetry residuals are about four times smaller for the vertical than for the oblique axes if the triangle is oriented upright; they are still two times smaller if it is oriented top-down. These findings suggest that extrinsic reference systems also contribute to spatial memory distortions: Based on intrinsic stimulus geometry alone, there would be no reason to expect the vertical axis to display stronger symmetry than others. Does the vertical axis induce stronger symmetry only because it coincides with a major axis of an observer- or environment-based extrinsic reference system, like the body axis or gravity? The fact that the alignment effect is different for upright and top-down triangles shows that this cannot be the whole story. It is possible that this axis becomes special only by virtue of the attentional state of the observer cognitively coding it as especially salient. In this view, alignment would not enhance symmetry if observers were not explicitly noticing or strategically using this relationship. Support for the attentional view comes from experiments by Palmer and Bucher (1981, 1982; Sekuler, 1996) who asked participants to indicate the perceived pointing direction of equilateral 3 Note that orientations of axes as reported here (as well as in the corresponding sections of Experiments 2 and 3) refer to the original, unnormalized, and untransformed landmark configurations in monitor coordinates. Angles are measured counterclockwise; 0 marks the upward direction.

11 REFERENCE SYSTEMS OF SPATIAL MEMORY DISTORTIONS 323 Figure 3. Symmetry calculations in Experiment 1. a) Normalization procedure. Distortional fields from the top-down condition were first rotated by 180. Then, distortional fields were rotated so that the symmetry axis of current interest (1) was upright (2). Distortional vectors on the left side of the axis were then mirror-imaged and subtracted from those on the right side (3). Large black arrows show the original vectors of distortion; large gray arrows the mirror images from the left side; small gray arrows the vector difference of left side minus right side (symmetry residuals). b) Correlations between vectors of distortion on both sides of the symmetry axis of interest, separately for the error components perpendicular ( ) and parallel ( ) to that axis. Insets show the symmetry axis of interest prior to the normalization procedure described in a). c) Average length of symmetry residuals after normalization. Filled symbols are for the upright condition; open symbols for the top-down condition. Insets on the abscissa show the symmetry axis of interest in the respective condition prior to normalization.

12 324 SCHMIDT triangles. Because these stimuli are inherently tri-stable, participants reported attentional switching between percepts. However, aligning one of the triangle s axes with a rectangular frame induced a strong bias to perceive the triangle pointing into the direction induced by that frame. Experiment 2 If the alignment effect in the previous experiment was due to attentional mechanisms rather than mere physical alignment of intrinsic and extrinsic reference systems, the effect might survive small rotations of the landmark configuration out of alignment with the extrinsic vertical. In contrast, if strict physical alignment was required, such rotations should be sufficient to abolish or at least alleviate the effect. In Experiment 2, stimulus configurations were used where all intrinsic axes were clearly misaligned with any extrinsic axis. This was accomplished by rotating the stimulus configuration from Experiment 1 by plus or minus 15. Note that if one symmetry axis of the triangle is tilted by plus or minus 15 from the extrinsic vertical, this means that one of the other symmetry axes must be tilted by minus or plus 15 from the extrinsic horizontal. Therefore, nearly-horizontal and nearly-vertical axes can be compared. Method Participants. Eight students (age 18 to 31, three of them male, all righthanded or ambidextrous) of the Institute of Psychology at the University of Göttingen participated for course credit or for a payment of 15 DM per hour. Their vision was normal or corrected-to-normal. Informed consent was obtained according to institutional guidelines. Stimuli and Procedure. Methods were identical to those used in Experiment 1, with the one exception that the triangular configuration of landmarks was presented in orientations of +15 (counterclockwise) or 15 instead of 0 and 180. As before, the center of gravity of the triangle was centered on the screen. Data Treatment and Statistical Methods. Criteria for outlier elimination were as described in Experiment 1. The procedure eliminated 0.68% of the raw data. Statistical analysis proceeded as described in Experiment 1, after rotating the vector fields into an upright (0 ) position. Results Overall Effects. Vector fields of memory distortions were similar to those observed in Experiment 1 (Figure 4). In both the +15 and 15 orientation, there was distortion away from the landmarks, from the midpoints of the virtual sides of the triangles, and from the center of gravity. MANOVA confirmed that the vectors of distortion depended on target position, Λ(78, 544) = 0.388, p <.001. There was no main effect of landmark orientation, Λ(2, 6) = 0.852, p =.619, indicating that the net vectors of distortion (averaged over all target positions) were comparable across conditions. However, there was a significant interaction of target position and landmark

13 REFERENCE SYSTEMS OF SPATIAL MEMORY DISTORTIONS 325 Figure 4. Spatial distortions in Experiment 2. orientation, Λ(78, 544) = 0.669, p =.003, suggesting some difference in vector fields after normalization, although it is hard to tell the differences by inspecting the figures. Symmetry Properties. Symmetry in vector fields was evaluated using the methods described in Experiment 1. The results from the correlational approach are shown in Figure 5a. Symmetry was somewhat weaker than in Experiment 1. Although there were significant correlations about all axes of symmetry for the vertical ( ) coordinates, all r.630, all p.003, symmetry for the horizontal ( ) coordinates was significant only about the nearly-vertical axes (both p <.001), but not reliable about the other axes,.032 p.119. In the 15 condition, symmetry was significantly better for the nearly-vertical 15 symmetry axis than for the nearly-horizontal 105 axis, z( ) = 2.801, p =.003, z( ) = 0.865, p =.194, and also significantly better than for the misaligned 135 axis, z( ) = 2.554, p =.005, z( ) = 0.771, p =.220. The latter two axes did not differ significantly. In the 15 condition, there was a similar pattern, with better symmetry about the nearly-vertical 15 axis than about the nearly-horizontal 105 axis, z( ) = 2.594, p =.005, z( ) = 0.546, p =.293, and also better symmetry than about the misaligned 135 axis, z( ) = 2.273, p =.012, z( ) = 1.976, p =.024. Again, the two latter axes did not differ significantly. Euclidean symmetry measures agreed with the correlational approach: Symmetry residuals were about three times smaller for the nearly-vertical axes than for the other two axes irrespective of triangle orientation (Figure 5b).

14 326 SCHMIDT Figure 5. Symmetry in vector fields in Experiment 2. a) Correlations between vectors of distortion on both sides of the symmetry axis of interest as described in Figure 3a. b) Average length of residual vectors; conventions as in Figure 3. Discussion The pattern of spatial memory distortions was similar to the one reported in Experiment 1. As before, distortional fields followed major lines in the figure, with little deviations from the virtual sides of the triangles. There was a clear effect of distortion away from the landmarks, the midpoints of the virtual sides, and the center of gravity. If mere physical alignment with an extrinsic system of reference was required for enhancing the symmetry properties of an intrinsic stimulus axis, symmetry should be similar about all possible axes when they are all clearly misaligned with the extrinsic reference. However, symmetry about the nearlyvertical axis was clearly stronger than about the remaining two axes, suggesting that this line was preferred as a spatial reference even though it was misaligned with the extrinsic vertical. In contrast, the nearly-horizontal axis did not display any enhanced symmetry despite its being misaligned from a major extrinsic axis by the same angle. Thus, there is a discrepancy between nearly-horizontal and nearly-vertical symmetry axes similar to the one between vertical axes in upright vs. bottom-up configurations. These findings show that exact physical alignment of intrinsic and extrinsic axes is not required for one axis to receive special prominence as a spatial reference, suggesting the influence of attentional coding strategies. The results also echo earlier findings suggesting an anisotropy in early visual segmentation processes: For example, symmetry in natural images or dot patterns is detected easier about the vertical than about the horizontal axis (Evans, Wenderoth, & Cheng, 2000).

15 REFERENCE SYSTEMS OF SPATIAL MEMORY DISTORTIONS 327 Experiment 3 In Experiments 1 and 2, only equilateral triangular configurations of landmarks were used, and the effect of alignment or misalignment with extrinsic reference systems was studied. Such stimuli have three possible axes of symmetry, and are therefore ambiguous with regard to their apparent orientation. In Experiment 3, elongated equiscleral triangles were used where the axis of elongation was also the only axis of symmetry, thereby defining a cardinal axis in an unambiguous object-centered, intrinsic frame of reference which may become either aligned or misaligned with an extrinsic reference axis. An equilateral configuration of landmarks oriented upright on a computer display served as a control condition. Elongated equiscleral triangles were created by shifting the location of only one of the landmarks along a line perpendicular to the line connecting the remaining two landmarks (see Figure 6). This construction led to four stimulus configurations whose symmetry properties could be compared. Method Participants. Nine students (age 19 to 37, four of them male, one lefthanded) of the Institute of Psychology at the University of Göttingen participated for course credits or for a payment of 15 DM per hour. Their vision was normal or corrected-to-normal. Informed consent was obtained according to institutional guidelines. Stimuli. The apparatus was as described in Experiment 1. The stimuli were identical to those used in the previous two experiments but appeared in new arrangements. All displays consisted of configurations of three landmarks. In one quarter of the trials, landmarks were arranged in an equilateral triangle with a side length of 2.89, always in upright (0 ) orientation, with 18 possible target locations. In the remaining trials, equiscleral configurations were generated by stretching the triangles along one of the three lines of symmetry while leaving the two remaining landmarks in place (Figure 6). Resulting configurations had a narrow angle of 40 ; their axis of elongation had an orientation of either 0, 120, or 120. The grid of possible target locations was stretched accordingly. A dynamical mask was used as described in Experiment 1. Procedure. A trial started with the appearance of the target together with the three landmarks for 750 ms. The target was then replaced by the dynamic mask for 500 ms while the landmarks remained visible. Immediately after the masking interval, a mouse cursor looking exactly like the target appeared randomly in one of eight possible locations beyond the visible edges of the monitor. After the participant had reproduced the target position, a 1000 Hz, 100 ms tone was sounded for feedback that the response had been registered. After an intertrial interval of 500 ms, a new trial began. As before, accuracy was emphasized over speed, and free eye movements were allowed. Data Treatment. Criteria for outlier elimination were as described in Experiment 1. The procedure eliminated 1.93% of the raw data.

16 328 SCHMIDT Figure 6. Spatial distortions in Experiment 3. Stimulus conditions were counterbalanced such that each combination of landmark configuration (equiscleral 0, 120, 120 configurations, and equilateral configuration), and target position occurred quasi-randomly and equiprobably, with each combination appearing once in three consecutive blocks. The starting position of the mouse cursor was randomly determined in each trial. After each block, participants received summary feedback about their average Euclidean deviation from the true target position, rounded to the nearest pixel. Participants performed one session of 30 blocks with 24 trials each. Each session started with an additional practice block of 24 trials with stimuli drawn randomly from the experimental blocks. Practice trials were not analyzed. After the final session, participants were debriefed and received an explanation of the purpose of the experiment.

17 REFERENCE SYSTEMS OF SPATIAL MEMORY DISTORTIONS 329 Results Equilateral Condition. In the equilateral condition, there was striking symmetry of spatial distortions about all three possible axes (Figure 6). In line with the results from Experiments 1 and 2, reproductions were biased away from the landmarks and also slightly away from the midpoints of the triangles virtual sides. MANOVA confirmed that vectors of distortion depended on the location of the target, Λ(34, 304) = 0.357, p <.001. Correlational analyses of symmetry (Figure 7b, leftmost column) showed that symmetry was significant about all possible axes, all p.005. Symmetry about the vertical axis was better than symmetry about the 120 axis, z( ) = 3.235, p <.001, z( ) = 0.024, p =.509. It was also slightly better (not significantly so after Bonferroni correction) than symmetry about the +120 axis, z( ) = 1.554, p =.060, z( ) = 1.640, p = The latter two axes differed only slightly, z( ) = 1.681, p =.046, z( ) = 1.663, p =.952. Euclidean symmetry measures revealed that symmetry residuals were Figure 7. Symmetry in vector fields in Experiment 3. a) The congruence transformation proceeded by first rotating each vector field so that the axis of elongation was upright, then compressing the vector field until the landmark configuration was equilateral, and then proceeding with the normalization routine as described in Figure 3a. b) Correlations between vectors of distortion on both sides of the symmetry axis of interest after the congruence transformation. c) Average length of residual vectors; conventions as in Figure 3.

18 330 SCHMIDT about five times larger for the oblique than for the vertical axes (Figure 7c, filled symbols). Equiscleral Conditions. The three equiscleral conditions were subjected to MANOVA after rotating the axis of elongation into a vertical position. MANOVA confirmed that the vectors of distortion depended on target position, Λ(34, 304) = 0.342, p <.001. There was no main effect of orientation, Λ(4, 34) = 0.811, p =.454, but a significant interaction of both factors indicating that vector fields differed across orientations, Λ(68, 610) = 0.683, p <.001. Inspection of Figure 6 suggests that the equiscleral vector fields are just a stretched version of the equilateral one. Compressing the elongated triangles along their cardinal axes should then recover implicit symmetry about the noncardinal axes. After normalising the vector fields so that the axis of elongation was vertical, they were compressed along the vertical axes until they were equilateral again (from here on, this transformation will be referred to as the congruence transformation, Figure 7a). Analysis then proceeded as described in Experiment 1. For the vector field in upright orientation with its axis of elongation coinciding with the extrinsic vertical, correlational analyses shown in Figure 7b (second column) revealed strong symmetry about the 0 axis, r( ) =.956, r( ) =.972, both p <.001, but weaker symmetry about the 120 axis, r( ) =.479, p =.096, r( ) =.816, p =.004, as well as the 120 axis, r( ) =.585, r( ) =.616, both p.049. Symmetry was better about the axis of elongation than about the 120 axis, z( ) = 3.639, p <.001, z( ) = 2.600, p =.005. It was also better than about the 120 axis, z( ) = 3.247, p <.001, z( ) = 3.727, p <.001. The noncardinal axes did not differ in symmetry. For those vector fields with their cardinal axes departing from the extrinsic vertical, symmetry differences across the three axes were less pronounced. In both orientations, there was discernible symmetry about all possible axes, all r( ).665, all r( ).688, all p.025. For the triangle in 120 orientation, symmetry was better about the cardinal axis than about the non-cardinal axis running through the upper landmark, z( ) = 2.500, p =.006, z( ) = 0.319, p =.375, but not different from the axis running through the lower right landmark, both p.166. Also, these two axes did not differ significantly. For the triangle in 120 orientation, symmetry was somewhat better (not significantly so after Bonferroni correction) about the cardinal axis than about the axis running through the upper landmark, z( ) = 1.730, p =.042, z( ) = 0.443, p =.329, but not different from the axis running through the lower left landmark, both p.108. Again, the two non-cardinal axes did not differ. Euclidean symmetry measures agreed with the correlational approach but revealed interesting quantitative relationships among the conditions (Figure 7c, unfilled symbols). In the equilateral configuration, the vertical axis of symmetry had symmetry residuals that were about five times smaller than for the two oblique axes, as had been observed in Experiment 1. Symmetry was comparable in size to that observed for the cardinal axes in the equiscleral conditions. Thus,

19 REFERENCE SYSTEMS OF SPATIAL MEMORY DISTORTIONS 331 after the congruence transformation, symmetry about the vertical axis was retained when the triangle was oblonged along that axis, and even when this oblonged figure s orientation was changed. Second, in oblonged triangles, the cardinal axis displayed stronger symmetry than the other axes after a congruence transformation, regardless of stimulus orientation. Third and most surprising, symmetry about the non-cardinal axes in elongated triangles was not worse (indeed, even slightly better) than symmetry about the corresponding axes in the equilateral condition. This is striking because symmetry is only implicit here and must be reconstructed via the congruence transformation. Discussion Using elongated equiscleral triangles does not qualitatively change the pattern of distortions observed in visual short-term memory. Distortions mainly take place along the virtual sides of the triangles, with distortion away from the landmarks and away from the midpoints of the triangles sides. Distortional fields are strikingly symmetric and largely invariant with respect to the orientation of the landmark configuration, obviously taking place in an intrinsic, stimulus-centered system of reference. Even for elongated configurations with only a single cardinal axis, implicit symmetry properties of non-cardinal axes can be reconstructed if a congruence transformation is performed such that symmetry about all three axes is assessible. If this is done, symmetry about these axes is at least as strong as that observed in a configuration of triangles that was equilateral from the beginning. The data suggest a limit to interactive effects of intrinsic and extrinsic reference systems. Although the intrinsic vertical is still the best symmetry axis in an equilateral triangle, in an equiscleral triangle the amount of symmetry about the cardinal axis does not depend at all on its alignment with the extrinsic vertical. It seems that the cardinal axis is able to impose such a strong intrinsic reference system that all additional effects of alignment are overruled. There might also be a floor effect at work here; because the symmetry index is a Euclidean measure, it is sensitive to both the mean and variance of vector differences, reaching zero only if the variance is zero. Because some variability in reproductions is unavoidable, weak effects of alignment might be too small to reduce the index even further. In any event, the correlational measure leads to the same conclusion as the Euclidean. General Discussion Together, Experiments 1 to 3 show that both intrinsic and extrinsic reference systems play a role in determining the geometric properties of spatial short-term memory distortions, most notably their symmetry properties. Distortional fields from three-landmark configurations are in line with Schmidt et al. s (2003) notion of virtual landmarks: Virtual as well as physical landmarks provide an intrinsic reference system for spatial memory distortion that encompasses virtual

20 332 SCHMIDT connection lines and midpoints between landmarks as well as the center of gravity. Although the overall shapes of the distortional fields are strongly determined by these intrinsic references, additional alignment with the extrinsic vertical may enhance the symmetry properties of intrinsic axes. The experiments reported here do not allow to determine whether such extrinsic effects stem from viewer-centered reference systems, such as the body, head, or eye midlines, or from environment-centered reference systems, such as the monitor frame, room edges, or gravity. To dissociate such influences, it would be necessary to tilt the observer s body axis relative to the environment (Lackner, 1992; Lipshits, McIntyre, Zaoui, Gurfinkel, and Berthoz et al., 2001). Experiment 3 shows that if one landmark in a configuration of three is moved to a new position, the distortional field stretches accordingly as if the spatial representation was elastic and bendable. This is consistent with earlier findings by Schmidt et al. (2003) and Diedrichsen (1998) dealing with affine transformations of landmark configurations, like translation, rotation, dilation, and compression. It suggests that the distortional fields of novel landmark configurations can be predicted by performing an appropriate set of geometrical transformations on a configuration already studied. Elasticity of distortional fields will probably reach its limit where transformations lead to geometrically degenerate landmark patterns, like four landmarks arranged in a triangle or a single line, or a convex shape turning into a concave one. Another caveat is that these considerations only hold as long as intrinsic stimulus properties impose a strong enough system of reference so that extrinsic effects can be neglected. Alignment effects cannot be explained by purely stimulus-driven models of perceptual or memory distortion, like those of Watson (1977) or Nelson and Chaiklin (1980). These models explain spatial distortion strictly in terms of the geometric layout of the stimuli; therefore, a given configuration of landmarks should always lead to the same distortional field, independent of its orientation or alignment with extrinsic reference systems. Alignment effects are more in line with Huttenlocher et al. s (1991) model, where individual coding strategies may influence the exact pattern of distortions. The model allows observers to segment a triangular landmark configuration into two spatial categories based on the axis that seems most salient to them (e.g., because it is the only axis of symmetry, or because it is arbitrarily selected by attentional processes, or because of its alignment with an extrinsic reference system), and would predict enhanced symmetry of distortions about that axis. Note, however, that although the model is able to explain the pattern of spatial biases, it fails to predict the pattern of variances in reproductions. According to the model, the category boundaries should be regions of high spatial uncertainty, because knowledge of category membership cannot be used to aid in the reconstruction of target position located on the boundary. At the same time, the category centers should be regions of low uncertainty (called category prototypes in the model). However, we have shown that the pattern of variances follows the opposite pattern (Diedrichsen et al., 2004; Schmidt et al., 2003). Therefore, the notion of category prototypes should be abandoned in favor of the virtual landmark