MODELS FOR THE ADJUSTMENT OF RATING SCALES 1

Transcription

1 MODELS FOR THE ADJUSTMENT OF RATING SCALES 1 Gert Haubensak and Peter Petzold Justus Liebig University of Giessen, Germany gert.haubensak@psychol.uni-giessen.de Abstract In a category rating experiment using a graded series of squares we changed both stimulus range and frequency during the experimental session. Range-frequency theory (Parducci, 1965; Parducci & Perrett, 1971; Parducci & Wedell, 1986) predicts the same rate of adjustment to the new stimulus range and to the new stimulus frequencies. Its closest competitor, consistency theory (Haubensak, 1992a, 1992b), predicts same or different rates depending on the forgetting parameter, f. The experimental findings support the consistency model, for adjustment to the new stimulus frequencies was complete but adjustment to the new stimulus range was not. Range-frequency theory fails to predict such a dissociation. According to range-frequency theory (Parducci & Perrett, 1971), category ratings such as very large, large, medium, small, and very small, are governed by two principles, the range principle and the frequency principle. The range principle postulates that subjects judge each stimulus relative to its position between the endpoints of the stimulus range. The frequency principle postulates that each stimulus is judged according to its percentile rank. The two principles conflict when stimuli from different parts of the range are presented with unequal frequencies. The actual judgment is assumed to reflect a compromise between the two principles. Although the theory predicts mean judgments accurately, its implicit assumption that frequency effects are produced by the subjects tendency to use the available categories equally often, or to assign an equal number of stimuli to each category is open to question. According to consistency theory (Haubensak, 1992a), frequency effects arise from the subjects tendency to assign the first stimuli to the middle categories rather than to the extreme ones. These stimuli are likely to come from the more frequent ones if drawn at random. Thus, the subjects' scales shift towards the region of the most frequent stimuli. Because subjects, if not instructed otherwise, have a tendency to repeat the same responses to the same stimuli, the effect of the first judgments persists throughout the experiment. (Hence the name of the model.) Both theories provide a satisfactory account of category rating data but they differ in their predictions when both stimulus range and frequency change during an experimental session Proceedings of the 18 th Annual Meeting of the International Society for Psychophysics, Rio de Janeiro,

2 The range-frequency model predicts the same rate of adjustment to the new stimulus range and to the new stimulus frequencies, whichever that rate will be. The consistency model in contrast leaves room for a dissociation between the two rates of adjustment. To test the models' predictions, we performed an experiment with an unexpected change of both stimulus range and frequency. The experiment is described in the following. Method Experiment Subjects. 32 undergraduate students participated in partial fulfillment of course requirements. More than two thirds were females. Stimuli and Apparatus. The stimulus set consisted of 16 empty squares whose width varied from 12 mm to 102 mm, each step by 6 mm, in an arithmetic series. The squares appeared one at a time in black lines against the light gray background at the center of a 19 in. (48 cm) color monitor with 800 x 600 pixels resolution and a refresh rate of 85 khz. Design and Procedure. The subjects were divided into four groups. Group LP first judged the lower half of the total series (Squares 1 to 8) in a positively skewed distribution, Group LN first judged the same squares in a mirrored (negatively skewed) distribution, Groups HP and HN saw the upper half of the series (Squares 9 to 16). After four blocks of trials, all groups were shifted to the total series, U (Squares 1-16). Here, the squares were presented in a uniform distribution, each square just once in a block. The stimulus frequencies are presented in Table 1 below. The presentations proceeded smoothly from the preshift to the postshift stage without any interruption. There were six presentations with the total series (Series U), totaling 96 trials. Order of presentation was random in a block. The participants were seated in front of the computer at a distance of about 60 cm. They were asked to judge every square according to size. The integers 1 to 5 were to be used as categories. The next square appeared three seconds after the response to the preceding one. There was no warning of a possible change of the stimulus context. The room was dimly lit. Table 1. Presentation Frequencies in a Block Stimulus Series LP Series LN Series HP Series HN Series U

3 Results To track the upward and downward movement of the scales, we calculated the neutral point (NP) of the stimulus series by fitting a third-order polynomial to the stimulus values for each individual and each block separately, with the judgments as the independent variable. We then solved the least-square equations for the middle category (Category 3). The resulting stimulus values were taken as best estimates of the neutral points. Figure 1 below shows the movement of the NPs over the experimental session. The preshift scales exhibit the usual frequency effect: The NPs shift toward the region of the more frequent stimuli, downward with the positively distributed stimuli, upward with the negatively distributed ones. What is new is that this frequency, or skewing, effect vanishes after two blocks of postshift trials (32 presentations) whereas the effect of the previous range diminishes up to that point but then persists to the end of the experiment nearly unchanged. In other words, the two effects dissociate after the shift. Consistency model Predictions The consistency predictions are based on the original version of the model (Haubensak, 1992a). According to the model, individuals, if not instructed otherwise, try to retain the presented stimuli in memory along with the response for later use as standards. There may be more than two standards in memory. Each subsequent stimulus is judged relative to the subjective value of the immediately higher and lower standard if there is any (Assumption 1). If, for example, the immediately higher standard is associated with Category 5 and the immediately lower with Category 3, a stimulus lying half-way between the two standards would be called 4. A stimulus closer to the lower standard than to the midpoint would be assigned to Category 3, and a stimulus closer to the higher standard would be called 5. The physical width of squares can be substituted for their subjective size (Haubensak, 1982; Parducci & Wedell, 1986). A standard may be forgotten on the next trial. Forgotten standards are treated as new stimuli. The model makes a few additional assumptions: At the start, subjects select a category close to the center of the scale to leave room on both sides for the judgments to come (A2). Stimuli do not enter STM a second time unless they are forgotten (A3). If a new stimulus matches or even exceeds the highest or the lowest of the current standards, subjects switch to a higher or lower category if there is anyone left (A4). A1 is a variant of the well-known range principle. A2 is in line with the empirical findings (e.g., Poulton, 1968). A3 prevents standards from duplicating, and A4 ensures that the scale rapidly spreads through the range of stimuli as the presentations proceed. The learning and retention assumptions have been adopted from the LS-2 model of Atkinson and Crothers (1963) but other memory models would also do. According to the LS-2 model, a new stimulus is stored in long-term memory (LTM) with probability b on each trial

4 along with the response. If not stored in LTM (1 - b), the stimulus-response pair still enters short-term memory (STM), and is forgotten with probability f or remains there with probability 1 - f (cf. Haubensak, 1992a). Past experience has shown that the learning parameter, b, can be dropped without any loss, leaving only the so-called forgetting parameter, f. Figure 2 shows the simulated NPs for the above experiment, with b = 0 and f =.08. A value of f =.08 limits the number of internal standards to about seven. From the figure one can see that the model predicts the general characteristics of the empirical data fairly well. In particular, it predicts the dissociation between the effect of the previous range and the effect of previous skewing. Figure 1. Upward and downward movement of the neutral points. Empirical data. Vertical dotted line indicates the point of shift. Figure 2. Upward and downward movement of the neutral points as predicted by the consistency model with f =.08. Range-frequency model According to the range-frequency (RF) model (Parducci & Wedell, 1986), the actual mean judgment, J, represents a weighted compromise between the range value, R, and the frequency value, F, of stimulus i, J i = wr i + (1 - w)f i For unpracticed participants, the weighting coefficient, w, is assumed to be about 0.5 (Parducci & Perrett, 1971). The range values are given by R i = (S i - S min )/(S max - S min ) where S max and S min denote the subjective value of the largest and the smallest stimulus. Past experience has shown that the physical width of squares can be substituted for their subjective

5 size (Haubensak, 1982; Parducci & Wedell, 1986). The F-Values are given by F i = (r i - 1)/(N - 1) where r i is the mean rank of stimulus i and 1 and N are the ranks of the smallest and the largest of the stimulus values. Thus, F i is proportional to the conventional percentile rank. Eventually, the predicted mean judgment, J i, is adjusted to the range of the possible ratings (e.g., 4 for a 5- category scale) and the rank of the lowest rating (1 on a scale from 1 to 5) is added. There are two versions of the RF model. In the older version (Parducci & Perrett, 1971), the effective frequency context (the one used for calculating the frequency values) consists of all previous presentations including the preshift series. In the newer version (Parducci & Wedell, 1986), the effective frequency context, not the range context, is limited to the last 12 presentations (Wedell, 1984). Neither the older nor the newer version of the RF model can explain the dissociation between the effect of the previous range and the effect of previous skewing. According to the newer version, adjustment to the new stimulus context should be complete within the first 10 to 12 postshift trials. The older version is less restrictive but fails to account for the different rates of adjustment to the new range and to the new stimulus frequencies. For illustration, we calculated the mean judgments trial by trial according to the range-frequency equations (cf. Parducci, 1992) and then determined the neutral points of the series the same way as before. Figure 3 shows the results for w =.5. (Because the range-frequency equations are deterministic, the curves are smoother than those shown above.) One can see that the effects of previous range and previous skewing diminish at about the same rate. Changing the weight of the range coefficient, w, from 0.5 to 0.7 does not alter this characteristic (see Figure 4) Mean NP [Stimulus Units] HN HP LN LP Block Figure 3. Upward and downward movement of the neutral points as predicted by the older version of the range-frequency model with w =.5. Figure 4. Upward and downward movement of the neutral points as predicted by the older version of the range-frequency model with w =.7.

6 Discussion The computer simulations suggest that neither the older nor the newer version of the rangefrequency model can account for the different rates of adjustment to a new stimulus range and a new distribution. The consistency model, however, does. According to that model, it all depends on the value of the so-called forgetting parameter, f. If f is about.08, the effect of the previous range survives whereas the effect of the previous skewing rapidly disappears. Therefore, there is no need to postulate different mechanisms for range and frequency effects, even though this might be tempting at first sight. As shown elsewhere, both effects diminish at about the same rate and disappear after 8 to 10 blocks of postshift trials when f is greater than.15. It is interesting that varying the one parameter of the model can produce patterns of adjustment so different. References Atkinson, R. C., & Crothers, E. J. (1963). A comparison of paired-associate learning models having different acquisition axioms. Journal of Mathematical Psychology, 1, Haubensak, G. (1982). An extension of Parducci and Perrett's range-frequency theory of absolute judgment. In H.-G. Geissler & P. Petzold (Eds.), Psychophysical judgment and the process of perception (pp ). Berlin: VEB Deutscher Verlag der Wissenschaften. Haubensak, G. (1992a). The consistency model: A process model for absolute judgments. Journal of Experimental Psychology: Human Perception and Performance, 18, Haubensak, G. (1992b). The consistency model: A reply to Parducci. Journal of Experimental Psychology: Human Perception and Performance, 18, Parducci, A. (1965). Category judgment: A range-frequency model. Psychological Review, 72, Parducci, A. (1992). Comment on Haubensak's associative theory of judgment. Journal of Experimental Psychology: Human Perception and Performance, 18, Parducci, A., & Perrett, L. F. (1971). Category rating scales: Effects of relative spacing and frequency of stimulus values. Journal of Experimental Psychology Monograph, 89, Parducci, A., & Wedell, D. H. (1986). The category effect with rating scales: Number of categories, number of stimuli, and method of presentation. Journal of Experimental Psychology: Human Perception and Performance, 12, Poulton, E. C. (1968). The new psychophysics: Six models for magnitude estimation. Psychological Bulletin, 69, Wedell, D. H. (1984). A process model for psychophysical judgment. Unpublished Dissertation, University of California, Los Angeles. Dissertation Abstracts International, 45, 3102-B. Author note. This work was supported by DFG grant Ha 936/6. The computer programs for simulating the category ratings are available from the address above.

7