EFFECTS OF INTERRESPONSE-TIME SHAPING ON MULTIPLE SCHEDULE PERFORMANCE. RAFAEL BEJARANO University of Kansas

Transcription

1 The Psychological Record, 2004, 54, EFFECTS OF INTERRESPONSE-TIME SHAPING ON MULTIPLE SCHEDULE PERFORMANCE RAFAEL BEJARANO University of Kansas The experiment reported herein was conducted to determine whether interresponse-time (IRT) shaping can produce different response rates in 2 components of a multiple schedule that are equated with respect to reinforcement rate. To this end, pigeons' key pecks were reinforced with food, if they terminated IRTs that were more extreme (shorter in 1 component and longer in the other) than 20 of the most recent 24 IRTs, and an average of 30 s had elapsed since the previous reinforcement. Large intercomponent differences in response rate occurred in all of the subjects, even though the intercomponent differences in reinforcement rate were negligible. The relevance of the present findings to prior research on IRT shaping is discussed. I n a series of experiments, Alleman and Platt (1973) showed that different response rates can be generated by differentially reinforcing progressively longer or shorter interresponse times (lrts). To this end, these investigators rank ordered a number of the most recent IRTs and reinforced those that exceeded a preselected rank-a procedure known technically as a percentile schedule of reinforcement (Galbicka, 1988; Platt, 1973). Specifically, the rank above which a given IRT had to fall to be eligible for reinforcement is given by the equation k = (m + 1)(1 - w) (1 ) where k is the rank that a given IRT had to exceed to be reinforced, m is the length of a list of the most recent IRTs that is updated after every response, and w is the probability of reinforcement per IRT. In some This research was conducted at the University of Florida and prepared for publication at the University of Kansas under the support of National Institute of Child Health and Human Development Grant No. T I thank Timothy Hackenberg for the generous use of his laboratory facilities, Chris Bullock and Teresa Foster for their assistance with data collection, and Kate Saunders and Dean Williams for their helpful comments concerning this paper. Correspondence pertaining to this manuscript should be addressed to Rafael Bejarano, Parsons Research Center, P. O. Box 738, Parsons, KS ( Bejarano@ku.edu).

2 480 BEJARANO experimental conditions, for example, m was equal to 40, and w was equal to 0.1, which means that a given I RT was reinforced if it was more extreme (shorter or longer) than 37 of the most recent 40 IRTs. Because percentile schedules control reinforcement probability, Alleman and Platt attributed their findings to the differential reinforcement of extreme IRTs (i.e., to IRT shaping). Another possibility, however, is that their findings were due to the direct relation between response rate and reinforcement rate arranged by their procedures. That is, reinforcing progressively shorter IRTs would have produced response- and reinforcement-rate increases, whereas reinforcing progressively longer I RTs would have produced response- and reinforcement-rate decreases. In their fourth experiment, for example, these investigators shaped short and long IRTs in two components of a multiple schedule (i.e., in the presence of two exteroceptive stimuli that alternated), thereby producing large response- and reinforcement-rate differences between components. Numerous studies have shown that intercomponent differences in reinforcement rate are sufficient to produce intercomponent differences in response rate (for reviews, see McSweeney, Farmer, Dougan, & Whipple, 1986; Williams, 1983), suggesting that the intercomponent differences in response rate in Alleman and Platt's fourth experiment likewise may have been due to intercomponent differences in reinforcement rate, and not to IRT shaping as they supposed. In subsequent studies, Platt and his colleagues (Galbicka & Platt, 1986; Kuch & Platt, 1976) showed that IRT shaping can produce different response rates across experimental conditions that are equated with respect to reinforcement rate (i.e., in situations where the direction of IRT shaping is reversed across blocks of sessions while reinforcement rate is kept constant). Such studies suggest that IRT shaping also should be capable of establishing different response rates in two components of a multiple scheduie that yield equal rates of reinforcement. This possibility needs to be investigated empirically, however, given the aforementioned finding that intercomponent differences in response rate vary directly with intercomponent differences in reinforcement rate. The experiment reported herein was conducted to determine whether selectively reinforcing extreme IRTs can produce different response rates in two components of a multiple schedule that are equated with respect to reinforcement rate. To this end, pigeons' key pecks were reinforced with food, if they terminated IRTs that were more extreme (shorter in one component and longer in the other) than 20 of the most recent 24 IRTs, and an average of 30 s had elapsed since the previous reinforcement (i.e., reinforcements were scheduled to occur at a rate of 2 per min in both components). If IRT shaping is sufficient to produce different rates of responding under multiple schedules, then this procedure should establish significant intercomponent differences in response rate, even though it minimizes any intercomponent differences in reinforcement rate. If different rates of reinforcement are necessary to

3 EFFECTS OF INTERRESPONSE-TIME SHAPING 481 produce differences in response rate between components, however, then the procedure used in the present study should not be capable of producing large intercomponent differences in response rate, because it minimizes any intercomponent differences in reinforcement rate. Method Subjects The subjects were 4 white Carneau pigeons (Columba livia), numbered 30, 40, 77, and 863, that were maintained at 80% of their freefeeding weights by means of postsession feeding. Before serving in this experiment, all 4 birds had been exposed briefly to basic schedules of intermittent reinforcement in a laboratory course on operant behavior. The pigeons were individually housed in a colony room, where they had continuous access to water and grit. The room was illuminated on a 16.5:7.5 hr light/dark schedule. Apparatus One standard Lehigh Valley Electronics operant conditioning chamber for pigeons (35 cm long, 35 cm high, and 30 cm wide) was used. Two response keys (2.5 cm in diameter) were located on the experimental panel cm above the grid floor. The right and left keys were 16.5 and 8.5 cm from the right and left walls, respectively. A force of approximately 0.2 N was required to operate the response keys. The right key, which was the only one used in this experiment, could be lit from behind by red and white lights. A houselight (mounted 4.4 cm above the right key) provided diffuse illumination, and a centrally located aperture (measuring 5 cm by 5.6 cm and positioned 10 cm above the grid floor) provided access to mixed grain. A ventilation fan helped mask extraneous sound. Experimental events were programmed and data were collected with MED-PC software on an IBM-compatible computer located in a different room. Procedure The procedure involved a two-component multiple schedule, each component of which was associated with a different key color (red and white). The first 24 responses that occurred during the first presentation of each color in each session were reinforced on a tandem variable-time 30-s random-ratio 5 schedule (i.e., with a 0.2 probability, once an average of 30 s had elapsed since the color's onset). All subsequent responses were reinforced according to variable-time (VT) 30-s schedules that operated in tandem with percentile schedules. The VT schedules were constructed from 15 intervals according to the procedure described by Fleshier and Hoffman (1962), and the percentile schedules were programmed according to Equation 1, with values of m and w equal to 24 and 0.2, respectively. Thus, after the first 24 responses, food was made available in the presence of each key color, if an average of 30 shad

4 482 BEJARANO elapsed since its onset, and a key peck terminated an IRT that was more extreme than 20 of the most recent 24 IRTs. Before each cycle of two components, the computer sampled a probability gate to determine which component would be presented first in that cycle (i.e., if the red-key component was presented first, the whitekey component was presented second, and vice versa). To minimize schedule interactions, all lights in the experimental chamber were darkened for 30 s between components. Each component was presented 15 times in each session, for a total of 30 reinforcements per session. Sessions were conducted 7 days a week. The experiment consisted of two conditions. In Condition 1, long IRTs were shaped in the presence of red, and short IRTs were shaped in the presence of white. In Condition 2, the reinforcement contingencies were reversed, so that short IRTs were shaped in the presence of red, and long IRTs were shaped in the presence of white. Each condition was in effect for a minimum of 30 sessions, and until no increasing or decreasing trend was evident in the mean red- and white-keycomponent IRTs from each of the most recent five sessions. Thus, responding had to be stable in both components before conditions could be terminated. Results Figure 1 shows means and standard deviations of the response and reinforcement rates (responses per minute and reinforcers per hour, respectively) in the red- and white-key components. The response-rate means and standard deviations were calculated on the basis of the individual red- and white-key rates of responding in each of the last five sessions of Conditions 1 and 2, which in turn were computed by multiplying the reciprocals of the mean red- and white-key IRTs in these sessions by 60. Likewise, the reinforcement-rate means and standard deviations were calculated on the basis of the individual red- and whitekey rates of reinforcement in each of the last five sessions of these conditions, which in turn were computed by multiplying the reciprocals of the mean interreinforcement intervals in the red- and white-key components of these sessions by The top panels of this figure show that large intercomponent differences in response rate occurred in every subject. The largest difference occurred in Subject 863 (Condition 2), whose mean response rates were (SO = 5.75) responses per min in the red-key component and (SO = 2.14) responses per min in the white-key component. The smallest difference, in contrast, occurred in Subject 40 (Condition 1), whose red- and white-key response rates averaged (SO = 10.80) and (SO = 10.07) responses per min, respectively. The bottom panels of Figure 1 show that the intercomponent differences in reinforcement rate were negligible. The largest difference occurred for Subject 77 (Condition 2), whose mean reinforcement rates

5 EFFECTS OF INTERRESPONSE-TIME SHAPING ~ 180 E160 -E ~100 IJ) c 8. IJ) a:: Condition 1 Red D White Condition "- <1) 0- IJ)... <1) e.e c '(5 a:: Subjects Figure 1. Means and standard deviations of the rates of responding (top panels) and reinforcement (bottom panels) for Subjects 30, 40, 77, and 863, calculated from the last five sessions in each condition. Filled and unfilled bars show data from the red- and white-key components, respectively. were (SO = 9.05) reinforcers per hr in the red-key component and (SO = 12.77) reinforcers per hr in the white-key component. The smallest difference occurred for Subject 863 (Condition 2), whose redand white-key reinforcement rates averaged (SO = 5.56) and (SO = 8.47) reinforcers per hr, respectively. Figures 2 through 5 show relative frequency distributions of all and reinforced IRTs (in.2-s bins) for Subjects 30, 40, 77, and 863, respectively. The left-hand panels show the red-key IRTs from the last five

6 484 BEJARANO 1 0.8~ 0.6-} Red-key components II D AIIIRTS Reinforced IRTS Subject 30 Condition 1 White-key components 0.2 Condition s I RT bins Figure 2. Relative frequency distributions of interresponse times (in.2-s bins) for Subject 30, calculated from the last five sessions in each condition. Fi lled and unfilled bars show distributions of all and reinforced interresponse times, respectively. sessions in Conditions 1 and 2, and the right-hand panels show the whitekey IRTs in the last five sessions of these conditions. The filled bars show all IRTs, and the unfilled bars show reinforced IRTs. Two characteristics of these distributions are worth noting. First, shaping short IRTs produced all-irt distributions with a highly positive

7 EFFECTS OF INTERRESPONSE-TIME SHAPING 485 Subject 40 Condition 1 Red-key components White-key components g:~~ II AJllRTs ~ Reinforced D IRTs o o 1 Conoition o s IRT bins Figure 3. Relative frequency distributions of interresponse times (in.2-s bins) for Subject 40, calculated from the last five sessions in each condition. Filled and unfilled bars show distributions of all and reinforced interresponse times, respectively. skew, whereas shaping long IRTs produced all-irt distributions that are considerably flatter in shape. This difference is most striking in Subject 863's Condition-2 IRT distributions (Figure 5, bottom panels), although it is readily discernible in every case. Second, the reinforced-irt distributions are displaced to the left of the all-irt distributions in those cases wherein short IRTs were shaped (Condition 1, white-key component; Condition 2, red-key component), and to the right of the all-irt distributions in those cases wherein long IRTs were shaped (Condition 1, red-key component;

8 486 BEJARANO Subject 77 Condition 1 1 Red-key components White-key components 0.8~ _ ~. 0.6-j, _ ALL IRTs -j,- D Reinforced IRTs C/) 0 c ::J 0- ~ ~1 o ro 0.8~ ~ 0.6-)0 2 3 o 1 Condition 2 ~ 0.2 o o s I RT bins Figure 4. Relative frequency distributions of interresponse times (in.2-s bins) for Subject 77, calculated from the last five sessions in each condition. Filled and unfilled bars show distributions of all and reinforced interresponse times, respectively. Condition 2, white-key component), indicating that the procedure selectively reinforced extreme IRTs, as intended. Table 1 lists the median, and the first and third quartiles, of the latencies to the first response in each component, calculated from the last five sessions in each condition. This table shows that reversing the directions of IRT shaping in Condition 2 (i.e., toward short IRTs in the redkey component and toward long IRTs in the white-key component) had

9 EFFECTS OF INTERRESPONSE-TIME SHAPING O.8~ 0.6-j. 0 Red-key components II D AU IRTs Reinforced IRTs Subject 863 Condition 1 ~ White-key components C/) '0 c :l 0- ~ ~ +=i n:s c:: 0 1 O.8~ o 1 Condition 2 o o 1.2-s IRT bins Figure 5. Relative frequency distributions of interresponse times (in.2-s bins) for Subject 863, calculated from the last five sessions in each condition. Filled and unfilled bars show distributions of all and reinforced interresponse times, respectively. reliable, albeit small, effects on these measures. Specifically, 3 of the 4 subjects (Subjects 30, 40, and 77) showed decreases in their red-key latencies and increases in their white-key latencies, even though reinforcement was not contingent on latency duration.

10 488 BEJARANO Table 1 Latencies to the First Response in the Red- and White-Key Components of the Multiple Schedule Subject Condition Component red white red white red white red white red white red white red white red white Note. This table shows the medians (02) and the first and third quartiles (01 and 03, respectively) from the last five sessions of each condition for each subject. Discussion The results of this experiment demonstrate that differentially reinforcing extreme IRTs can establish different response rates in two components of a multiple schedule, even though the reinforcement rates in the two components are comparable. This is evident both in Figure 1 (which shows that response rates differed considerably between components but that reinforcement rates did not) and in Figures 2 through 5 (which show large intercomponent differences in the distributions of all and reinforced IRTs). Moreover, the latencies to the first response in each component also varied systematically across conditions, further attesting to the effects of the reinforcement contingencies. These findings add to the literature on behavior shaping in several ways. First, recall the aforementioned possibility, that the intercomponent differences in response rate reported by Alleman and Platt (1973, Experiment 4) might have been due to the direct relation between response rate and reinforcement rate arranged by their procedures. Although this possibility cannot be ruled out, the present findings suggest that Alleman and Platt's results also could have been a function of the differential reinforcement of extreme IRTs, as these investigators supposed. Second, by showing that differentially reinforcing extreme IRTs can establish different response rates in two components of a multiple schedule that yield comparable reinforcement rates, the present results establish the generality of prior findings, that IRT shaping can produce

11 EFFECTS OF INTERRESPONSE-TIME SHAPING 489 large differences in response rate across experimental conditions that yield comparable rates of reinforcement (Galbicka & Platt, 1986; Kuch & Platt, 1976). This is important because, as Alleman and Platt (1973) noted, the generality of percentile reinforcement schedules would be seriously limited if they could not establish different response rates in close temporal proximity to one another. Finally, the finding that the latencies to the first response in each component varied systematically across conditions is worth noting, given that reinforcement was not contingent on latency duration. The rationale for excluding the response latencies from the reinforcement contingencies was that, by definition, they are not IRTs. That is, latency is defined as the time to the first response after stimulus onset. IRT, in contrast, is defined as the interval between the end of one response and the start of the next, although it typically is measured as the time between the start of two consecutive responses. Given the present findings, however, excluding the response latencies from the reinforcement contingencies may not be necessary, as latencies and IRTs may be controlled by the same variables. Future research might be conducted to investigate this possibility. Future studies also might examine certain variables that may determine the size of the intercomponent differences in response rate that IRT shaping can generate. Two such variables that may be worth investigating are the number of observations used to determine whether a given response is eligible for reinforcement (m in Equation 1) and reinforcement probability (w in that equation), both of which have been shown to influence IRT shaping under procedures other than multiple schedules. In particular, Alleman and Platt (1973, Experiment 2) found that between-condition reversals in the direction of IRT shaping produced changes in response rate, the size of which varied directly with m and inversely with w. This finding suggests that m and walso may determine the size of the intercomponent differences in response rate that IRT shaping can generate, although, as just noted, additional research is necessary to determine whether this is the case. References ALLEMAN, H. D., & PLATT, J. R. (1973). Differential reinforcement of interresponse times with controlled probability of reinforcement per response. Learning and Motivation, 4, FLESHLER, M., & HOFFMAN, H. S. (1962). A progression for generating variable-interval schedules. Journal of the Experimental Analysis of Behavior, 5, GALBICKA, G. (1988). Differentiating the behavior of organisms. Journal of the Experimental Analysis of Behavior, 50,

12 490 BEJARANO GALBICKA, G., & PLATT, J. R. (1986). Parametric manipulation of interresponsetime contingency independent of reinforcement rate. Journal of Experimental Psychology: Animal Behavior Processes, 12, KUCH, D. O., & PLATT, J. R. (1976). Reinforcement rate and interresponse time differentiation. Journal of the Experimental Analysis of Behavior, 26, MCSWEENEY, F. K., FARMER, V. A., DOUGAN, J. D., & WHIPPLE, J. E. (1986). The generalized matching law as a description of multiple-schedule responding. Journal of the Experimental Analysis of Behavior, 45, PLATT, J. R. (1973). Percentile reinforcement: Paradigms for experimental analysis of response shaping. In H. Bower (Ed.), The psychology of learning and motivation: Advances in research and theory (Vol. 7, pp ). New York: Academic Press. WILLIAMS, B. A. (1983). Another look at contrast in multiple schedules. Journal of the Experimental Analysis of Behavior, 39,