Signaled reinforcement effects on fixed-interval performance of rats with lever depressing or releasing as a target response 1

Transcription

1 Japanese Psychological Research 1998, Volume 40, No. 2, Short Report Signaled reinforcement effects on fixed-interval performance of rats with lever depressing or releasing as a target response 1 KATSUYA KITAGUCHI and SADAHIKO NAKAJIMA Department of Psychology, Kwansei Gakuin University, Uegahara, Nishinomiya 662, Japan Abstract: The effect of a diffuse auditory signal filling a brief delay between a response and reinforcement depends upon the nature of the trained target response. Rats in a signaled condition responded slower than rats in an unsignaled condition if the target response was lever release. If the target response was lever depression, however, rats in the signaled condition responded faster than rats in the unsignaled condition at the end of training Key words: signaled reinforcement, response topography, fixed-interval schedule, rats. In a typical operant chamber, a lever-press response emitted by a rat consists of two components: depressing and releasing the lever. It is conventional to define lever depression as a response, to count it, and to deliver the outcome for it (Ator, 1991). We accidentally found that if we chose lever release instead of lever depression as a target response, different results were obtained. The following is a report of serendipity (cf. Skinner, 1956) in our research on signaled reinforcement. In a recent paper (Nakajima & Kitaguchi, 1996), we reported the effects of signals of forthcoming reinforcement on lever-press performance maintained on fixed-interval (FI) schedules of briefly delayed reinforcement. Although a localized panel light signal filling the delay attenuated the overall rate of responding (Tarpy, Roberts, Lea, & Midgley, 1984), a diffuse tone signal increased it. We attributed the former effect to the interference of responding by tracking behavior to the localized signal (Iversen, 1981; Reed, 1992; Reed, Schachtman, & Hall, 1988b; Williams, 1982; Williams & Heyneman, 1982) and the latter to the facilitative effect of the diffuse signal on rapid responding before reinforcement (Reed, 1989a, 1989b; Reed & Hall, 1988, 1989; Reed et al., 1988a, 1988b; Reed, Schachtman, & Rawlins, 1992; Schachtman & Reed, 1990). After a series of experiments reported by Nakajima and Kitaguchi (1996), we did another experiment with the switching off of the house light as a signal. Assuming that this signal does not elicit tracking behavior, which interferes with lever-press responses, we expected an increment in the rate of responding with this signal, as with the tone signal. However, a 1 This research was conducted with a grant in aid for scientific research (H5-0808) from the Ministry of Education, Science, Sports, and Culture of Japan to the second author when he was a postdoctoral fellow by the Japan Society for the Promotion of Science. He was also supported by JSPS Postdoctoral Fellowships for Research Abroad when the manuscript was being written. We are grateful to Hiroshi Imada for encouragement of the research and to K. Matthew Lattal for helpful comments on a draft and for refinement of English expression. Portions of the data were reported at the 55th annual meeting of the Japanese Society for Animal Psychology, Osaka, August Japanese Psychological Association. Published by Blackwell Publishers Ltd, 108 Cowley Road, Oxford OX4 1JF, UK and 350 Main Street, Malden, MA 02148, USA.

2 Signaled reinforcement effects in rats 105 mistake occurred in electric wiring after an early session, and we, unaware of the mistake, continued the experiment under this condition, in which lever release, instead of lever depression, now triggered an input to the computer interface. The results obtained in this experiment surprised us: The signaled group responded more slowly than the unsignaled group, though the difference failed to reach statistical significance. We then conducted another experiment, still unaware of the mistake in wiring. Although this experiment used a tone signal, as in our previous report (Nakajima & Kitaguchi, 1996), rats in the signaled condition responded significantly more slowly than in the unsignaled condition. Fortunately, we had videotaped rats behavior on several sessions in this and the preceding experiments. The video prompted us to examine the equipment and we found the wiring mistake. We were uncertain whether the wiring mistake was the cause of the inconsistencies in the results. We therefore conducted another experiment, and report its results here. In this experiment, rats in the tone-signaled condition were compared with rats in the unsignaled condition. Some rats were trained to depress the lever; others were trained to release the lever. We call the former release rats and the latter depress rats for convenience. Our question here was whether the effect of the signal depends on the nature of target response. Method Subjects Subjects were 24 experimentally naive male Wistar rats housed individually in wire-mesh cages with free access to water. Throughout the experiment, they were maintained at 85% of their free-feeding weights (range g) by supplementary feeding after each session. When the experiment started, they were approximately 90 days old. Apparatus Each of two chambers 30.5 cm long, 24.7 cm wide, and 27 cm high were housed in an individual ventilated sound-attenuating shell with an observation window. Each chamber was made of stainless-steel front and back panels, two acrylic side walls, an acrylic ceiling, and a grid floor consisting of stainless-steel rods (0.5 cm in diameter) spaced 1.3 cm apart. A stainless-steel lever 3 cm wide by 1 cm thick protruded 2.6 cm from the front panel. The edge of the lever was 2.4 cm from the left wall and 4 cm above the floor. The conventional lever action from the up position to the down position required 0.10 N to operate. A food cup 1 cm wide and 2 cm above the floor protruded 1.8 cm from the center of the front panel. A pellet dispenser delivered a 45-mg pellet (Bio-Serve, DPP F-200) into the food cup with a click (92 db, Scale C). A bulb (24 V d.c.,.11 A) on the top of the front panel of each chamber was used as a house light during the session. This bulb had a white jeweled cover and was 25 cm above the floor. An electric buzzer (National EB-2114, Japan) on the front panel, 21.5 cm above the floor, was used to present a tone signal (2300 Hz, 100 db). In addition to ventilation fan s noise, a speaker in the room provided a masking noise. The background noise level in each chamber was 75 db. Wiring of the lever microswitch of each chamber and the computer program made it possible to accept an input from either lever depression or lever release. Procedure Experimental sessions were conducted 7 days a week. Acclimation to the chamber was conducted on the first day. After the food cup was filled with six pellets, rats were exposed to the chamber for 20 min during which the house light was on and the lever was inoperative. One rat was eliminated from the experiment because it did not eat the pellets during the acclimation. The experiment was conducted with the remaining 23 rats. During the next two days, 12 rats were trained to depress the lever and the remaining 11 were trained to release the lever. For the latter rats, a release response was registered when the lever moved back from the down position to the up position. The pellet was

3 106 K. Kitaguchi and S. Nakajima delivered automatically by the dispenser every 60 s. In addition, every response immediately produced one pellet and reset the 60-s timer for the free pellets. Before these sessions, the food cup was filled with pellets and the lever was baited with crushed pellets to facilitate establishment of the target response. The sessions began with the switching on of the house light and terminated with its switching off 1 s after the 30th pellet. Establishment of responding by hand shaping was conducted just after the second training session for those rats that failed to respond reliably. Each group was then divided into signaled (Sig) and unsignaled (Unsig)/lever-depression (D) and lever-release (R) subgroups: six rats for each of Groups D-Sig, D-Unsig, and R-Sig, and 5 rats for Group R-Unsig. The groups were matched as far as possible for performance during the pretraining, the order of running on daily sessions, the assigned chamber, and body weights. Groups D-Sig and D-Unsig received lever-depress training under an FI schedule with a 500-ms delay of reinforcement: The first responses after the fixed interval from the last reinforcer (or from the onset of the session for the first interval) initiated the delay and then produced the reinforcer. Groups R-Sig and R-Unsig received the same training except that the target response was lever release rather than lever depression. The delay was filled with a 500-ms tone signal for Groups D-Sig and R-Sig, but it was unsignaled by any stimuli for Groups D-Unsig and R-Unsig. Again, the sessions began with the switching on of the house light and they were terminated with its switching off 1 s after the 30th reinforcer. On the first day, the interval value was 15 s. For the remaining 15 sessions, the interval value was 30 s. Measures Each FI 30-s session consisted of 30 intervals and reinforcement delays, and the cumulative waste time. Each period of waste time was the time from the end of the interval to the response which initiated the delay and then the reinforcer. Each interval was split into 10 3-s consecutive time segments, and the number of responses during each segment was accumulated for the whole session. In addition, the number of responses during the delay was accumulated. Thus, the total number of responses in a given session was the sum of the interval responding, the delay responding, and the 30 responses which initiated the delay. The data accumulated over the entire session were used to calculate the following measures. Response rate. The sum of the total number of the interval responses and the responses which initiated the delay was divided by the sum of the total interval duration and the cumulative waste time. This is the rate except during the delay period, and we will call this measure the response rate as in the previous study (Nakajima & Kitaguchi, 1996). Delay rate. The cumulative number of delay responses was divided by the total delay duration. Waste time per interval. The cumulative waste time was divided by the number of intervals (i.e., 30). Index of curvature (IC). Calculation of this response-pattern measure was based on the method of Fry, Kelleher, and Cook (1960) for the accumulated data. Constant responding gives 0, and convergence of responding on the later segments gives a high value. With the 10-segment interval used here, the maximum value is.9, when all responses are located in the final segment. The IC was directly used for parametric analysis, but the rate measures and the waste time were subjected to a square-root transformation before the analyses because the variances in the latter measures tended to increase with the mean. All statistical tests were evaluated with respect to an alpha level of.05 unless otherwise noted. Results Each panel of Figure 1 shows group averages of each measure for 15 FI 30-s sessions. A splitplot factorial analysis of variance (ANOVA) with response (D vs. R), signal (Sig vs. Unsig), and session (repeated) as factors was used for each measure.

4 Signaled reinforcement effects in rats 107 R-Unsig R-Sig D-Unsig D-Sig Index of curvature Response rate Waste time (s) Delay rate Sessions Figure 1. Group mean measures during FI 30-s training: (a) the number of responses per minute, excluding the delay period; (b) the responses per minute during the delay period; (c) the index of curvature; (d) the waste time per interval. Groups R-Unsig and D-Unsig were trained with unsignaled delay of reinforcement, and Groups R-Sig and D-Sig received the tone filling the delay. The reinforced target response was releasing the lever for Groups R-Unsig and R-Sig, and depressing it for Groups D-Unsig and D-Sig. In the response rate (Figure 1a), there was a significant session effect, F(14, 266) = Although the other main effects and all two-way interactions were nonsignificant, the response signal session interaction was significant, F(14, 266) = This interaction seems to reflect the fact that Group R-Sig responded slower than Group R-Unsig throughout the training and that Group D-Sig responded slower on the early sessions but faster on the later sessions than Group D-Unsig. Further analyses supported this impression. For R-Sig and R-Unsig, the effects of signal, F(1, 9) = 5.93, and session, F(14, 126) = 2.04, were significant, but the signal session interaction was not. On the other hand, for D-Sig and D-Unsig, the session effect, F(14, 140) = 8.09, and the signal session interaction, F(14, 140) = 2.81, were significant, but the signal effect was not. Subsequent least-squares difference tests for D-Sig and D-Unsig revealed significant group differences on sessions 3, 12, 14, and 15. In the delay rate (Figure 1b), the three main effects were significant: response, F(1, 19) = 6.77; signal, F(1, 19) = 13.54; and session, F(14, 266) = That is, the rats trained to release the lever responded faster than those trained to depress it, the unsignaled groups responded faster than signaled groups, and all groups showed gradual increment in the response rate throughout the training. No interaction was significant. In the IC (Figure 1c), there were significant signal, F(1, 19) = 11.61, and session, F(14, 266) = 88.46, effects. Though Figure 1c suggests the signal effect was greater in R-Sig and R-Unsig than in D-Sig and D-Unsig, the response signal interaction failed to reach significance,

5 108 K. Kitaguchi and S. Nakajima F(1, 19) = 4.21, p =.054. The response effect and the other interactions were far from significance. Figure 1d shows the waste time. Only the response, F(1, 19) = 4.51, and the session, F(14, 266) = 30.70, effects were significant. Discussion These results replicated those of our previous experiments. For the rats releasing the lever, the signal attenuated the response rate and the delay rate. For the rats depressing the lever, the signal attenuated the delay rate. On the response rate measure of the rats depressing the lever, the signal initially had an attenuating effect, but it enhanced responding in the later sessions. The initial decrement was also observed in Nakajima and Kitaguchi (1996) and it was probably caused by unconditioned aversive properties of the auditory signal (cf. Reed, Collison, & Nokes, 1995). This experiment, thus, clearly demonstrated that the effect of the signal depends on the nature of the target response, depressing or releasing the lever. The signal attenuated the lever-releasing response, but enhanced the lever-depressing response at the end of training. One plausible account of this finding is that the lever release was not the operant to be affected by the consequences. In order to release the lever, rats have to depress the lever, so that the lever depression may have been the operant even in the rats which received the arranged consequences on lever releasing. The signal could have become a secondary reinforcer because of its temporal contiguity with the pellet delivery. The rats were trained under briefly delayed reinforcement schedules, thus, the delay-filling signal may have modified the situation into immediate reinforcement. Although this immediate reinforcement contingency may have increased responding of the rats depressing the lever in the later sessions, the same contingency may have had a different effect on responding of the rats releasing it. If lever depression was the operant for the latter, then the signal immediately following lever release must have been a briefly delayed reinforcement of lever depression. This is because there was the inevitable delay between the lever release and the preceding lever depression. In the absence of the signal, on the other hand, the rats may have emitted an additional lever depression during the delay which had been set between the lever release and the pellet delivery, and the lever depression may have received immediate reinforcement. This superstitious contingency (cf. Herrnstein, 1966) may have been a cause of the higher response rate of the rats in the unsignaled condition compared with those in the signaled condition. If the reasoning described above is correct, we expect that without the delay and the signal, rats trained to depress the lever should respond faster than those trained to release the lever. The former rats should receive immediate reinforcement of lever-depression but the latter should receive delayed reinforcement of that operant which preceded lever release. We examined this prediction in another experiment, but there was no significant group difference and, indeed, the group average suggested the opposite pattern. Thus, another explanation of the strange effect of the signal on the briefly delayed reinforcement for depressing or releasing the lever is required. For the rats depressing the lever, a diffuse signal can enhance responding by facilitation of rapid responding prior to reinforcement (Reed, 1989a, 1989b; Reed & Hall, 1988, 1989; Reed et al., 1988a, 1988b, 1992; Schachtman & Reed, 1990). The underlying mechanism of this facilitating function of the signal is unknown, but secondary reinforcement, delay bridging (Rescorla, 1982) and marking (Lieberman, McIntosh, & Thomas, 1979) are potential candidates (cf. Reed, 1989b). On the other hand, the attenuating effect of the tone signal on FI performance of the rats releasing the lever does not fit with this account. Instead, these results match the hypothesis that the signal stimulus attenuates the response rate under FI schedules by facilitation of temporal discrimination (Tarpy, Lea, & Midgley, 1983; Tarpy et al., 1984). The rats trained to release the lever without the signal showed poorer temporal discrimination than their signaled counterparts (see Figure 1c). If

6 Signaled reinforcement effects in rats 109 the signal enhanced the salience of the FI offset and afforded better temporal discrimination, it led to attenuation of the response rate by inhibiting futile responding during the interval (see Figure 1a). However, if this had been the case, the signal should have had the same effect in the rats depressing the lever. The signal had a small facilitative effect on the temporal discrimination of these rats (see Figure 1c) and it enhanced, rather than attenuated, the response rate of the former rats (see Figure 1a). Although the experiment reported here indicates that the signaled reinforcement effect depends on the nature of the target response, we have no satisfactory explanation of this fact. Investigation with other responses, manipulanda, species, length of delay and reinforcement schedules is necessary. The limitation of our apparatus disallowed us more finely grained recordings of events, such as each interresponse time and duration of each response. Future exploration would require such recordings to resolve the enigma underlying this phenomenon. References Ator, N. A. (1991). Subjects and instrumentation. In I. H. Iversen & K. A. Lattal (Eds.), Experimental analysis of behavior: Part 1 (pp. 1 62). Amsterdam: Elsevier. Fry, W., Kelleher, R. T., & Cook, L. (1960). A mathematical index of performance on fixedinterval schedules of reinforcement. Journal of the Experimental Analysis of Behavior, 3, Herrnstein, R. J. (1966). Superstition: A corollary of the principles of operant conditioning. In W. K. Honig (Ed.), Operant behavior: Areas of research and application (pp ). New York: Appleton. Iversen, I. H. (1981). Response interactions with signaled delay of reinforcement. Behaviour Analysis Letters, 1, 3 9. Lieberman, D. A., McIntosh, D. C., & Thomas, G. V. (1979). Learning when reward is delayed: A marking hypothesis. Journal of Experimental Psychology: Animal Behavior Processes, 5, Nakajima, S., & Kitaguchi, K. (1996). Signaled reinforcement effects on fixed-interval performance of the rat. Animal Learning and Behavior, 24, Reed, P. (1989a). Influence of interresponse time reinforcement on signalled-reward effect. Journal of Experimental Psychology: Animal Behavior Processes, 15, Reed, P. (1989b). Marking effects in instrumental performance on DRH schedules. Quarterly Journal of Experimental Psychology, 41B, Reed, P. (1992). Signalled delay of reward: Overshadowing versus sign-tracking explanations. Learning and Motivation, 23, Reed, P., & Hall, G. (1988). The schedule dependency of the signaled reinforcement effect. Learning and Motivation, 19, Reed, P., & Hall, G. (1989). The quasi-reinforcement effect: The influence of brief stimuli uncorrelated with reinforcement on variable ratio schedules. Learning and Motivation, 20, Reed, P., Collison, T., & Nokes, T. (1995). Aversive properties of auditory stimuli. Learning and Motivation, 26, Reed, P., Schachtman, T. R., & Hall, G. (1988a). Overshadowing and potentiation of instrumental responding in rats as a function of the schedules of reinforcement. Learning and Motivation, 19, Reed, P., Schachtman, T. R., & Hall, G. (1988b). Potentiation of responding on a VR schedule by a stimulus correlated with reinforcement: Effects of diffuse and localized signals. Animal Learning and Behavior, 16, Reed, P., Schachtman, T. R., & Rawlins, J. N. P. (1992). The effect of signalled reinforcement on a synthetic VI schedule. Learning and Motivation, 23, Rescorla, R. A. (1982). Effects of a stimulus intervening between CS and US in autoshaping. Journal of Experimental Psychology: Animal Behavior Processes, 8, Schachtman, T. R., & Reed, P. (1990). The role of response-reinforcer correlation in signaled reinforcement effects. Animal Learning and Behavior, 18, Skinner, B. F. (1956). A case history in scientific method. American Psychologist, 11, Tarpy, R. M., Lea, S. E. G., & Midgley, M. (1983). The role of response-reward correlation in stimulusresponse overshadowing. Quarterly Journal of Experimental Psychology, 35B, Tarpy, R. M., Roberts, J. E., Lea, S. E. G., & Midgley, M. (1984). The stimulus-response overshadowing phenomenon with VI versus FI schedules of reinforcement. Animal Learning and Behavior, 12,

7 110 K. Kitaguchi and S. Nakajima Williams, B. A. (1982). Blocking the responsereinforcer association. In M. L. Commons, R. J. Herrnstein, & A. R. Wagner (Eds.), Quantitative analyses of behavior, Vol. 3: Acquisition (pp ). Cambridge, MA: Ballinger. Williams, B. A., & Heyneman, N. (1982). Multiple determinants of blocking effects on operant behavior. Animal Learning and Behavior, 10, (Received Nov. 27, 1995; accepted Nov. 6, 1996)