Psychophysics of Reinforcement Delay

Transcription

1 Western Michigan University ScholarWorks at WMU Dissertations Graduate College Psychophysics of Reinforcement Delay Larry A. Morse Western Michigan University Follow this and additional works at: Part of the Educational Psychology Commons Recommended Citation Morse, Larry A., "Psychophysics of Reinforcement Delay" (1981). Dissertations This Dissertation-Open Access is brought to you for free and open access by the Graduate College at ScholarWorks at WMU. It has been accepted for inclusion in Dissertations by an authorized administrator of ScholarWorks at WMU. For more information, please contact

2 PSYCHOPHYSICS OF REINFORCEMENT DELAY by Larry A. Morse A Dissertation Submitted to the Faculty o The Graduate College in partial fulfillment of the requirements for the Degree of Doctor of Philosophy Department of Psychology Western Michigan University Kalamazoo, Michigan August 1981

3 PSYCHOPHYSICS OP REINFORCEMENT DELAY Larry A. Morse, Ph.D. Western Michigan University, 1981 Various models have been used to describe the nature of the relation between reinforcement delay and relative response rate. Delay has been incorporated as one aspect of reinforcement within the matching law which states that the relative response rates in concurrent schedules will equal the relative immediacy of the reinforcement. A variation of matching applicable in concurrent-chain schedules states that the relative rates of responding in the initial links will match the relative reduction in the average delay to reinforcement correlated with the entry to each terminal link. The momentary maximizing position states that the molar matching relation is a product of a more general molecular phenomenon of maximizing. Data have been provided on the sequential patterns of left and right pecks which indicate that in choice situations, responses are emitted to the key which has the greatest momentary probability (or value) of reinforcement. Because of the manner in which concurrent variable-interval schedules assign reinforcement, the probability of reinforcement for a given alternative changes as a function of time and responding. Momentary

4 maximizing in this situation averages over a session to produce the matching relation. If one assumes maximizing to be the underlying process responsible for behavior in choice situations, then one accepts its basic premise that preference is exclusive for the better alternative at the moment. If one alternative always produces the better outcome, it should always be chosen. A lack of exclusive preference could be attributed to a failure to discriminate between the alternative reinforcers or to some uncontrolled factors producing a bias to the other alternative. The problem then becomes a psychophysical one involving the reinforcer as the stimulus which can be varied along a number of physical dimensions. The present study was designed to be a psychophysical analysis on the dimension of reinforcement delay to obtain indexes of discriminability in rats. A discrete trial procedure was used where the better outcome (shorter delay) was always available for one alternative. A lamp was illuminated over the lever which produced the shorter delay. Both levers were retracted during the delay and until the start of the next trial. The absolute value of the longer delay was varied across groups of rats (4, 8, and 12 seconds). Within each group the difference between the longer and shorter delay was decreased over conditions by increasing the duration of the shorter delay. After a point of indifference was found, the difference between delays was made progressively larger. with permission of the copyright owner. Further reproduction prohibited without permission.

5 At each difference between delays, a signal detection measure of discriminability was obtained. The signal detection analysis was chosen because of its advantages over conventional psychophysics in factoring out the effects of uncontrolled biasing factors. The signal detection measure of sensitivity was found to decrease as the difference between delays was decreased. When the difference between delays was then increased, the measure of sensitivity remained low for most rats. A comparison of data across groups of rats suggested that the sensitivity measure was a function of the relative rather than the absolute difference between delays.

6 ACKNOWLEDGEMENTS I would like to express my appreciation for those who have contributed directly to the completion of this work and for those who, over the past several years, have shaped my behavior in a way which has no less directly influenced the work described in this paper. Barry Bruns, Mark Stephens, Pat White, and Jim Yanna have collectively spent two and one half years in the laboratory helping to collect the data. They have contributed greatly to the expansion and refinement of the laboratory operations and have provided the majority of reinforcement for my professional activities at Eau Claire. I would like to also thank Dr. Keith Rodewald for introducing me to the significance of behaviorism, Dr. James Kopp for strengthening my skills and expanding my inter ists in behavior analysis, and Dr. Arthur Snapper whose guidance, support, and confidence have never been absent, even through times when they were perhaps not deserved. Larry A. Morse ii

7 INFORMATION TO USERS This was produced from a copy of a document sent to us for microfilming. While the most advanced technological means to photograph and reproduce this document have been used, the quality is heavily dependent upon the quality of the material submitted. The following explanation of techniques is provided to help you understand markings or notations which may appear on this reproduction. 1. The sign or "target for pages apparently lacking from the document photographed is "Missing Page(s). If it was possible to obtain the missing page(s) or section, they are spliced into the film along with adjacent pages. This may have necessitated cutting through an image and duplicating adjacent pages to assure you of complete continuity. 2. When an image on the film is obliterated with a round black mark it is an indication that the film inspector noticed either blurred copy because of movement during exposure, or duplicate copy. Unless we meant to delete copyrighted materials that should not have been filmed, you will find a good image of the page in the adjacent frame. If copyrighted materials were deleted you will find a target note listing the pages in the adjacent frame. 3. When a map, drawing or chart, etc., is part of the material being photographed the photographer has followed a definite method in "sectioning the material. It is customary to begin filming at the upper left hand corner of a large sheet and to continue from left to right in equal sections with small overlaps. If necessary, sectioning is continued again beginning below the first row and continuing on until complete. 4. For any illustrations that cannot be reproduced satisfactorily by xerography, photographic prints can be purchased at additional cost and tipped into your xerographic copy. Requests can be made to our Dissertations Customer Services Department. 5. Some pages in any document may have indistinct print. In all cases we have filmed the best available copy. University Microfilms International 300 N. ZEEB RD., ANN ARBOR. Ml 48106

8 M o r s e,larry A l l a n PSYCHOPHYSICS OF REINFORCEMENT DELAY Western Michigan University PH.D University Microfilms International 300 N. Zeeb Road, Ann Arbor, MI 48106

9 TABLE OP CONTENTS ACKNOWLEDGEMENTS... ii LIST OF TABLES... iv LIST OF F I G U R E S Chapter I. INTRODUCTION... 1 II. M E T H O D...11 S u b j e c t s A p p a r a t u s P r o c e d u r e Preliminary training Discrimination training Psychophysical testing Psychophysical method III. R E S U L T S...27 Psychophysical Testing ROC Analysis IV. DISCUSSION B I B L I O G R A P H Y iii

10 LIST OF TABLES TABLE PAGE I. Schedules Used to Delay Reinforcement II. Sequence of Discrimination Training Conditions. 15 III. Sequence of Psychophysical Testing Conditions. 22 IV. Sequence of Psychophysical Testing Conditions. 24 V. Sequence of Psychophysical Testing Conditions. 26 VI. Hit and False Alarm R a t e s VII. Hit and False Alarm R a t e s VIII. Hit and False Alarm R a t e s IX. Hit and False Alarm R a t e s X. Hit and False Alarm R a t e s XI. Hit and False Alarm Rates XII. Equal d" Delay Differences iv

11 LIST OF FIGURES FIGURE PAGE

12 vi

13 CHAPTER I INTRODUCTION Delayed reinforcement may be defined as the interposition of a period of time between the delivery of the reinforcer and the response upon which it is contingent. There are a number of specific procedures by which delays have been imposed. Each may be described as a separate schedule of reinforcement by using the termonology of Ferster and Skinner (1957). The distinctions between them are illustrated in Table 1 in terms of the stimulus conditions and the response-reinforcement contingencies in effect during the delay period. For purposes of the illustration the delays are imposed within variable-interval (VI) schedules of reinforcement. All delay procedures may be viewed as either chain or tandem schedules since the organism's response causes a transition from an initial link schedule (in this case a VI) to a terminal link schedule (delay period). The procedure is a chain schedule if an exteroceptive stimulus change accompanies the transition; otherwise, it is a tandem schedule. In each case some delay procedures impose a contingency in the terminal link such that any response occurring in the delay restarts the. delay interval. Since this requires a time of not responding equal to the delay 1

14 2 Table I. Procedures for delaying reinforcement are described as schedules of reinforcement depending upon the stimulus conditions and contingencies in effect during the delay period. Most delay procedures involve two successive schedules with the completion of the first causing the transition to the second. If the transition is signalled, the schedules are chain schedules. Unsignalled transitions would result in their definition as tandom schedules. The terminal link may be defined as an FT schedule if food is delivered contingent upon the passage of time only, or a DRO schedule if responses in the terminal link restart the delay.

15 3 Table I Schedules Used to Delay Reinforcement Contingency Stimulus Condition in the Delay xn Delay Signalled Unsignalled None Responses Restart Delay Chain VI, FT Chain VI,DRO Tandom VI, FT Tandom VI,DRO

16 interval before reinforcement, it is a schedule of differential-reinforcement-of-other behavior (DRO). Other procedures schedule the delivery of reinforcement at the end of the delay irrespective of behavior in the delay. Here the terminal link is a fixed-time (FT) schedule. Depending upon the reasons for studying delayed reinforcement, there are advantages and disadvantages with the use of each procedure. DRO procedures provide control over the actual delay between the last response and reinforcement. However, they allow the number of responses per reinforcement to vary and the reinforced response may not be the one intended by the initial link schedule. Further, response rate reduction may be a function of the delay or the response reducing contingency of the DRO. Signalled delays involving blackouts or retracted levers also provide control over obtained delays. However, rates of responding may be influenced by the immediate conditioned reinforcement provided by the signal. While using unsignalled fixed-time delays avoids these confounding factors, they result in loss of control over the obtained delay durations since responding is likely to occur in the delay period. These problems have been most apparent in attempts to assess the importance of immediacy of reinforcement by imposing delays in single schedules of reinforcement. Some or all of the maintained responding found when signalled delays have been used (Perin, 1943; Ferster, 1953; Ferster &

17 Hammer, 1965; Silver & Peirce, 1969; Morgan, 1972) may be attributed to the conditioned reinforcement of the signal. Early attempts to reduce this potential confounding effect by eliminating the signal introduced another by using DRO terminal links to maintain control over the obtained delays (Skinner, 1938; Gonzalez & Newlin, 1976, Azzi et.al., 1964). The most recent studies (Williams, 1976; Sizemore & Lattal, 1977, 1978) have successfully minimized both of these factors by using unsignalled FT delays. However, the lack of control over the obtained delays leaves open the possibility that the maintained responding under these conditions was a function of the contiguus pairings of responses and reinforcers which may still have occurred, although less frequently than under conditions of immediate reinforcement. Within the framework of single schedules these problems seem unavoidable. Consequently, consistant quantitative relationships between reinforcement delay and rate of responding have not been demonstrated. Efforts to relate response measures to reinforcement delay seem to have been somewhat more successful when delays have been imposed in concurrent or concurrent-chain schedules. This may in part be due to the advantages of these schedules in conrtolling the confounding factors mentioned earlier. When FT signalled delays are used, the conditioned reinforcement effects of the signal may be controlled by equating the frequency and durations of such sig

18 nal presentations across both components of the schedules. Therefore, the effects of manipulating delay may be independently observed by looking at the relative rates of responding in the two components. Various models have been used to describe the nature of the relation between reinforcement delay and relative response rate. Delay has been incorporated as one aspect of reinforcement within the matching law (Herrmstein, 1970) which states that the relative response rates in concurrent schedules will equal the relative immediacy of the reinforcement. Fantino (1969) has presented a variation of matching applicable in concurrent-chain schedules stating that the relative rates of responding in the initial links will match the relative reduction in the average delay to reinforcement correlated with the entry to each terminal link. Data have been provided in support of each of these models. Chung (1965), for example, examined the effects of varying the delays of reinforcement in one component of a concurrent schedule while maintaining immediate reinforcement in the other. Delays were signalled with a blackout. Since their introduction alone could alter behavior, an equal number of response contingent blackouts were presented in the other component. These were, however, independent of reinforcement. A later analysis of Chung's data (Chung and Herrnstein, 1967) suggested that the relative rates of

19 responding on the two keys matched the relative immediacies of reinforcement. While this supported the matching law, one problem remained. The obtained frequencies of reinforcement covaried with immediacy. It is not clear to what extent this factor contributed to matching. Herbert (1970) attempted to correct this problem by using a single VI schedule to program reinforcement for both concurrent schedules. The termination of an interval made reinforcement available for the next response on one of the two keys with the key selected on a random basis with equal probability. This assured that the pigeons received an equal number of reinforcers on each key regardless of the distribution of responding on the two keys. Under these conditions matching was not found. devilliers (1977) contended, however, that both Herbert (1970) and Chung (1965) were not looking purely at the choice between delayed and immediate reinforcement. It was, rather, a choice between delayed and immediate reinforcement with the addition of response contingent punishment (blackout). Gentry and Marr (1980) may have answered this criticism by using what appears to be the cleanest control procedure yet. A reinforced response on either key initiated a delay period in a darkened chamber, followed by food presentation, and then a final time-out period in a darkened chamber. The overall duration of the sequence was the same for responses on either key. The only factor which was varied across the

20 two keys was exactly where in the sequence food was delivered (i.e. the delay of reinforcement). They kept the relative relationship between the reinforcement delays constant at 4:1 but varied the absolute values. They did not find matching at all values. Instead, when the delays were short in duration (shorter delays equal to one to eight seconds), their data more closely fit Fantino's (1969) predictions. That is, as the absolute values of the delays increased so did the preference for the shorter delay. Only at the moderate delays did the relative response rates match the relative immediacies. At longer delays preference for the shorter (relative) delay began to drop, approaching indifference when the shorter delay was equal to 32 seconds. This drop in preference is not predicted by either Herrn- stein's (1970) or Fantino's (1969) model. Gentry and Marr's data may be seen as providing some support for Silberberg's model of choice (Silberberg & Williams, 1974; Silberberg, et. al., 1978). This model, like Shimp's (Shimp 1969), argues that the molar matching relation is a product of a more general molecular phenomenon of maximizing. Both Shimp (1969) and Silberberg (Silberberg, et. al., 1978; Silberberg and Williams, 1974) have provided data on the sequential patterns of left and right pecks which indicate that in choice situations, responses are emmitted to the key which has the greatest momentary probability (or value) of reinforcement. Because of the

21 manner in which concurrent variable-interval schedules assign reinforcement, the probability of reinforcement for a given alternative changes as a function of time and responding. Momentary maximizing in this situation averages over a session to produce the matching relation. According to Silberberg (Silberberg, et. al., 1978) errors in the maximizing sequence may be made if the pigeon does not "recall the location of its last choice response. This notion was supported by using a discrete trial procedure where an inter-trial-interval (ITI) between responses could be imposed (Silberberg & Williams, 1974). When the ITI's became long ( sec.), the sequential maximizing patterns broke down. Errors due to forgetting the location of the last response predominated and behavior was randomly distributed over the two keys even when the matching law predicted a preference for one key. The blackouts after reinforced responses in the Gentry and Marr (1980) study may have served a function similar to the ITI's. As the absolute values of the reinforcement delays become long, so did the total blackout duration. The drop in preference for the shorter delay may have been due to an increase in errors in the maximizing sequence resulting from the long ( sec.) blackouts. If one assumes maximizing to be the underlying process responsible for behavior in choice situations, then one accepts its basic premise that preference is exclusive for the

22 better alternative at the moment. If one alternative always produces the better outcome, it should always be chosen. A lack of exclusive preference could be attributed to a failure to discriminate between the alternative reinforcers or to some uncontrolled factors producing a bias to the other alternative. The problem then becomes a psychophysical one involving the reinforcer as the stimulus which can be varied along a number of physical dimensions. The present study was designed to be a psychophysical analysis on the dimension of reinforcement delay to obtain indexes of discriminability in rats. A discrete trial procedure was used where the better outcome (shorter delay) was always available for one alternative. The absolute value of the longer delay was varied across groups of rats. Within each group the difference between the longer and shorter delay was decreased over conditions by increasing the duration of the shorter delay. After a point of indifference was found, the difference between delays was made progressively larger. At each difference between delays, a signal detection measure of discriminability was obtained. The signal detection analysis was chosen because of its advantages over conventional psychophysics in factoring out the effects of uncontrolled biasing factors (Green & Swets, 1966). An approach similar to that used by Gentry and Marr (1980) was used to equate the total delay sequence time for each alternative. Unlike their procedure, the sequence time remained constant across all absolute delay values.

23 CHAPTER II METHOD Subjects The subjects were six hooded rats bred at the University of Wisconsin-Eau Claire animal laboratories. At the beginning of the study they were approximately 120 days old and experimentally naive. The rats were housed in individual cages with free access to water. They were maintained at approximately 80% of their free-feeding weight by restricting their daily food ration. Apparatus Two identical Coulbourn modular test chambers (Model E10-10) were used. On one wall of each chamber were located two retractable response levers, a dry pellet dispenser, and three white signal lamps. The opening for the pellet dispenser was centered on the wall with its lower edge two centimeters above the grid floor. The retractable levers were located on either side of the opening nine centimeters above the floor and 16 cr ipart (center to center). A signal lamp was located 6.5 cm above each lever. The third lamp functioned as a house light and was centered on the wall 24.5 cm above the grid floor. Each chamber was housed in a separate isolation cubicle (Coulbourn Model E20-10). 11 With permission o,,0e copyright owner. Pul1her re p ro d u c e propmed wppou, p e n s io n.

24 12 At the start of the experiment, all environmental manipulations and data recording were done with Coulbourn solid state programming equipment. The solid state timers which determined the reinforcement delay intervals were calibrated and checked periodically by using a digital stop clock (Lafayette Model 54015). During the psychophysical testing phase, program control and data acquisition functions were switched to a PDP8a computer using the Supersked software and interfacing (Snapper and Inglis, 1978). Procedure All subjects were run seven days per week at approximately the same time each day. In all conditions, sessions were ended after 72 reinforcers had been delivered or after 72 trials in the testing phase. Immediately after each session the rats were given supplementary food sufficient to maintain their body weights at 80% of the free-feeding weights. Preliminary training In the first session, rats were magazine trained while both levers were inserted into the chamber. The next two sessions were used to shape lever pressing using the method of successive approximations. One session was used to shape responding on each lever. The alternate lever was retracted

25 13 during shaping. Presses, when they occured, were reinforced on a continuous basis. Sessions five and six were used to establish discrete trial responding on the left and right levers respectively. With only one lever inserted, a response to it caused the lever to be retracted and reinforcement delivered immediately. The lever was reinserted five seconds later. Discrimination training The purpose of this phase of the study was to establish discriminative control by the location of an illuminated lamp over discrete trial lever pressing. Pilot work had indicated that differentially delayed reinforcement would not likely produce the initial accomplishment of this goal. Therefore, the differential contingencies involved immediate versus no reinforcement. Table 2 shows the sequence of conditions used in this phase and the relation between the location of the illuminated lamp and the lever which would produce reinforcement. In each successive condition this relation was reversed. A 15 second recycling clock started each trial by inserting both levers and illuninating a lamp over one of them. On the first trial of each session and on each trial following a reinforced response, either lamp was equally likely to be illuminated. On trials following unreinforced

26 14 Table II. The sequence of conditions used in the discrimination training phase of the experiment are shown. Column two shows the relationship between the location of the illuminated lamp and the lever which produced food. In conditions la and lb the lamp was on over the lever producing food but in conditions 2a and 2b it was on over the lever on which no reinforcement was available. Also shown are the sessions to criterion and the percentage of reinforced responses (averaged over the last four days of each condition) for each rat.

27 Condition Table II Sequence of Discrimination Training Conditions State of Lamp Over Lever Producing Reinf / No Reinf Rat Sessions to Criterion % of Trials Reinforced for Last Four Days la Illuminated / Dark a Dark / Illuminated lb Illuminated / Dark b Dark / Illuminated

28 16 responses or no responses, the lamp illuminated was always the same as it had been on the previous trial. In condition 1, a press to the lever under the illuminated lamp within ten seconds from the start of the trial produced immediate reinforcement. In condition 2, it was a press to the lever under the dark lamp which produced reinforcement. In either condition, both levers were retracted and the illuminated lamp darkened upon the occurrence of any response or ten seconds without a response after lever insertion. The levers remained retracted until the start of the next trial. All rats were exposed to a regularly alternating sequence of the two conditions involving two exposures to each. The conditions were changed for a rat after it had reached a criterion of reinforced responding on 90% of the trials for four consecutive days. When this criterion was met after the second exposure to condition 2, discrimination training was complete. Psychophysical testing In testing, a similar discrete trial arrangement was maintained. However, on any trial, responses to either lever were reinforced but with different delays. The location of the illuminated signal lamp was correlated with the location of the lever which produced the shorter delay of reinforcement. A "standard" delay was always available for one response alternative defined as a response to the lever

29 under the darkened lamp. A "comparison" delay was produced 17 by a response to the other alternative, the lever under the illuminated lamp. Trials in this phase were initiated by inserting one or both of the levers at each termination of a 30 second recycling clock. A response to a lever within ten seconds from the start of the trial caused all levers to be retracted and innitiated a delay timer appropriate to the alternative chosen. The pellet dispensor was operated at the end of the delay interval. The levers remained retracted until the start of the next trial. If ten seconds passed from the start of a trial with no response, all levers were retracted until the start of the next trial. The first twelve trials were "single alternative" trials in which only one lever was inserted. Three randomized blocks of the four possible light/lever combinations were presented. These combinations were: insertion of the left lever with the lamp above it illuminated, insertion of the left lever with the lamp above it dark, insertion of the right lever with the lamp above it illuminated, or insertion of the right lever with the lamp above it dark. This arrangement gave equal exposure to both delays on each lever. Following the "single alternative" trials were 60 "two alternative" choice trials. Both levers were inserted and the signal lamp above one of them was illuminated. The location of the illuminated lamp was randomized with the

30 18 restriction that it appeared over each of the side levers equally often. Figure 1 is a state diagram of the choice trial procedure (Snapper & Inglis, 1978). Psychophylical method The psychophysical method of limits was used with one descending and one ascending series of differences between the standard and comparison delays. The standard delay remained unchanged throughout. The comparison was initiated at zero seconds and then increased over conditions until a preference for the comparison was no longer shown. The comparison delay was then decreased over conditions. Condition changes were contingent upon stability of choice behavior. Choice was considered stable if over four consecutive days the percent of responses to the comparison alternative was within a five percent range while showing no systematic increasing ar decreasing trends. The standard delay was manipulated across groups of rats with two rats being run at each of the three standard delay values (4, 8, and 12 seconds). Tables 3, 4, and 5 show the sequence of conditions and the number of sessions run in each condition for each of the three groups of rats.

31 19 Figure 1. The two alternative choice trial procedure is described in this figure using a state diagram format. Trials were started at each termination of a 30 second recycling clock. This was accompanied by the insertion of both levers and the illumination of a lamp above one of the levers. A response to the lever under the illuminated lamp resulted in food reinforcement after the comparison delay. A response to the other lever produced food after the standard delay. The levers were retracted and the light darkened during the delay and until the start of the next trial.

32 Levers Out Levers In, Left Light v On y Levers In, Right Light V On y Figure 1 Left R Left R Levers Out, Light Out. Levers Out) Light Out i Food

33 21 Table III. The sequence of psychophysical testing conditions for Group 1 are described by showing the standard and comparison delays used in each of the ascending and descending series of conditions. The number of sessions for which each rat in the group was run is also listed. Dashed lines ( ) indicate that the rat was not run under that condition. Rat 25 died in condition 9a. Subsequent reports of data from this condition for Rat 25 were taken from the last four days prior to death.

34 22 Table III Sequence of Psychophysical Testing Conditions Group I Condition Standard Delay In Seconds Comparison Delay In Seconds Sessions Rat 25 Rat 26 la a a a a a a a a a a a a d lid d d d d d Id

35 23 Table IV. The sequence of psychophysical testing conditions for Group 2 are described by showing the standard and comparison delays used in each of the ascending and descending series of conditions. The number of sessions for which each rat in the group was run is also listed. Dashed lines ( ) indicate that the rat was not run under that condition.

36 24 Table IV Sequence of Psychophysical Testing Conditions Group II Condition Standard Delay In Seconds Comparison Delay In Seconds Sessions Rat 27 Rat 28 la a a a a a a a a a a a a a a a a d d d d d lid lod d d d d d d d d Id

37 25 Table V. The sequence of psychophysical testing conditions for Group 3 are described by showing the standard and comparison delays used in each of the ascending and descending series of conditions. The number of sessions for which each rat in the group was run is also listed. Dashed lines ( ) indicate that the rat was not run under that condition.

38 26 Table V Sequence o Psychophysical Testing Conditions Group III Condition Standard Delay In Seconds Comparison Delay In Seconds Sessions Rat 29 Rat 32 la a a a a d d d Id

39 CHAPTER III RESULTS Psychophysical Testing The criterion for condition changes in the testing phase of the experiment required stability in the proportion of responses made to the comparison delay. Figures 2 through 7 show the percentage of responses made to the comparison delay for each rat averaged over the last four days of each condition. Over the ascending series of comparison delays this percentage clearly dropped for all rats from near 100% at the shortest comparison delay to near 50% at durations approaching the standard delay value. Through the descending series of comparison delays, however, the level generally remained near 50%, with rats 29 and 32 failing to show any tendency to differentially respond to the comparison delay even when it was shortened to.01 seconds. For rat 26, the differential responding recovered only at the.01 second comparison while rats 27 and 28 showed some recovery at longer comparison delays. Figures 2 through 7 also show the percentage of responses made to the prefered lever as well as the overall response probability. It is clear that as the tendency to respond to the lever producing the shorter delay diminished, 27

40 28 Figure 2. This figure shows percentages of responding of Rat 25 averaged over the last four days of each condition. The triangles represent the percent of two-alternative choice trials in which any response was made. Also shown are the percent of trials which resulted in the comparison delay (filled circles) and the percent of trials in which a response was made to the preferred lever (open circles).

41 29 Figure in OS* I. OS* COMPARISON DELAY (sec) o O o> o 00 o Ni <0 o w ONIQNOdSBU ln 30U 3d

42 30 Figure 3. This figure shows percentages of responding of Rat 26 averaged over the last four days of each condition. The triangles represent the percent of two-alternative choice trials in which any response was made. Also shown are the percent of trials which resulted in the comparison delay (filled circles) and the percent of trials in which a response was made to the preferred lever (open circles).

43 OS'O 09'I PO <D u 9 O' H fe V # % 09'Z OS'S OS'fr OS'S OS'S COMPARISON DELAY (sec) OS'L co CM OS' O) o 00 o N. o CO o W ONIQNOdSBU ln 30U 3d

44 32 Figure 4. This figure shows percentages of responding of Rat 27 averaged over the last four days of each condition. The triangles represent the percent of two-alternative choice trials in which any response was made. Also shown are the percent of trials which resulted in the comparison delay (filled circles) and the percent of trials in which a response was made to the preferred lever (open circles).

45 33 WO 00 I OO'S Figure * 00 S OO Z OO Z oo*s COMPARISON DELAY (sec) - oo*e N. CM 00* I- I S o o <0 o lo ONIQNOdS3a ln30u3d

46 34 Figure 5. This figure shows percentages of responding of Rat 28 averaged over the last four days of each condition. The triangles represent the percent of two-alternative choice trials in which any response was made. Also shown are the percent of trials which resulted in the comparison delay (filled circles) and the percent of trials in which a response was made to the preferred lever (open circles).

47 35 M T O OO'L OO'C in 0) 14 9 O' - r i i <b I I l < OO'S 00 I 00'6 00 I OO'S OO'S 00* I COMPARISON DELAY (sec) 00 CM I L o o 0> o 00 o o CO o io 0NIQN0dS3U ln 30U 3d

48 36 Figure 6. This figure shows percentages of responding of Rat 29 averaged over the last four days of each condition. The triangles represent the percent of two-alternative choice trials in which any response was made. Also shown are the percent of trials which resulted in the comparison delay (filled circles) and the percent of trials in which a response was made to the preferred lever (open circles).

49 37 W O Q < OS I- OS'fr vo <U u 9 O' H u* OS'Z OS'OI. os*ei. OS'OI. OS Z COMPARISON DELAY (sec) OS'fr o o> CM o o o> o 00 o h- <0 o in OS'I. ONICINOdSBU!N 30d3d

50 * 38 Figure 7. This figure shows percentages of responding of Rat 32 averaged over the last four days of each condition. The triangles represent the percent of two-alternative choice trials in which any response was made. Also shown are the percent of trials which resulted in the comparison delay (filled circles) and the percent of trials in which a response was made to the preferred lever (open circles).

51 39 <30 O S'I. Figure OS'Z OS'S I- COMPARISON DELAY (sec) CM co o o o 00 o <0 o 10 9NIQN0dS3H!N 30d3d

52 a lever position preference became predominant and continued 40 until a preference for the comparison delay was reestablished. The overall response tendency does not appear to systematically vary with conditions and remained at or above 80%. ROC Analysis An analysis based on the theory of signal detectability (Green & Swets, 1966) was chosen over the more traditional threshold analyses of Fechner (1860) or Thurstone (1927). Threshold analyses assume that the organism's behavior of reporting or not reporting stimuli is entirely under the control of the antecedent stimuli. The stimuli are reported each and every time that their sensory effects exceed a sensory threshold. A signal detection analysis, however, assumes the existance of a response threshold rather than a sensory threshold to accomodate the notion of consequent control of behavior. A single stimulus is presumed to be transformed into a randomly varying internal sensory effect by the addition of neural noise to the constant effect of the stimulus. This transformation process has been diagrammed by Boneau and Cole (1967) similarly as presented in Figure 8. Within this scheme, all sensory effects are perceived (i.e. there is no sensory threshold). However, when the signal distribution of effects overlaps with the distribution resulting from noise alone, some sensory ef-

53 41 Figure 8. This figure is a graphic representation of the contributions of the physical value, of stimuli and neural noise in determining the underlying signal and noise distributions refered to in signal detection theory. and s may be two values of a physical dimension of stimuli or S may be the absence of a stimulus (zero intensity) and S* some value of intensity. In the latter case, the probability distribution of sensory effects of S would result from neural noise alone. A single physical stimulus is transformed into a randomly varying distribution of sensory effects through the addition of neural noise. This is an adaptation of Thurstone's discriminal process.

54 n Probability Density Distribution of Sensory Effects Random Neural Noise Physical Dimension of the Stimuli

55 43 fects result from either signal or noise. Therefore, some signal presentations are not distinguishable from noise presentations. The subject is assumed to establish a response threshold or criterion on the continuum of sensory effects. When a sensory effect exceeds the criterion, it is reported as being a stimulus. The location of the threshold is dependent on the consequences of reporting or not reporting as well as the probability of a signal presentation. The ability to discriminate along the physical dimension of the stimulus is a function of the degree of overlap of the distributions of the sensory effects of the signal and noise. With complete overlap, each sensory effect is equally likely to have resulted from signal or noise. Signal presentations are not distinguishable from noise. On the other hand, if there is no overlap of the signal and noise distributions, then all sensory effects of the signal are different from, and therefore distinguisable from, all noise effects. The degree of overlap is a function of the separation of the physical values of the signal and noise as well as the magnitude and variability of neural noise. The degree of overlap is usually measured in terms of the distance between the distribution means in z-score units (d ). The response threshold (B) is the ratio of the heights of the distributions. Each ratio has a distinct location on the dimension of sensory effects. Both measures

56 may be determined by assuming that the distributions are normal and equally variant and then measuring the probability of correctly reporting the presence of a signal (a hit) and the probability of erroneously reporting a signal when it was absent (a false alarm). Referring to the probability density functions in Figure 8, the probability of a hit is the area under the signal curve to the right of the response criterion. That is the proportion of signal presentations which result in sensory effects that exceed the response criterion and will, therefore, be reported as signals. The false alarm rate is the area under the noise distribution to the right of the criterion. This is the proportion of noise trials which result in sensory effects exceeding the criterion and, therefore, are reported as signals. The independence of d' and B can be best observed graphically by plotting the hit rate as a function of the false alarm rate as shown in Figure 9. The receiver operating characteristics (ROC) curve at A was generated by maintaining a constant signal strength (distance between means of the probability density functions) and varying the response criteria by biasing the subject though the consequences of reporting. Bias is represented by the location of the point within an ROC curve with no bias being a location on the negative diagonal. Decreasing the signal strength and repeating the biasing process generates a new

57 45 Figure 9. Two hypothetical receiver operating characteristics (ROC) curves are shown. Each curve was generated by holding the distance between the signal and noise distributions constant and varying the placement of the response criterion along the dimension of sensory effects. The areas under the signal and noise distributions to the right of the criterion determine the hit and false alarm rates respectively. The signal and noise distributions were farther apart for curve A than for curve B.

58 46 ot Figure P(False Alarm) o o

59 47 ROC curve closer to the main diagonal at B. If the ROC curve is symetric, then d" will remain constant at all levels of bias within the curve and is reflected by the distance of the curve from the main diagonal. This relation may be most easily seen when the hit and false alarm axes are transformed to z-score or normal-normal coordinates. The ROC curve becomes a straight line parallel to the main diagonal when the signal and noise distributions are normal and equally variant. The present study used a procedure which is very much like the forced choice trial procedure used in many human signal detection studies (see Green & Swets, 1966) but not previously used with animal subjects. The defining characteristic of the forced choice method is the inclusion of two observation intervals, usually sequentially presented. One contains a signal presentation and the other just noise. The interval in which the signal is contained is randomly varied. The subject is required to indicate which of the two intervals contained the signal. This procedure is usually considered to have one advantage over most by minimizing bias since seldom are there factors which would bias a subject to pick one interval over the other. Consequently, ROC curves are usually not generated and the percent correct (as used in traditional psychophysics) is considered to be a valid index of sensitivity.

60 The present study may be viewed as incorporating two 48 simultaneously presented "observation intervals" corresponding to the insertion of the two response levers. One of the levers, when pressed, produced the comparison delay which is defined here as a signal. The other produced a standard delay which is defined as noise. The lever which produced the signal was randomlu varied from side to side but always accompanied by an illuminated light above it. The subject was expected to respond to the lever producing the comparison (usually the shorter) delay. In the present study, however, a considerable degree of bias to one interval (lever) or the other was expected since rats typically develop strong lever preferences. Varying levels of bias will alter the percent correct measure of sensitivity but should have no effect on d". Green and Swets (1966) have described a technique to conduct an ROC analysis of forced choice data under such circumstances. Each interval may be considered, separately, as a yes-no procedure from which hit and false alarm rates may be determined. Hits are responses to an interval in which a signal is presented. The corresponding false alarm rate is determined from responses to that same interval when only noise is present. Another pair of hit and false alarm rates may be determined from responses to the other interval. Two points on a single ROC curve are therefore obtained from one set of trials. As Green and Swets point out, these points

61 49 are not independently derived (a "yes" in one interval is a "no" in the other) and therefore necessarily generate a symetric ROC curve. This technique was applied in the present study by considering responding to eac*.ever separately. Since preference for the shorter delay is assumed, a response to a lever was considered a report that the shorter delay was available there. The proportion of trials in which the shorter delay was scheduled for the left lever and the rat responded on the left lever constituted the hit rate for that lever. The same was done for right lever responses to obtain another pair of hit and false alarm rates. Trials on which no responses were emitted were not included in the calculation. procedure, Due the the constraints of the forced choice the hit rate on one lever is always equal to one minus the false alarm rate on the other. Therefore, knowing one pair of hit and false alarm rates is sufficient to determine the other and to calculate d". Tables 6 through 11 show the left lever hit and false alarm rates for each rat for each of the last four days of each condition. A high hit rate relative to the false alarm rate indicates a high degree of differential responding based on delay differences. As can be seen, this relation is universal across all rats in the early conditions where there are large differences between the comparison and standard delays. Over conditions, as the comparison approaches

62 50 Table VI. Hit and false alarm Rates are presented for Rat 25 for each of the last four days of each condition. The hit rate is the probability of responding to the left lever given that the comparison delay (signal) was available there. The false alarm rate is the probability of responding to the left lever given that the standard delay (noise) was there. Only trials in which some response was made were included in the calculations.

63 51 Table VI Hit and False Alarm Rates Rat 25 Condition Hit / False Alarm Day 1 Day 2 Day 3 Day 4 la 0.97/ / / /0.03 2a 0.93/ / / /0.03 3a 1.00/ / / /0.03 4a 0.53/ / / /0.00 5a 0.83/ / / /0.00 6a 0.37/ / / /0.10 7a 0.47/ / / /0.03 8a 0.66/ / / /0.10 9a 0.47/ / / /0.55

64 52 Table VII. Hit and false alarm Rates are presented for Rat 26 for each of the last four days of each condition. The hit rate is the probability of responding to the left lever given that the comparison delay (signal) was available there. The false alarm rate is the probability of responding to the left lever given that the standard delay (noise) was there. Only trials in which some response was made were included in the calculations. Dashed lines indicate an exception to the stability criterion. In these conditionsr data analyses were based on the last three days. The requirement of four days was reduced when performance showed no upward or downward trends over an extended period of time yet the session to session variability was often greater than the stability requirement.

65 53 Hit Table VII and False Alarm Rat 26 Rates Condition Hit / False Alarm Day 1 Day 2 Day 3 Day 4 la 1.00/ / / /0.04 2a 0.73/ / / a 0.97/ / / /0.17 4a 0.93/ / / /0.13 5a 0.81/ / / /0.27 6a 1.00/ / /0.87 7a 0.97/ / / /1.00 8a 0.97/ / / /0.92 9a 0.97/ / / / a 1.00/ / / / a 1.00/ / / / a 1.00/ / / / a 0.93/ / / / d 1.00/ / / /0.90 lid 0.97/ / / /0.90 6d 1.00/ / / /0.83 5d 0.93/ / / /0.87 4d 0.90/ / / /0.83 3d 0.90/ / / /0.80 2d 0.93/ / / /0.86 Id 0.93/ / / / / / / /0.07