Kathryn L. Kalafut. M.S., Brown University. Providence, Rhode Island. May 2014

Transcription

1 The Quantification of Behavior in the Presence of Compound Stimuli by Kathryn L. Kalafut M.S., Brown University Providence, Rhode Island May 214

2 Copyright 214 by Kathryn L. Kalafut

3 iii This dissertation by Kathryn L. Kalafut is accepted in its present form by the Cognitive, Linguistic, and Psychological Sciences as satisfying the dissertation requirement for the degree of Doctor of Philosophy. Date Russell M. Church Recommended to the Graduate Council Date Donald S. Blough Date Rebecca D. Burwell Approved by the Graduate Council Date Peter M. Weber, Dean of the Graduate School

4 iv CURRICULUM VITAE Kathryn L. Kalafut Date of Birth: Contact Address: Brown University Nationality: USA Birth Place: Omaha, NE Box 1821 Education Cognitive, Linguistic and Psychological Sciences Providence, RI 2912 USA University of North Texas, Denton, TX M.S. Behavior Analysis, 29 Thesis Title: The Captive Animal Activity Tracking System: A systematic method to measure captive animal welfare. Indiana University, Bloomington, IN B.A. Cognitive Science and Animal Behavior, 25 Teaching Experience Teaching Assistant Quantitative Methods in Psychology, Spring and Summer, 211, and Spring 212 with Dr. Jack Wright, Brown University, Providence, RI. Prepared and instructed the required lab portion of the course and held review sessions. Quantitative Analysis of Behavior, Fall, 211 and 212 with Dr. Russell Church, Brown University, Providence, RI. Instructed both the lab portion of class, which included the behavioral observation of live rats, as well as the data analysis portion, which included teaching students basic data analysis techniques using Matlab software. Laboratory for Genes and Behavior, Spring, 21 with Dr. Rebecca Burwell, Brown University, Providence, RI. Instructed students to implement and collect data using various behavioral tasks with live mice.

5 v Instructor Animal Communication Seminar, Fall 21 with Dr. Ruth Colwill, Brown University, Providence, RI. Held discussion groups regarding class material and supervised group projects. Introduction to Behavior Analysis, Fall, 27, 28, 29, Spring, 26 and 27, University of North Texas, Denton, TX. Developed course material to introduce students to the principles of behavior analysis. Introduction to Behavior Analysis II, Spring, 28, 29, University of North Texas, Denton TX. Extended on the pre-requisite course to include data collection methods, behavioral modification techniques and the exploration of real work applications for behavior analysis. Anatomy and Physiology Laboratory, Fall, 27, 28, University of North Texas, Denton, TX. In collaboration with the instructor for the lecture portion of this course, we prepared the course outlines to mirror one another. I personally instructed students through hands on examples and a variety of laboratory techniques. Instructor of Introduction to Biology Laboratory, Fall, 27, 28, Spring, 27, 28, University of North Texas, Denton, TX. Working with other lab instructors, I designed and implemented a curriculum to educate non-science majors using a combination of lecture, discussion and hands-on learning approaches. Posters Presented at Professional Conferences Kalafut K. L., & Church, R. M. (213). The Quantification of Overexpectation. Presented at the Quantitative Analysis of Behavior Conference, Minneapolis, MN. Kalafut K. L. (212). Invited Panel Discussant: Future Careers in Applied Animal Behavior. Meeting of the Association for Behavior Analysis International Annual Conference Seattle, WA. Kalafut K. L., Mitchell, N. & Finn, W. (212). Urban Rhode Island Coyotes: Environmental Effects on Movement Patterns. Presented at the Comparative Cognition Conference, Melbourne, FL.

6 vi Kalafut K. L. (211). Invited Panel Discussant: Research in Applied Animal Behavior. Meeting of the Association for Behavior Analysis International Annual Conference, Denver, CO. Kalafut, K. L. & Rosales-Ruiz, J. (25). Is Enrichment Enriching? Presented at the Animal Behavior Management Alliance Conference (ABMA), San Diego, CA. Kalafut, K. L. & Rosales-Ruiz, J. (25). The Effects of Using a Double Click. Presented at the Association for Behavior Analysis International Annual Conference, Atlanta, GA. Professional Presentations Publications Published Submitted In Preparation Kalafut K. L., & Church, R. M. (213). Brief Stimuli as Context. Presented at the Quantitative Analysis of Behavior Conference, Minneapolis, MN. Kalafut, K. L. & Rosales-Ruiz, J. (27). The Poison Cue and Double Clicks. Presented at Association for Behavior Analysis International Annual Conference, San Diego, CA. Kalafut, K. L. & Rosales-Ruiz, J. (27). Are Enrichment Items Enriching? Presented at the Association for Behavior Analysis International Annual Conference, San Diego, CA. Kalafut, K. L. & Rosales-Ruiz, J. (26). The Effects of Using a Double Click in Clicker Training. Presented at Taming the Domestic Beast seminar, sponsored by the University of North Texas in Dallas, TX. Kalafut, K. L., & Church, R. M. (in press). Brief Stimuli as Context. Behavioral Processes. Kalafut, K. L., & Rosales-Ruiz, J. The captive animal activity tracking system: A method to measure the behavior change of captive animals over time. Journal of Applied Animal Welfare Science. Kalafut, K. L., Freestone, D. M, MacInnis, M. L. M., & Church, R. M. Conditioning Over Time. Journal of Experimental Psychology.

7 vii Mitchell, N., & Kalafut, K. L. Identifying Types of Food Sources from GPS clusters in urban Coyotes (Canis latrans). Kalafut, K. L., Feuerbacher, E., & Rosales-Ruiz, J. The multiple functions of conditioned reinforcers and the effects on free operant behavior. Service Graduate Student Representative, Brown University, Cognitive, Linguistic and Psychological Science, 211. Vice President, Applied Animal Behavior Special Interest Group, 21. Member, Training Committee, Animal Behavior Management Alliance, Member, Proceedings Committee, Animal Behavior Management Alliance, Academic Honors Dissertation Fellowship, Brown University, 213. Recipient of the Edgar Lewis Marston Scholarship, Brown University, 21. Sharing the Knowledge Award for Behavior Research, Presented by the Animal Behavior Management Alliance, 28. Marian Breland-Bailey Exceptional Research Award in Animal Training, Presented at the Association for Behavior Analysis annual conference, 27. Recipient of the Donald L. Whaley Memorial Scholarship, University of North Texas, 26.

8 viii Preface and Acknowledgments I would like to first thank Russ Church for his help on this thesis and his support and enthusiasm during its completion. Past Timing Lab members, David Freestone and Mika MacInnis also deserve thanks for their encouragement and incredibly beneficial teachings and discussions. Thank you to my committee members, Don Blough and Rebecca Burwell, whose comments and insight were much appreciated through the development of this thesis. Last, but not least, thanks to my family and friends who have been such a huge source of support. Especially Dan Polifka, without his help, encouragement, support, and sense of humor this might not have been possible.

9 ix Table of Contents Title Page... i Copyright Page... ii Signature page... iii Vitae... iv Preface and Acknowledgements... viii List of Tables... x List of Figures... xi List of Equations... xiii General Introduction... 1 Experiment 1: Methods Experiment 1a.. 17 Experiment 1b... 2 Experiment 1 Results: Experiment1 Discussion: Experiment Experiment 2: Methods Experiment 2: Results Experiment 2: Discussion General Discussion References... 64

10 x List of Tables Table 1: Manipulations made each day of Experiment 1a Table 2: Manipulations made each day of Experiment 1b Table 3: Slope and intercept values for the best fitting lines for data collected in Experiment 1a and 1b Table 4: Coefficients and goodness-of-fit measures for data collected in Experiment 1a Table 5: Coefficients and goodness-of-fit measures for data collected in Experiment 1b Table 6: Coefficients and goodness-of-fit measures for data collected in Experiment Table 7: Coefficients and goodness-of-fit measures for average model and summation model Table 8: Coefficients for Andrew & Harris (211) Experiment Table 9: Coefficients for Andrew & Harris (211) Experiment

11 xi List of Figures Figure 1: Experiment 1: Experimental Cycles... 5 Figure 2: Experiment 1: Raw response rates Figure 3: Experiment 1: Mean response rates Figure 4: Experiment 1: Compound data and lines of best fit Figure 5: Experiment 1: Difference in the line of best fit intercept values Figure 6: Experiment 1: Rank order of rats mean rat of responding and coefficient of variation Figure 7: Experiment 1: Raw response rates and z- score response rates Figure 8: Experiment 1a: Mean response rates in compound with line drawn from linear regression coefficients Figure 9: Experiment 1b: Mean response rates in compound with plane drawn from linear regression coefficients using response rates as predictor variables Figure 1: Experiment 1b: Mean response rates in compound with line drawn from linear regression coefficients using the probability

12 xii of food as predictor variables Figure 11: Experiment 1b: Mean response rates in compound with line drawn from linear regression coefficients using the rates of reinforcement as predictor... 4 variables Figure 12: Experiment 2: Mean response rates across stimuli Figure 13: Experiment 2: Differences in mean response rates between stimuli Figure 14: Experiment 2: Mean response rates to compound and line drawn from regression coefficients Figure 15: Predictions of averaging and a weighted summation models Figure 16: Experiment 2: Lines of best fit for the combination response rates for each stimulus

13 xiii List of Equations Equation 1: The Rescorla-Wagner Model... 5 Equation 2: Rescorla-Wagner method of... 5 combination Equation 3: Rescorla-Wagner unique cue... 6 addition Equation 4: Rate Expectancy Theory... 7 Equation 5: Probability of a Response Equation 6: Linear Regression Model Equation 7: Aikake Information Criteria (AIC) Equation 8: Averaging Model... 51

14 1 General Introduction Animals behave in complex environments where cues provide information regarding the current environmental contingencies. Multiple environmental cues may provide consistent information, or they can provide conflicting information. How an animal responds in the presence of multiple cues that provide conflicting information should provide insight into how the animal combines these multiple sources of information. Identifying this combination mechanism has been a persistent focus of experimental psychology since the late 196s (see Kehoe & Macrae (22), and Brandon, Vogel and Wagner (22) for summaries and extensions). One method is to bring behavior under the control of two independent cues and then test the behavior in the presence of a compound stimulus consisting of both individual stimuli. Decades of empirical research have shown that animals respond more to the compound stimulus than to either of the elements alone, a result termed summation (Hull, 194; Pavlov, 196; Wolf, 1963; Weiss, 1964; 1969; Miller, 1971; Kehoe & Gormezano, 198; Bellingham & Gillette-Bellingham, 1985; Kehoe, 1988; Rescorla, 1997; Gomez- Sancho, et al, 213). Perhaps the first example came from Pavlov. He observed more drops of saliva in the presence of multiple stimuli that were previously followed by food, than either of the stimuli had when presented individually (Pavlov, 1927, pg. 79; Kimble, 1961). This general effect has been replicated again and again in many different variants, including different schedules of reinforcement (Wolf, 1963; Weiss, 1964; Miller, 1971), different stimulus intensities (Kehoe, 1982), stimulus durations (Kehoe, 1986), stimulus modalities (Kehoe et al. 1994), and the proportion of element trials vs. compound trials (Kehoe, 1986). Interestingly, it has not yet been studied rigorously in cases where the

15 2 outcomes of the stimuli (as opposed to properties of the stimulus) have been varied systematically. The term summation suggests a specific type of combination, that the response rate in the compound is the sum of the response rates in the elements. But a direct summation of the elements is not necessarily what has been observed (Miller & Price, 1971; Weiss, 1972; Rescorla, 1997). In nearly all experiments, researchers have focused on the ordinal relationship between the stimuli (Kehoe and Macrae, 22), in that one can expect responses in the presence of the compound to be greater than those made in the presence of the individual stimuli. If two stimuli that make up the compound are trained to have inhibitory properties (behavior in their presence is inhibited), the response rate to the compound may be lower than the response rate to either of the two individual stimuli. Miller (1969) exposed rats to a contingency where the presentation of a tone was followed by a shock, and the presence of a light was also followed by shock. In compound trials, when the light and tone were presented together, the rats responded far less in the presence of the compound than in the presence of either stimulus alone. This outcome has been replicated and extended upon (Van Houton et al. 197). This result is often interpreted as a summation of the conditioned inhibition from each stimulus, but a more precise quantification is needed. In the examples above, the outcomes of each stimulus were similar, in that both were followed by either food or shock. When individual stimuli are not trained in the same way, other results are typically found. Specifically, if one of the stimuli is followed by food, and the other is followed by shock, the rate of responding to the compound of

16 3 these stimuli is typically between the response rates to the individual stimuli. For example, following one stimulus with a shock during training (CS! ) and a different stimulus with no consequence (CS! ), when the two stimuli are presented together rats responded to the compound more than what was observed to CS! alone, but less than to CS! alone (Rescorla, 1969). This result is typically referred to as averaging (Weiss, 1972). The result is certainly consistent with what is typically thought of as an average, but it can also be thought of as a summation if one (but not both) of the stimuli inhibits behavior. Without a better understanding of the combination rule used to produce the response rate in the presence of the compound, only qualitative descriptions of behavior can be assigned. In this case, average is a qualitative description of the behavior in the presence of the compound stimulus. A direct summation (the response rate in the presence of Stimulus A plus the response rate in the presence of Stimulus B) would account for the greater than result that is so often observed in the presence of the compound. But are animals actually adding their response rates in the presence of the individual stimuli together when faced with a compound? There are many ways in which a greater than result can be obtained mathematically, and an additive process is only one of them. This can also be said for the description of averaging, there is more than one method of combination that would account for a response rate to the compound that falls between the rates observed in the individual stimuli. But before the combination rule can be identified, the question of what is being combined to produce the response rates in the compound should also be addressed.

17 4 In the experiments referenced above, it is the behavior during the compound that provides insight into the combination process that takes place to produce this behavior. Systematic changes in the treatment of the individual stimuli and testing in compound can shed light on the combination process. While this has not been done to quantify the summation result, changes in behavior as a result of changes in stimulus properties have been explored extensively. For example, in an experiment conducted by Blough (1972) pigeons were reinforced for pecking a keylight, presented for 3 s, of a particular wavelength. He also trained pigeons to expect food approximately 128 s following the last food delivery. Following training, Blough measured the rate of responding to keylights at different wavelengths during different times between food deliveries. The results indicated that the pigeons modulated their pecks accordingly; the more similar the wavelength was to the reinforced wavelength, and the closer to the estimated time to food, the more the pigeons pecked. The interpretation of these results is that the pigeons have some expectation of food given a particular wavelength, as well as the time of food delivery (which Blough termed certainty ). Changes in properties of the stimuli (here the wavelength and time to food) indicate that both of these features have effects on peck rates. The control that has been demonstrated by altering the physical properties of the stimuli, as described above, can also be done when the outcomes of the individual stimuli are manipulated. To account for the behavior in the presence of compound stimuli, two questions need to be answered: what is combined to produce the behavior during the compound, and how is it being combined. These questions have been asked most commonly by making general comparisons between various behavioral models (Myers et

18 5 al. 21; Andrews & Harris, 211). Specifically to the summation and averaging outcomes, behavioral models make assumptions about what the animals learns, and then propose the method in which what is learned is combined to best account for empirical results. In the following section, four behavioral models will be discussed. Emphasis will be placed on how each of these models answers what is being combined, and how is it being combined to account for the behavior observed in the presence of the compound stimulus. Rescorla-Wagner Model The Rescorla-Wagner model (1972) is one of the most influential models of behavior in experimental psychology (Siegel & Allen, 1996). Equation 1 describes associative strength of a given stimulus (V) as a function of the number of trials (n). Equation 1 ΔV!!!! = α! β(λ V!"! ) V!!!! = V!! + ΔV α! is the salience of the stimulus on trial X, β is the strength of the reinforcer value, λ is the maximum amount of conditioning for the reinforcer, and V!"! is the total associative strength of all stimuli on trial n. The parameter V can be interpreted as the expected value of reward on a given trial, and is the expected value of lambda as n increases (Sutton & Barto, 1998). The rule used to account for the combination of associative strengths in the presence of a compound stimulus within the Rescorla-Wagner model can be described as, Equation 2 V! = V! + V!

19 6 A and B denote the individual stimuli, V represents their associative strength, and V! represents the associative strength of the compound stimulus. This equation formalizes the idea that the associative strengths of each stimulus are added together to account for the behavior in the presence of the compound. This can account for summation results that have been shown empirically, in that the associative strength of the compound is greater than that of the individual elements (Kremer, 1978; Miller, et al. 1995). If associative strength is the value that drives response rate, and the associative strengths for each stimulus are summed (as stated in Equation 2) this cannot account for the results where the response in the presence of the compound is greater than the summation of its parts (Wolf, 1963; Miller, 1971, Weiss, 1972). To account for this result, as well as a number of other empirical results (i.e. negative and positive patterning; (see Myers, et al. 21)) a hypothetical third stimulus has been added to this model. This hypothetical third stimulus represents the unique element of the compound that is not present when A or B is presented alone. This hypothetical stimulus is necessary to increase the positive associative strength of the other two elements beyond their sum. This idea is referred to as the unique cue hypothesis, and has served as an addition to the original model (Rescorla, 1972b; 1973). The combination rule with the addition of the unique cue hypothesis is shown here, Equation 3 V! = V! + V! + V! In addition to the associative strength of the individual stimuli (V! and V! ) there is the associative strength of the compound cue, V!. The sum of these three values (V!, V!, and V! ) will produce associative strengths that are greater than the sum of the two individual

20 7 stimuli alone. Based on the assumptions of the Rescorla-Wagner model, the behavior in the presence of compound stimulus is determined by the summation of the associative strength (a theoretical feature) of the individual stimuli that make up the compound. Rate Expectancy Theory In contrast to assuming that the associative strengths of individual stimuli are combined, Rate Expectancy Theory (RET; Gallistel & Gibbon, 2) suggests that animals develop estimates of the time to food for individual stimuli, and these values are what are summed to produce the response rate in the compound. This model suggests that the rate of responding reflects an animal s certainty that a given CS has an effect on the overall rate of reinforcement. Similar to the Rescorla-Wagner model, RET assumes a direct summation, but the summation is not associative strength values, it is the estimated rate of reinforcement, shown here, Equation 4 λ!" = λ! + λ! λ represents the learned estimate in the reinforcement rate for the individual stimuli, A and B. For example, if a light signals that food will be delivered in 2 seconds, λ! = 1/2, and a tone signals food will be delivered in 1 seconds, λ! = 1/1, during the compound the expected rate of reinforcement is 1/2 + 1/1 1/6.6. In words, when rates sum, the result is a new, smaller, rate. The rat expects food to occur more often than in either element alone. The rate expectancy theory suggests that behavior in the compound stimulus is a result of a summation. The features that are summed are the experienced rates of reinforcement given each stimulus. Pearce

21 8 The model suggested by Pearce (1987; 1994) is similar to the Rescorla-Wagner in that stimuli gain associative strength based on their pairings with food, but these associations develop in a different way. The model utilizes the notion of stimulus generalization to account for the development of a stimuli s associative strength. In this model, the generalization that stimulus AB receives from A trials is determined by the proportion of each stimulus constituted by the shared element A (similarly with Stimulus B). Based on this model, the similarity of A and B to AB is.5 (Pearce & Bouton, 21). Therefore the estimated associative strength of the compound presentation of AB would be equal to that of A or B presented alone. This does not produce summation. In order to account for summation, Pearce suggests that the context, or background stimuli present during an experiment, plays a role in the accumulation of associative strength. If the contextual stimuli are considered, Stimulus A is no longer simply A it is AX, X referring to the background stimuli that accompany A. If reinforcement follows Stimulus A, A will gain positive associative value, and so will X. X will also gain associative value in the cases where Stimulus B is present and followed by reinforcement. The associative value gained from X in both AX and BX trials, allows for the total associative value during a compound trial to be greater than what was observed in the presence of either AX or BX alone. This model suggests that behavior in the compound stimulus is a result of the summation. The features that are summed are the associative strengths of the individual stimuli and the associative strength of the context. Packet Theory The last model that will be discussed in terms of the method and content of the combination rule use to produce behavior in a compound is Packet Theory. Packet

22 9 Theory has been used to describe bouts of responses in time in a variety of experimental procedures (Kirkpatrick, 22; Guilhardi, et al. 25). The input to this model is the procedure, including the onset and termination of stimuli and the time of food deliveries. The models output, the times of responses, are calculated using four modules: perception, memory, decision and responses. The model suggests that rats have a method to track the time to food, and their decision to respond is a function of this estimation of the time to food. When the current time to food exceeds a threshold for responding, a packet of responses is predicted to begin. The combination rule that accounted for the empirical results best was a weighted sum applied to each of the time markers. This model suggests a weighted summation of the estimated time to food provided by each time marker. Model Comparison The models discussed above attempt to account for the changes in the rate of responding observed in the compound stimulus by assuming what feature(s) of the environment or stimuli account for these response rates. These models suggest different ways in which these features are combined. Though it is not done often, it is possible to compare these models directly by fitting the models to empirical data (Thein et al. 28). This would allow for models to be compared and to gauge the accuracy of their assumptions. For example, RET assumes a summation of reinforcement rates, while the Rescorla-Wagner model assumes a summation of the individual stimuli s associative strengths. Both of these models provide predictions regarding behavior in the presence of a compound stimulus, but which can account for empirical data better? In a paper by Andrew & Harris (211), the authors asked this question and pit these models against one another.

23 1 The Rescorla-Wagner model does not have a component to deal with time and RET is based on the estimated time to food. In order to compare these models directly, modifications had to be made. To make formal predictions the authors need to identify what has been learned regarding each stimulus (V for the Rescorla Wagner model, and λ for RET) and how this changes as a function of reinforcement rate. Because RET assumes that what is learned is the rate of reinforcement, the relationship between the learned rate of reinforcement and the actual reinforcement rate is linear. The Rescorla- Wagner model is less straightforward. Andrew and Harris assume that the associative value of the stimuli decrease from moment to moment when a reinforcer is not present. If this is true, the relationship between V and reinforcement rate is hyperbolic. The differences between these two accounts (a hyperbolic relationship, Rescorla-Wagner, and a linear relationship, RET) can be used to test empirically distinct predictions made by these models. Each model assumes that what is learned about each stimulus is summed. That is, when two CS s are trained and presented in compound, the Rescorla-Wagner model will sum according to Equation 2 and RET will sum according to Equation 4. Now, imagine the rat is trained on a third stimulus, X, with a reinforcement rate equal to the sum of the other two stimuli (A and B). Because RET assumes what is learned is the rate of reinforcement for each stimulus, and these rates are added, RET predicts that the rats will respond to Stimulus X exactly as they respond to the AB compound. In Andrew and Harris version of the Rescorla-Wagner model, the relationship between associative strength V and the reinforcement rate λ is hyperbolic. This means that, as the

24 11 reinforcement rate grows, the change in V becomes smaller and smaller. In this case, the prediction is that V! + V! > V!. To test on these predictions of each model, the authors used a delay conditioning procedure where two CS s of different modalities (e.g. a light and a tone) were presented for different durations of time; their termination was followed by food. The duration of Stimulus A was drawn from a uniform distribution with a mean of 5 s and a range of 2 s to 98 s; the duration of Stimulus B was drawn from a uniform distribution with a mean of 75 s and a range of 2 s to 148 s. They also trained a third stimulus that was terminated with food following an interval equivalent to the estimated rate of reinforcement of Stimulus A and Stimulus B combined. Following Equation 4, this rate can be calculated as 3 s (1/75 + 1/5 = 1/3). Rats responded in the presence of the compound Stimulus AB at the same rate observed in the presence of the third stimulus, X. These results suggest that the rate of responding during the compound is better predicted by a summation of the response rates than by the associative strengths of the stimuli. In the second experiment rats were trained with four separate CS s, each with a different reinforcement rate ((A) 1/3 (2 s - 58 s), (B) 1/5 (2 s - 98 s), (C) 1/75 (2 s s), (D) 1/15 (2 s s)). The rate of responding during compound presentations of Stimulus BC and CD were compared to the response rates made during the individual CS s. If the results from experiment 1 were replicated, one would expect that the observed response rates in the presence of BC would be the same as A, and the response rates in the presence of CD would be the same as B. These changes in methodology were made to account for response rates in the presence of compounds that may have been limited by a ceiling effect. The authors also wanted to be able to account for the

25 12 possibility of stimulus generalization between stimuli of the same modalities. The results indicate that the rate of responding in the presence of BC was indeed the same as that observed in the presence of A, and the response rate observed in the presence of CD was the same as that observed in the presence of B. Again, these results suggest that rats sum reinforcement rates, not associative strengths. In the last experiment conducted by Andrew and Harris, and the most pertinent to the experiments to follow, the authors manipulated the time to food in a different way in order to test the generality of their findings. All of the trained CS s had the same mean time to food (1 s and a range from 2 s to 18 s) but the probability of food delivered following their termination was manipulated. It was delivered either 1% of the time (Stimulus A), 33% of the time (Stimulus B), or 17% of the time (Stimulus C). Using the same logic as above, if rats are combining the rate of reinforcement, one would expect that the rate observed in the presence of CD to equal the response rate observed in the presence of B. Alternatively, the rate of responding in the presence of B, and the rate of responding observed in the presence of CD should be less than the rate observed in the presence of A. This is indeed what was found. Rats respond more in the presence of A than any other element or compound. The rate of responding in the presence of CD was equal to that observed in the presence of B. Overall, these results suggest that in a delay conditioning preparation, a summation of the estimated rates of reinforcement accounts for the empirical data better than a summation of the associative strengths of the stimuli. Beyond the specific results found here, the question being asked by the authors is one of the questions emphasized in this thesis; what is being combined to account for behavior during compound stimuli?

26 13 The authors conclude that RET accounts for behavior better than Rescorla- Wagner model for these data, that is, the time to food is the feature being summed. The authors accept the assumptions made by each of these models, like the assumption that the rats are summing (two features are added together to account for behavior in the compound). The authors do not question how the combination takes place, only, which behavioral models assumptions account for the behavior better. The questions asked in this paper approach those posed here, but are limited to the assumptions made by the models being compared. Model Free Accounts of Behavior in Compound An alternative to the assumptions of behavioral models is to use assumptions well established in probability theory. If the probabilities of a behavioral response in the presence of two stimuli (P! and P! ) are independent, the probability of a response in the presence of the compound stimulus (P!" ) can be estimated by the following equation Equation 5 P!" = P! + P! (P! P! ) A and B denote two different stimuli, and P represents the percent of trials in which a response was observed in the presence of that stimulus. This equation only relies on the behavior observed in the presence of the individual stimuli; no assumptions are made regarding what is learned. Kehoe (1982; 1998; Kehoe & Gormezano, 198; Kehoe & Graham, 1988) tested this equations ability to predict the conditioned responses in the presence of a compound stimulus with data collected using a rabbit eye-blink paradigm. Kehoe found that this equation systematically mapped onto the observed responses to compound stimuli, but the predictions made by this equation were consistently lower than

27 14 the observed data by an error ranging between 2% and 1% (Kehoe & Graham, 1988; Kehoe, 1998). This equation is referenced in papers and books as the way to predict a response in the presence of the compound when using a classical conditioning paradigm (Kehoe, 1998; Kehoe & Macrae, 22). A clear benefit of this equation is that there are no free parameters. Each element in the equation comes directly from the animal; there is no need for an interpretation or assumption regarding what is learned. There are also shortcomings. The use of this equation does not require nor provide any additional information regarding behavior, for example, the time of behavior, its intensity, or how it might change over trials. This equation can provide no insight into what is being combined to produce the behavior in the compound. Using a discrete behavioral variable, such as the occurrence of an eye blink, limits the interpretive value of the results. Quantification of Combination Rule Behavioral models have suggested methods of combination, but rely on assumptions about what is being combined. Model-free attempts have focused on the prediction of a response in the presence of the compound and cannot address questions regarding what features of the stimuli are combined. What is lacking is a general method to account for the behavior in the presence of a compound stimulus. The purpose of this thesis is to address these issues and to extend the current state of knowledge regarding the combination rule used by animals when presented with a compound stimulus. The following experiments will vary the probability of reinforcement in the presence of two individual stimuli before testing in compound. Manipulating the probability of reinforcement allows for values along a complete scale (

28 15 to 1) and the combinations of these values to be explored. The data collected in these experiments will then be used to answer the two questions in focus here; first, how are animals combining features of individual stimuli, that is, identify the combination rule that can account for the various outcomes that have been observed in the presence of compound stimuli. Second, identify what features of the stimuli are being combined. More specifically, isolate the specific feature(s) of the environment or experimental methodology that can account for the behavior in the presence of the compound. These questions will be answered through the use of a linear regression. The input of the linear regression are the response rates observed during training and testing, and the features of the environment or experiment that may be controlling this behavior, for example, the probability of food assigned to each stimulus. The ability to supply different features to the same regression model allows for comparisons regarding which features best account for the observed response rates. The outputs of the linear regression are the estimated coefficients or weights, for each of the stimuli that make up the compound. These weights, in turn, can be used to ask and answer more specific questions regarding the combination rule used by both rats (Experiments 1a and 1b) and humans (Experiment 2). Methods Subjects Twenty-four male Sprague Dawley rats (Taconic Laboratories, Germantown, NY) were used in these experiments (12 rats in Experiment 1a and 12 rats in Experiment 1b). The rats were kept in a colony room on a 12:12 light-dark cycle (lights off at 8:3am). Dim red lights provided illumination in the colony room and testing rooms. The rats were

29 16 3 to 37 days old upon arrival and weighed between 75 and 1 grams. During the first five days following their arrival the rats were fed freely. All of the following days all rats daily food (FormuLab 58) amount was rationed to maintain their weight at 85% of the free-feeding weight. Water was available ad libitum in both the home cage and the testing chamber. The reinforcers used in the experimental chamber were 45-mg Dustless Precision Pellets (Bio-Serv, Rodent Grain-Base Formula, Frenchtown, NJ). Apparatus Twelve experiment chambers (Med Associates, dimensions 25x3x3 cm) contained in a sound-attenuating box (Med Associates, dimensions 74x38x6 cm) with a fan for ventilation were used. Each experiment chamber was equipped with a pellet dispenser (Med Associates, ENV-23) on the front wall that delivered the reinforcer into a food cup. A head entry into this cup interrupted a photo beam (Med Associates, ENV- 254). On the opposite wall, a water bottle protruded into the chamber allowing ad libitum access to water during the session. A lick on the spout of the water bottle completed an electric circuit, which allowed each lick on the spout to be recorded. A white noise generator (Med Associates, ANL -926), and a house light (Med Associates, ENV-227M) were mounted on the back wall (near the water bottle). One Dell Pentium III/5 computer running the MED-PC for Windows Version 1.15 using Medstate Notation Version 2. (Tatham & Zurn, 1989) controlled experimental events and recorded the time at which events occurred. The interruption of the photo beam, the completion of the lick circuit, and the noise and light onsets and terminations were recorded in time-event format with 2-ms accuracy. Lever Training

30 17 Rats were first trained to press the lever located next to the food cup within the experimental chamber. A lever press training session lasted two-hours, or until all available foods were earned. The first 3 lever presses of the session were followed by a food pellet (FR-1). Following these 3 foods, a fixed ratio ten (FR-1) schedule was implemented, where a food was delivered following every 1 th lever press. Thirty additional foods were available on this schedule. Lever press training was complete when all rats received all 6 pellets within the two-hour period. All rats met this criterion within two training sessions. Experiment 1a Experimental Task. The stimuli used were the house light and an 8-db white noise. At the beginning of each session the left lever was inserted into the box and a random 16 s interval was initiated where no stimuli were presented and lever presses had no programmed effect. Following this interval, one of the two stimuli (light or noise) was presented with equal probability. In the presence of either stimulus, the first lever press following a random interval schedule with a mean of 4 s (RI-4) terminated the stimulus and food was delivered probabilistically. What differed between the presentations of either stimulus was the programmed probability of food being delivered following the first lever press after the RI-4 s interval. Table 1 shows the assigned probability of food for each stimulus across each phase of the experiment. Note that the probability of food assigned to the stimulus listed as A remained at.9, while the probability of food assigned to the other stimulus, B, was altered about every six days (approximately 325 cycles). In Table 1, the first probability assigned to Stimulus B,.9, lasted from day one to day six. The next assigned probability for Stimulus B was.33

31 18 and is listed as beginning on day 15. The reason for the gap in days is due to an attempt to counterbalance the order rats received the probabilities of food given Stimulus B. Half of the rats were moved from a probability of food equal to.9 to.66---a change in probability that appeared to be too small for their behavior to adjust in the time frame allocated for this phase. Once this was noted, Stimulus B was assigned a probability of food equal to.33 for all rats. Because of this, half the rats received more training days in this phase than the others. This additional training did not change behavior across rats at the time of testing. This also meant that the order in which the rats received the probability of food for Stimulus B was not counterbalanced. All rats received the order indicated in Table 1. The modality of the stimulus serving as Stimulus A and B was counterbalanced across rats. Experiment 1a Training Days p(food Stimulus A) p(food Stimulus B) Table 1. The probability of food delivery assigned to each stimulus on each day of Experiment 1a. Rats were tested for three two-hour sessions each day. Following each two-hour session, there was a programmed 2-minute break during which the lever was retracted and no stimuli were presented. Following this 2-minute break, the lever was reinserted and a new two-hour training session began with the RI-16 s interval where no stimuli were present and presses had no programmed effect. Figure 1 provides examples of the four different cycles that would be experienced by the rat within a two-hour test session.

32 19 There were four cycle types: one of the two stimuli were presented, and food was either delivered or not delivered depending on the probability of food assigned. p(food Stimulus A) =.9 p(food Stimulus B) =.1 RI-16s RI-4s RI-16s RI-4s RI-16s RI-4s RI-16s RI-4s Figure 1. The four cycle types experienced during training in Experiment 1a. All cycles began with a RI-16s inter-cycle interval before either Stimulus A or Stimulus B was presented. The stimulus remained on until the first lever press (arrow) following the RI- 4 s interval (white triangle), and food was delivered probabilistically (black triangle). The left panels show cycles including Stimulus A when food was delivered (top panel) or not (bottom panel). The right panels show Stimulus B cycles with food delivery (top panel) and no food delivery. Following approximately six days of training, test cycles were added. On these test days, the same cycle types shown in Figure 1 were presented, and the probability of food assigned to these cycle types remained intact, but test cycles were also presented. Test cycles occurred randomly throughout sessions, with a probability of.2. In test cycles both Stimulus A and B (noise and the light) were presented simultaneously for the duration of the cycle. Test cycles were the same as training cycles in that the first lever press following a RI-4 s schedule terminated both stimuli at the same time, and initiated the RI-16 s interval. No food was delivered at the end of any test cycle. Experiment 1b Experiment 1b was carried out with 12 different Sprague-Dawley rats which were brought in at the same age, handled, and trained to press a lever in the same manner discussed above. Experiment 1b was conducted with a near identical experimental set up

33 2 to Experiment 1a, with the exception that the interval initiated when a stimulus was presented was an RI-3 s interval (above it was an RI-4 s), and the time between stimulus presentations was an RI-12 s interval (it was RI-16 s in Experiment 1a). The interval lengths between Experiment 1a and Experiment 1b were of the same proportion (1:4 stimulus interval to between cycle interval). Because the reduced intervals allowed for more cycles to be completed within a session, training in Experiment 1b was lasted approximately four days followed by a single day including test cycles. Figure 1 can be used again as a reference for the four cycles types a rat would experience during training, with the exception to the changes made in the cycle and interstimulus interval lengths. Similar to Experiment 1a, all 12 rats ran the experimental task for three, two-hour sessions each day, and each session was separated by a 2-minute break. All 12 rats were exposed to different probabilities of food given both Stimulus A and Stimulus B. Table 2 shows the assigned probability of food given each stimulus throughout the experiment. Note that the probability of food for Stimulus A was altered approximately every 15 days, while the probability of food given Stimulus B was altered every five days. This allowed for all rats to be exposed to each probability and the combinations of these probabilities. The probability of food in the presence of Stimulus A was presented in the same order for all rats,.5,.1 and.25. The order in which the probability of food assigned to Stimulus B was presented was counterbalanced across rats.

34 21 Experiment 1b Training Days p(food Stimulus A) p(food Stimulus B) Table 2. The probability of food delivery assigned to Stimulus A and Stimulus B across the three phases of Experiment 1b. During the last day of training, test cycles were added. They occurred randomly throughout sessions with a probability of.2 (same as in Experiment 1a). Test cycles included presenting both stimuli simultaneously. The first lever press following the RI- 3s schedule terminated both stimuli at the same time, and initiated a RI-12s. Food was never delivered during any test cycle. Data Analyses Results Over the course of training, rats adjusted their rate of responding as a function of the probability of food. Figure 2 shows the mean response rate across rats for individual cycles across the 5 phases of Experiment 1a. These data come from the final day of training in each phase to emphasize the difference in the rates of responding when the probability of food is altered. The probability of food assigned to Stimulus B is shown in ascending order (.1,.33,.5,.66,.9). The open circles are the mean response rate across rats for the cycles where Stimulus A was present; the probability of food given Stimulus A remained at.9 for the entirety of Experiment 1a. The black diamond

35 22 markers are the mean response rate across rats for the cycles when Stimulus B was present; the probability of food assigned to Stimulus B varied across the experiment. There is variability within the response rates, but overall the rate of responding changes with the changes made in the probability of food; the higher the probability of food given a stimulus, the higher the rate of responding in the presence of that stimulus. Mean Responses per Minute p(food A) =.9 5 p(food B).1 p(food B).33 p(food B).5 p(food B).66 p(food B).9 Cycles Figure 2. The mean response rates across rats in the presence of the Stimulus B (filled diamond markers) and Stimulus A (open circle markers) in each phase of Experiment 1a. The probability of food given stimulus A remained at.9. During testing, rats were exposed to cycles including Stimulus A, Stimulus B and compound test cycles. The mean rates of responding across rats in the presence of these three cycles is shown Figure 3, all are fit with the lines of best fit. The slope and intercept values for each line are shown in Table 3. Two main features of Figure 3 stand out. The first is the similarity in the slope of the best fitting line for Stimulus B and the compound stimulus within each panel. No statistical differences between these slopes were observed within subjects (F(1,11) =.47, p =.54, for p(food A) =.9 data; F(1,11) = 3.3, p =.9, for p(food A) =.5

36 23 data; F(1,11) =.17, p =.89, for p(food A) =.1 data). Rats responded similarly to the compound stimulus in each panel of Figure 3; as the probability of food for Stimulus B increased (gray line, triangle markers), the rate of responding in the presence of the compound stimulus also increased (black dashed line, black circle markers). In each panel of Figure 3, the slopes of the best fitting lines for the compound stimuli also appeared to be very similar. The best fitting lines for the compound data are shown on the same axes in Figure 4. Mean Responses per Minute Mean Responses per Minute Mean Responses per Minute Mean Responses per Minute Exp 1a A C p(food A) =.9 B p(food B) Exp 1b C A p(food A) =.5 B p(food B) Exp 1b p(food A) = p(food B) Exp 1b p(food A) =.1 C B A p(food B) A C B Figure 3. Mean response rates in the presence the individual stimuli (A and B) and the compound stimulus (C) (the top panel is from Experiment 1a, and the bottom three panels are from Experiment 1b).

37 24 Stimulus Intercept Slope p(food A) =.9 A B C p(food A) =.5 A B C p(food A) =.25 A B C p(food A) =.1 A B C Table 3. The slope and intercept values for each of the lines of best-fit shown in Figure 3. This comparison in Figure 4 highlights the similarities in the slopes of these lines, though they were produced with different probabilities of food assigned to each individual stimulus that made up the compound. Note that the response rates in the presence of the compound stimuli during Experiment 1a are not shown in Figure 4. These data were omitted to preserve the within-subject comparisons of response rates. Mean Responses per Minute p(food A) =.1 p(food A) =.25 p(food A) =.5 Compound p(food B) Figure 4. The mean response rate across rats in the presence of the compound stimulus for each phase of Experiment 1b. Data are shown with the line of best fit.