Evidence for a Magnitude Effect in Temporal Discounting With Pigeons

Transcription

1 Journal of Experimental Psychology: Animal Behavior Processes 2012, Vol. 38, No. 1, American Psychological Association /12/$12.00 DOI: /a BRIEF REPORT Evidence for a Magnitude Effect in Temporal Discounting With Pigeons Randolph C. Grace University of Canterbury Rebecca J. Sargisson University of Waikato K. Geoffrey White University of Otago A magnitude effect in human intertemporal choice is well established larger rewards or outcomes are discounted over time at a lower rate than are smaller rewards. However, many recent studies have failed to find a corresponding effect in nonhuman animals. Here we report a magnitude effect in temporal discounting for pigeons choices involving a tradeoff between reward delay and amount. Pigeons chose between a small reward (1-s access to food) after a 2-s delay, and a large reward (4.5-s access to food) after a 28-s delay. Across conditions, the delays to the small and large rewards were increased or decreased, respectively. Temporal discounting functions obtained through a value-estimation procedure showed clear evidence of a magnitude effect: The value of the large reward decreased more slowly with increasing delay than the value of the small reward. We linked this result to a nonlinear relationship between choice and the delays associated with the small and large rewards. The nonlinearity was contrary to the generalized matching law but was predicted by the contextual choice model. Our results confirm the existence of a magnitude effect in nonhuman temporal discounting, showing that this adaptation is not unique to humans. Keywords: temporal discounting, magnitude effect, self-control, choice, pigeons Research on intertemporal choice investigates the effect of time in decision making. Many choices involve tradeoffs between the amount of a reward and its delay from the time of choice. It is now well established that the value of a reward is discounted according to a hyperbolic function of delay for both humans and nonhumans (Ainslie, 1974; Green, Fisher, Perlow, & Sherman, 1981; Kirby, 1997: Mazur, 1984; Myerson & Green, 1995; Rodriquez & Logue, 1988; see Green & Myerson, 2004, for review). One of the most pervasive phenomena in intertemporal choice is the so-called magnitude effect: Larger rewards are discounted over time at a lower rate than are smaller rewards. The magnitude effect has been obtained with humans choosing between different kinds of outcomes, including money (Benzion, Rapoport, & Yagil, 1988; Grace & McLean, 2005; Green, Myerson, & McFadden, 1997; Myerson & Green, 1995; Thaler, 1981), health (Chapman, 1996), tipping scenarios (Chapman & Winquist, 1998), employment (Schoenfelder & Hantula, 2003), and personal relationships (Tayler, Arantes, & Grace, 2009). Randolph C. Grace, Department of Psychology, University of Canterbury, Christchurch, New Zealand; Rebecca J. Sargisson, Department of Psychology, University of Waikato, Hamilton, New Zealand; and K. Geoffrey White, Department of Psychology, University of Otago, Dunedin, New Zealand. Correspondence concerning this article should be addressed to Randolph C. Grace, Department of Psychology, University of Canterbury, Christchurch, New Zealand, or K. Geoffrey White, Department of Psychology, University of Otago, Dunedin, New Zealand. randolph.grace@canterbury.ac.nz or geoff.white@otago.ac.nz Although the magnitude effect is a robust finding for humans, research with nonhumans has failed to find similar results. Richards, Mitchell, de Wit, and Seiden (1997, Experiment 3) used a titration procedure to determine rats indifference points for a standard, delayed amount of water that is, the amount of water available immediately that was equally preferred to the standard, delayed, amount. Across conditions, the standard amounts (100 l, 150 l, and 200 l) and delays (0 s, 2 s, 4 s, 8 s, and 16 s) were varied factorially. Richards et al. found that the present value of the standard reward (defined as the point of indifference between immediately available water and the delayed standard amount, divided by the standard amount) decreased with delay and did not differ significantly depending on the standard amount. Green, Myerson, Holt, Slevin, and Estle (2004) used a similar adjustingamount procedure to determine indifference points for rats and pigeons for various amount delay combinations. Green et al. also found that the rate of discounting was not affected by reward amount. Similarly, Freeman, Green, Myerson, and Woolverton (2009) did not find an effect of reward amount on temporal discounting in rhesus monkeys. Additionally, the magnitude effect in rats does not occur for rewards differing in quality as well as amount (Calvert, Green, & Myerson, 2010), in contrast with human studies where there are clear magnitude effects for outcomes differing in quality (Estle, Green, Myerson, & Holt, 2007), and amount (Green & Myerson, 2004). The above procedures involved a series of discrete trial choices from which indifference points are inferred. Grace (1999) used an alternative method to test for a magnitude effect in nonhuman temporal discounting. He used a concurrent chains procedure in 102

2 A MAGNITUDE EFFECT IN NONHUMAN ANIMALS 103 which pigeons responded to two concurrently available keys in a choice phase to produce access to one of two schedules of food reinforcement in the outcome phase. An advantage of the procedure is that preference is sensitive to differences in the outcome schedules. When the keys in the choice phase were red, the amount of food reinforcement was small for both alternatives, whereas when the keys were green the amount of reinforcement was large. The average delay to reinforcement associated with both outcomes was changed across conditions. The pigeons preferred the outcome with the shorter delay to a similar extent regardless of whether the reinforcement amounts were large or small, as measured by the relative response rate to the concurrently available keys. That is, there was no evidence for a magnitude effect. Taken together, the above studies appear to provide strong evidence against the existence of a magnitude effect in nonhumans, because results were similar for different species (rats, pigeons, and monkeys), types of reward (liquid and food), and choice procedures (adjusting delay, adjusting amount, and concurrent chains). However, we were skeptical about this conclusion for two reasons. First, there is considerable evidence of cross-species generality in research on choice: Not only do variables such as delay, amount, rate, and probability of reward have qualitatively similar effects across species, but quantitatively as well. For example, a hyperbolic function provides an excellent description of the relationship between reward value and delay for every species that has been studied including pigeons, rats, monkeys, and humans (see Green & Myerson, 2004, for review). Second, the magnitude effect is adaptive because it increases the likelihood of maximizing the long-term benefit to the organism when faced with a tradeoff between amount and delay of reward. These situations are often described as involving self-control (Rachlin, 2000) because the individual can either choose impulsively by selecting the smaller, more immediate reward, or show self-control by choosing the larger, more delayed reward. It seemed unlikely to us that such a fitness-enhancing adaptation would not have evolved in nonhumans, particularly as no theoretical explanation for the magnitude effect in humans has emerged that would attribute it to an ability that might not be shared by nonhumans (e.g., language). Here we asked whether pigeons would show a magnitude effect in a self-control situation in which they faced a tradeoff between delay and amount of reward. We used a concurrent-chains procedure so that pigeons would repeatedly experience the same delay amount combinations, as compared with titration or adjusting procedures where delay amount combinations change within sessions. The baseline condition was a choice between a smaller, sooner reward (SS; 1-s access to grain delayed by 2 s) and a larger, later reward (LL; 4.5-s access to grain delayed by 28 s). Across conditions, when the choice keys were red the SS delay was constant while the LL delay was increased. When the choice keys were green, the LL delay was constant while the SS delay was increased. By holding one delay constant and varying the other it was possible to trace out temporal discounting functions. Using a simple method for preference scaling described below, we obtained discounting functions for both the small and large amount outcomes over the range of delays from 2 s to 28 s.the critical question was whether the value of the small outcome would decrease more rapidly as a function of increasing delay, indicative of a magnitude effect. Subjects Method Five pigeons (Columba livia), between 8 and 13 years of age were individually housed and had free access to water and grit. The pigeons were weighed daily and fed a mixture of wheat, corn, peas, and pellets if their weight fell below 85% of their free-feeding weights. All pigeons had previous experience with delayed matching-to-sample procedures (but not concurrent chains) and required no training. Apparatus Five Med Associates Inc. chambers, each measuring 295 mm high 295 mm wide 245 mm deep, were used. Three translucent plastic response keys, 21 mm in diameter, were recessed 10 mm into the front panel of each chamber, 210 mm from the grid floor, and 60 mm apart. Only left and right keys were used and could be illuminated red, green, or white. The keys required a force of at least 0.15 N to be operated. A Gerbrands hopper situated behind an aperture 125 mm below the center key provided access to wheat when raised. An infrared beam inside the hopper aperture was broken when the pigeon inserted its head, allowing precise timing of food delivery from the time of head entry. The hopper was illuminated with a 1-W white bulb when raised. A computer linked to the chambers via a Med Associates Inc. interface presented stimuli and recorded the pigeons responses. Procedure Each session lasted for 60 trials or 90 minutes, whichever came first. Sessions were conducted 7 days a week. Each trial began with both side keys lit either red or green, signaling the beginning of the choice phase. Trials with red and green in the choice phase directly alternated throughout the session. Responses to left and right keys during the choice phase were recorded and a single response on one of the keys allowed access to an outcome phase at intervals averaging 30 s according to a variable interval (VI) schedule. Variable interval schedules were arranged concurrently on left and right keys, separately on red and green trials. The variable interval for each key on each trial (red or green) was chosen randomly from one of four lists of 12 intervals constructed from Fleshler and Hoffman s (1962) progression. The schedules on left and right keys were nonindependent, in that when the interval programmed for the schedule on one key timed out and stopped the timer for that key, thus setting up an opportunity for access to the outcome phase, the timer for the schedule on the other key also stopped. Access to the outcome phase on each trial, therefore, was only possible via a response on the key associated with the timed-out interval. As a result, the different events in the outcome phases were experienced equally often in each session. This procedure was first suggested by Stubbs and Pliskoff (1969) and allows preference in the choice phase to be influenced by amount and delay of reward in the outcome phase but not by frequency of reward which is kept equal by the procedure. With VI 30-s schedules, the average time spent in the choice phase on each trial was 15 s.

3 104 GRACE, SARGISSON, AND WHITE If the outcome phase was entered via a left key peck, the left key was then lit white and the right key was dark. The right key was lit white (and the left key dark) if entry was via the right key. In the outcome phase, the first response after a fixed interval (FI) of time had elapsed was followed by food delivery. In five separate conditions, different FI schedules operated on the left and right keys in the outcome phase following red and green choice phases. The FI schedules in the different conditions are listed in Table 1. The five pigeons completed the five conditions in different orders according to a Latin square design in which each condition occupied each of five places in the order across the five pigeons. Each condition was conducted for a minimum of 20 sessions. After completing all five conditions, each pigeon repeated the five conditions in the same order, thus completing at least 200 sessions. When the fixed interval in the outcome phase timed out, different durations of access to wheat followed left and right responses. In both red and green components, the first response on the left key after the FI had elapsed was rewarded with 1-s access to wheat and the first response on the right key with 4.5-s access to wheat. An intertrial interval (ITI) followed food delivery on each trial. The ITI duration was chosen so that the interval from the beginning of the outcome phase and the beginning of the next choice phase was 60 s. Results Table 1 Outcome Phase Schedules and Reinforcer Durations in Red and Green Components Condition Red Green Left Right Left Right 1 FI2s FI2s FI2s FI28s 2 FI2s FI6s FI6s FI28s 3 FI 2 s FI 15 s FI 15 s FI 28 s 4 FI 2 s FI 24 s FI 24 s FI 28 s 5 FI 2 s FI 28 s FI 28 s FI 28 s Reinforcer duration 1 s 4.5 s 1 s 4.5 s Figure 1. The upper panel shows response allocation, measured as the logarithm of the ratio of responses made to the left and right alternatives, as a function of the logarithm of the FI schedule value associated with the 4.5-s outcome (red component; filled squares) or 1-s outcome (green component; unfilled squares). Data are averaged across pigeons, and bars indicate one standard error. The center and lower panels show, respectively, predictions of the generalized matching law (GML; Equation 1) and contextual choice model (CCM; Equation 2) for the average data. Model predictions are shown by dashed lines. See text for more explanation. Responses to the left and right keys were summed over the last five sessions in each condition. Because an analysis of variance (ANOVA) showed that there were no significant effects or interactions related to replication, data were aggregated across replications. The top panel of Figure 1 shows preference for the SS outcome relative to the delayed outcome, measured as the logarithm of the ratio of responses to the left and right keys. The preference measure in Figure 1 was averaged across pigeons separately for responding in the red and green components, and was plotted as a function of the logarithm of the FI schedule value for the changing alternative (log delay). An ANOVA with delay and component as factors found a significant effect of component, F(1, 4) 9.38, p.05, and a significant Component Delay interaction, F(4, 16) 30.36, p.001. The main effect of component indicated that preference for the left alternative was overall greater when the delay associated with the SS outcome was 2 s, and the interaction confirmed that as expected, preference for the SS outcome in the red component increased as the delay for the LL outcome increased, but decreased in the green component as the delay for the SS outcome increased. Our primary goal was to determine if the rates of temporal discounting differed for the large and small rewards. To do this we used a procedure to obtain estimates of relative value for each of the amount delay combinations. We assumed that choice in the initial links was proportional to the relative value of the outcome schedules (the validity of this assumption is examined in the Discussion). The value of the SS outcome with 2-s delay was

4 A MAGNITUDE EFFECT IN NONHUMAN ANIMALS 105 arbitrarily assigned a modulus of 1, and the values of the nine other amount-delay combinations (1 s amount, 6-, 15-, 24-, and 28-s delays; and 4.5-s amount and 2-, 6-, 15-, 24-, and 28-s delays) were estimated as free parameters using nonlinear optimization, such that the variance accounted for in data for individual pigeons was maximized. The estimated values provided a good description of results, accounting for an average of 95% of the variance in data for individual pigeons. The left panel of Figure 2 shows the average values for the large (4.5 s; red component) and small (1 s; green component) rewards, as a function of delay (FI schedule). The right panel shows normalized data, obtained by dividing the value for delays greater than 2 s by the value for the reward with 2-s delay. For all delays, the relative value of the large reward was greater than the corresponding relative value of the small reward. An ANOVA with delay and amount as factors confirmed a significant Amount Delay interaction, F(3, 12) 4.25, p.05, and a main effect of amount, F(1, 4) 7.25, p.05. Post hoc tests (Fisher least significant difference) showed that the relative value of the large reward was significantly greater (p.05) than the small reward for delays of 6, 15, and 24 s, whereas the difference for 28 s was not significant (p.11). This confirms that a magnitude effect was obtained in the present data, as the value of the large reward decreased more slowly with delay than the value of the small reward. Finally we asked whether the data were compatible with models for behavioral choice (see Grace & Hucks, in press, for review). As applied to the current procedure, the generalized matching law (GML) predicts that response allocation, measured as the logarithm of the response ratio, is determined according to the following equation (Baum, 1974; Baum & Rachlin, 1969; Logue, Rodriguez, Pena Correal & Mauro, 1984): log B L a B D log 1/D R a R 1/D M log A L log b, (1) L A R where B indicates responses, and D and A indicate delay and amount of reward, subscripted for the left and right alternatives. There are three parameters: Sensitivity to delay (a D ) and amount of reward (a M ), and response bias (log b). Thus according to the GML, response allocation is determined additively by the log delay and amount ratios, each weighted by a sensitivity parameter, and response bias. Grace (1994) proposed an extension of the generalized matching law, called the contextual choice model (CCM) because it assumed that delay and amount sensitivities were affected by the temporal context of choice the average times spent in the choice and outcome phases of the concurrent chains procedure. The corresponding equation for CCM is log B L T o B R T D log 1/D R a c a 1/D M log A L L A R logb, (2) where T o and T c are the average times spent, per trial, in the outcome and choice phases of the procedure. According to CCM, sensitivity to delay and amount depend on the relative time spent in the outcome and choice phases, with sensitivities increasing as the duration of the outcome phases increases relative to the duration of the choice phase. Figure 1 shows fits of the GML (center panel) and CCM (bottom panel) to the average data. Overall, CCM provided a better fit, accounting for 91% of the variance (root mean square error [RMSE] 0.15), as compared with the GML which accounted for 78% (RMSE 0.24). CCM predicts a nonlinear relationship such that choice becomes more extreme at longer overall durations, which is the source of the magnitude effect in the present data. Consistent with CCM, choice for the larger reward was greater when the delays were 28 s 28 s than when they were 2 s 2 s. Indeed, it has been shown earlier that sensitivity to amount increases with increasing delay in the outcome phase (White & Pipe, 1987). Thus, CCM is able to predict the magnitude effect for pigeons choice in concurrent chains as a consequence of an interaction between delay and amount, such that sensitivity depends on the overall delay duration. Discussion Contrary to previous studies, we found clear evidence of a magnitude effect with pigeons: The relative value of a large reward Figure 2. The left panel shows values estimated for the 4.5-s outcome (red component) and 1-s outcome (green component). The value of the 1-s outcome, 2-s delay was set to a modulus of 1. The right panel shows normalized values obtained by dividing the value for delays 2 s by the value of the outcome at 2 s ( present value ). Data are averaged across pigeons, and bars indicate one standard error.

5 106 GRACE, SARGISSON, AND WHITE (4.5-s access to food) decreased more slowly as its delay increased from 2 s to 28 s than the corresponding value of a small reward (1-s access to food). Values were estimated from response ratios when pigeons pecked two concurrently available keys to produce the large and small rewards. The magnitude effect occurred because preference (measured as the log response ratio) varied with delay and amount ratios, but the relationship was nonlinear: Sensitivity to relative delay and amount increased as overall delay increased. This nonlinearity was contrary to the generalized matching law (Baum, 1974) but was predicted by the contextual choice model (Grace, 1994). In demonstrating the magnitude effect in pigeons, we are reassured that, as for many other behavioral phenomena, nonhuman animals exhibit an adaptive response which can maximize long-term benefit to the organism in a choice situation involving a trade-off between amount and delay of reward. The assumption that initial-link responding in concurrent chains can be used to measure the value of the terminal-link schedules has been controversial (for review see Grace & Savastano, 2000). The major problem was Fantino s (1969) result that initial-link duration affected preference independently of the terminal-link schedules. These difficulties prompted Mazur (1984) to introduce the adjusting-delay procedure, in which preference is registered with a single response. Mazur s work was highly influential, as similar discrete-trial procedures that yield indifference points and the hyperbolic-decay model that he proposed have been widely used in subsequent research on discounting with humans (Green & Myerson, 2004). The effect of initial-link duration means that choice in concurrent chains cannot provide a measure of absolute terminal-link value. However, the magnitude effect is a change in relative value specifically, that the value of a large reward decreases more slowly relative to its immediate value than does the relative value of a small reward. There is no reason to expect that our results would have been fundamentally different if, for example, a longer initial-link schedule had been used. Simulations using fits of CCM to the present data show that a similar magnitude effect would have been obtained using a wide range of initial-link durations. Because preference for the larger over smaller reward was more extreme when delays to both were 28 s than 2 s, if response allocation is interpreted as measuring relative value, then the value of the smaller reward must have decreased more as delay increased from2sto28s.butnote that if value is defined in terms of CCM, that is, reward amounts and delays and the associated sensitivity parameters, there would be no magnitude effect because CCM predicts the nonlinearity in Figure 1 with the same sensitivity parameters applying to both rewards. Thus, our conclusion depends on defining value directly in terms of response allocation. Although research on temporal discounting with titration procedures and on choice with free-operant procedures like concurrent chains has evolved somewhat independently, the value estimation method used here offers a way to integrate research in these domains. It yields estimates of relative value from responding in concurrent chains that could be compared, in principle, with results from titration procedures. Although concurrent chains has not featured prominently in discounting research, it provides an alternative method for assessing the relationship between delay and value that is arguably no less valid than the adjusting-delay procedure. Grace (1996) showed that pairs of fixed and variable-delay indifference points obtained for individual pigeons from the adjusting-delay procedure were equally preferred as terminal links in concurrent chains, suggesting that the determiners of schedule value were the same in the two procedures. Their similarity is further underscored by the fact that models which assume a common relationship between delay and value are able to describe results from both procedures (Grace, 1996; Mazur, 2001). Finally, it is worth noting that indifference points in the adjusting-delay procedure are also affected by factors other than reward value, including trial order (Robles, Vargas, & Bejarano, 2009) and contrast effects (Dai, Grace, & Kemp, 2009). Thus, both procedures should be viewed as providing measures of relative, not absolute value. Why were our results different from previous research with nonhumans? Richards et al. (1997); Green et al. (2004); Calvert et al. (2010) and others used an adjusting-amount or titration procedure to obtain indifference points for delayed rewards in rats and pigeons. On each trial the animals make a single choice between an outcome with a fixed amount and delay to reward ( standard ), and an outcome with an immediate reward but for which the amount is gradually adjusted over trials. If animals choose the adjusting alternative, then its amount is decreased, whereas if they choose the standard alternative, the adjusting amount is increased. In these studies, discounting functions obtained after substantial training with a standard amount-delay combination did not change with the size of the standard amount. For example, if rats indifference point for a reward of 5 pellets with an 8-s delay was 2.5 pellets, then the indifference point for a 20-pellet reward with an 8-s delay would be 10 pellets. By contrast, if a magnitude effect had been present then the indifference point for the 20-pellet reward would be larger, say 15 pellets. Thus a magnitude effect would be adaptive because it would increase the overall amount of food the animal earned during the session. The absence of a magnitude effect in previous studies might reflect the relative insensitivity of choice to the size of the standard delayed amount which varies across conditions, although it is possible that the animals discriminate the contingency between their choice and changes in the adjusting reward. In the present experiment, the standard amounts were varied within sessions (across red and green choice trials). Accordingly, the contrast between within-session self control choices (SS verses LL) was emphasized. An additional factor that might contribute to the absence of a magnitude effect in titration or adjusting procedures is related to the time between trials. Unlike for humans, who when given appropriate instructions can specify an indifference point directly (e.g., Chapman, 1996), indifference points for animals must be inferred from a series of choices. Typically, these choices oscillate between runs for the standard alternative and runs for the adjusting alternative, with the indifference point estimated as the average (Mazur, 1987). Previous studies have shown that rats and pigeons are more sensitive to reward amounts in discrete-trial choice procedures when the response opportunities are more closely spaced in time (e.g., Mazur, 2010; Williams, 1992). Recently, Mazur and Biondi (2011) showed that the indifference point obtained from an adjusting-delay procedure increased with increasing time between trials. It is possible that evidence for a magnitude

6 A MAGNITUDE EFFECT IN NONHUMAN ANIMALS 107 effect would be obtained if a short ITI was used in the adjustingamount procedure. Our results show that the magnitude effect is not an adaptation unique to humans. Although several explanations for the magnitude effect in humans have been proposed, none has been generally accepted. These include separate mental accounts for large and small rewards (Thaler, 1981), specific characteristics of the utility function for money ( increasing proportional sensitivity, IPS; Loewenstein & Prelec, 1992), and the reward differential hypothesis (i.e., that choices are more likely to be made on the basis of reward differences rather than ratios when the magnitudes are large; Grace & McLean, 2005). However, these accounts have not withstood empirical scrutiny (see Grace & McLean, 2005; Green et al., 1997). Recently, Killeen (2009) proposed a general model for discounting which assumes that the effects of utility and time are subtractive. Killeen s theory is able to predict the magnitude effect even though the scaling parameter for time is independent of reward amount. It is interesting to note that in this respect, Killeen s account is similar to CCM, which assumes that delay and amount sensitivities are constant but their effects are modulated by overall duration. Developing an integrated account of the magnitude effect with potential generality for both humans and nonhumans is an important goal for future research. References Ainslie, G. W. (1974). Impulse control in pigeons. Journal of the Experimental Analysis of Behavior, 21, doi: /jeab Baum, W. M., & Rachlin, H. C. (1969). Choice as time allocation. Journal of the Experimental Analysis of Behavior, 12, doi: / jeab Baum, W. M. (1974). On two types of deviation from the matching law: Bias and undermatching. Journal of the Experimental Analysis of Behavior, 22, doi: /jeab Benzion, U., Rapoport, A., & Yagil, J. (1988). Discount rates inferred from decisions: An experimental study. Management Science, 35, doi: /mnsc Calvert, A. L., Green, L., & Myerson, J. (2010). Delay discounting of qualitatively different reinforcers in rats. Journal of the Experimental Analysis of Behavior, 93, doi: /jeab Chapman, G. B., & Winquist, J. R. (1998). The magnitude effect: Temporal discount rates and restaurant tips. Psychonomic Bulletin & Review, 5, doi: /bf Chapman, G. B. (1996). Temporal discounting and utility for health and money. Journal of Experimental Psychology: Learning, Memory, and Cognition, 22, doi: / Dai, Z., Grace, R. C., & Kemp, S. (2009). Reward contrast in delay and probability discounting. Learning & Behavior, 37, doi: /LB Estle, S. J., Green, L., Myerson, J., & Holt, D. D. (2007). Discounting of monetary and directly consumable rewards. Psychological Science, 18, doi: /j x Fantino, E. (1969). Choice and rate of reinforcement. Journal of the Experimental Analysis of Behavior, 12, doi: / jeab Fleshler, M., & Hoffman, H. S. (1962). A progression for generating variable-interval schedules. Journal of the Experimental Analysis of Behavior, 5, doi: /jeab Freeman, K., Green, L., Myerson, J., & Woolverton, W. (2009). Delay discounting of saccharin in rhesus monkeys. Behavioural Processes, 82, doi: /j.beproc Grace, R. C., & Hucks, A. D. (in press). The allocation of operant behavior. In K. A. Lattal, T. D. Hackenberg, & G. J. Madden (Eds.), Handbook of behavior analysis. Washington, DC: American Psychological Association. Grace, R. C., & McLean, A. P. (2005). Integrated versus segregated accounting and the magnitude effect in temporal discounting. Psychonomic Bulletin & Review, 12, doi: /bf Grace, R. C., & Savastano, H. I. (2000). Temporal context and conditioned reinforcement value. Journal of Experimental Psychology: General, 129, doi: / Grace, R. C. (1994). A contextual model of concurrent-chains choice. Journal of the Experimental Analysis of Behavior, 61, doi: /jeab Grace, R. C. (1996). Choice between fixed and variable delays to reinforcement in the adjusting-delay procedure and concurrent chains. Journal of Experimental Psychology: Animal Behavior Processes, 22, doi: / Grace, R. C. (1999). The matching law and amount-dependent exponential discounting as accounts of self-control choice. Journal of the Experimental Analysis of Behavior, 71, doi: /jeab Green, L., Fisher, E. B. Jr., Perlow, S., & Sherman, L. (1981). Preference reversal and self-control: Choice as a function of reward amount and delay. Behaviour Analysis Letters, 1, Green, L., Myerson, J., & McFadden, E. (1997). Rate of temporal discounting decreases with amount of reward. Memory & Cognition, 25, doi: /bf Green, L., & Myerson, J. (2004). A discounting framework for choice with delayed and probabilistic rewards. Psychological Bulletin, 130, doi: / Green, L., Myerson, L., Holt, D. D., Slevin, J. R., & Estle, S. J. (2004). Discounting of delayed food rewards in pigeons and rats: Is there a magnitude effect? Journal of the Experimental Analysis of Behavior, 81, doi: /jeab Killeen, P. R. (2009). An additive-utility model of delay discounting. Psychological Review, 116, doi: /a Kirby, K. N. (1997). Bidding on the future: Evidence against normative discounting of delayed rewards. Journal of Experimental Psychology: General, 126, doi: / Loewenstein, G., & Prelec, D. (1992). Anomalies in intertemporal choice: Evidence and an interpretation. In G. Loewenstein & J. Elster (Eds.), Choice over time (pp ). New York, NY: Sage. Logue, A. W., Rodriguez, M. L., Pena Correal, T. E., & Mauro, B. (1984). Choice in a self-control paradigm: Quantification of experienced-based differences. Journal of the Experimental Analysis of Behavior, 41, doi: /jeab Mazur, J. E., & Biondi, D. R. (2011). Effects of time between trials on rats and pigeons choices with probabilistic delayed reinforcers. Journal of the Experimental Analysis of Behavior, 95, doi: / jeab Mazur, J. E. (1984). Tests for an equivalence rule for fixed and variable reinforcer delays. Journal of Experimental Psychology: Animal Behavior Processes, 10, doi: / Mazur, J. E. (1987). An adjusting procedure for studying delayed reinforcement. In M. L. Commons, J. E. Mazur, J. A. Nevin, & H. Rachlin (Eds.), Quantitative analysis of behavior: Vol. 5. The effects of delay and of intervening events on reinforcement value (pp ). Hillsdale, NJ: Erlbaum. Mazur, J. E. (2001). Hyperbolic value addition and general models of animal choice. Psychological Review, 108, doi: / x Mazur, J. E. (2010). Distributed versus exclusive preference in discretetrial choice. Journal of Experimental Psychology: Animal Behavior Processes, 36, doi: /a Myerson, J., & Green, L. (1995). Discounting of delayed rewards: Models

7 108 GRACE, SARGISSON, AND WHITE of individual choice. Journal of the Experimental Analysis of Behavior, 64, doi: /jeab Rachlin, H. C. (2000). The science of self-control. Cambridge, MA: Harvard University Press. Richards, J. B., Mitchell, S. H., de Wit, H., & Seiden, L. S. (1997). Determination of discount functions in rats with an adjusting-amount procedure. Journal of the Experimental Analysis of Behavior, 67, doi: /jeab Robles, E., Vargas, P. A., & Bejarano, R. (2009). Within-subject differences in degree of delay discounting as a function of order of presentation of hypothetical cash rewards. Behavioural Processes, 81, doi: /j.beproc Rodriquez, M. L., & Logue, A. W. (1988). Adjusting delay to reinforcement: Comparing choice in pigeons and humans. Journal of Experimental Psychology: Animal Behavior Processes, 14, doi: / Schoenfelder, T., & Hantula, D. A. (2003). A job with a future? Delay discounting, magnitude effects and domain independence of utility for career decisions. Journal of Vocational Behavior, 62, doi: /S (02) Online First Publication Stubbs, D. A., & Pliskoff, S. S. (1969). Concurrent responding with fixed relative rates of reinforcement. Journal of the Experimental Analysis of Behavior, 12, doi: /jeab Tayler, S., Arantes, J., & Grace, R. C. (2009). Temporal discounting for monetary and close relationship outcomes. Personal Relationships, 16, doi: /j x Thaler, R. (1981). Some empirical evidence on dynamic inconsistency. Economics Letters, 8, doi: / (81) White, K. G., & Pipe, M. E. (1987). Sensitivity to reinforcer duration in a self-control procedure. Journal of the Experimental Analysis of Behavior, 48, doi: /jeab Williams, B. A. (1992). Dissociation of theories of choice by temporal spacing of choice opportunities. Journal of Experimental Psychology: Animal Behavior Processes, 18, doi: / APA-published journal articles are now available Online First in the PsycARTICLES database. Electronic versions of journal articles will be accessible prior to the print publication, expediting access to the latest peer-reviewed research. All PsycARTICLES institutional customers, individual APA PsycNET database package subscribers, and individual journal subscribers may now search these records as an added benefit. Online First Publication (OFP) records can be released within as little as 30 days of acceptance and transfer into production, and are marked to indicate the posting status, allowing researchers to quickly and easily discover the latest literature. OFP articles will be the version of record; the articles have gone through the full production cycle except for assignment to an issue and pagination. After a journal issue s print publication, OFP records will be replaced with the final published article to reflect the final status and bibliographic information. Received March 28, 2011 Revision received July 11, 2011 Accepted August 4, 2011