Discounting of Monetary and Directly Consumable Rewards Sara J. Estle, Leonard Green, Joel Myerson, and Daniel D. Holt

PSYCHOLOGICAL SCIENCE Research Article Discounting of Monetary and Directly Consumable Rewards Sara J. Estle, Leonard Green, Joel Myerson, and Daniel D. Holt Washington University ABSTRACT We compared temporal and probability discounting of a nonconsumable reward (money) and three directly consumable rewards (candy, soda, and beer). When rewards were delayed, monetary rewards were discounted less steeply than directly consumable rewards, all three of which were discounted at equivalent rates. When rewards were probabilistic, however, there was no difference between the discounting of monetary and directly consumable rewards. It has been reported that substance abusers discount delayed drug rewards more steeply than delayed money, but this difference may reflect special characteristics of drugs or drug abusers, or it may reflect a general property of consumable rewards. The present findings suggest that abused substances (like beer) share the properties of other directly consumable rewards, whereas delayed monetary rewards are special because they are fungible, generalized (conditioned) reinforcers. Discounting plays an important role in decision making, especially in situations that involve self-control, such as when drug abuse, gambling, and other impulsive behaviors are involved. Temporal discounting refers to the decrease in the subjective value of an outcome as the delay until its occurrence increases. Studies have shown that drug abusers discount delayed rewards more steeply than nonabusers do. Moreover, using hypothetical delayed rewards, researchers have shown that drug abusers discount their drug of abuse more steeply than they discount money (for a review, see Bickel & Marsh, 2001). This finding could reflect either special characteristics of drugs or drug abusers or a general property of consumable rewards (Odum & Rainaud, 2003). That is, it is possible that there is some special characteristic of abused substances or that there is some special characteristic of drug abusers that selectively affects their Sara Estle and Daniel Holt are now at the University of Wisconsin- Eau Claire. Address correspondence to Leonard Green, Department of Psychology, Campus Box 1125, Washington University, St. Louis, MO 63130, e-mail: lgreen@wustl.edu. choices involving drugs and leads to steeper discounting. Alternatively, delayed consumable rewards in general may be evaluated differently than delayed monetary rewards, which are exchangeable for consumable rewards, but are not themselves directly consumable. Odum and Rainaud (2003) recently argued that directly consumable, primary reinforcers show steeper temporal discounting than conditioned reinforcers such as money. Participants in their study, screened for drinking, gambling, and eating problems, made choices between immediate and delayed amounts of the same commodity, either money or their favorite food or alcoholic beverage. Examination of participants temporal-discounting functions revealed that delayed food rewards and delayed alcohol rewards were discounted equivalently, but both were discounted significantly more steeply than delayed monetary rewards. The present study examined whether there are differences between the discounting of probabilistic consumable and monetary rewards similar to the differences in discounting observed with delayed rewards. Previous research has shown that in probability discounting, the subjective value of a monetary reward decreases as the odds against its receipt increase (e.g., Rachlin, Raineri, & Cross, 1991), and that the discounting of both delayed and probabilistic rewards is well described by a hyperboloid function of the form Y ¼ A=ð1 þ bxþ s ; ð1þ where Y represents the subjective value of a reward of amount A, the parameter b governs the rate of discounting, X is delay until or the odds against receipt of the reward, and the exponent s reflects the scaling of amount and either delay or odds (Green & Myerson, 2004; Myerson & Green, 1995). When s equals 1.0, Equation 1 reduces to a simple hyperbola (Mazur, 1987). Although hyperboloid discounting functions describe choice involving both probabilistic and delayed outcomes (for a review, see Green & Myerson, 2004), temporal and probability discounting are affected by the amount of reward in opposite ways: Larger delayed rewards are discounted less steeply than smaller delayed rewards, whereas smaller probabilistic rewards are 58 Copyright r 2007 Association for Psychological Science Volume 18 Number 1

S.J. Estle et al. discounted less steeply than larger probabilistic rewards (Du, Green, & Myerson, 2002; Green, Myerson, & Ostaszewski, 1999; Myerson, Green, Hanson, Holt, & Estle, 2003). The similarity in the form of temporal- and probability-discounting functions suggests that there may be similarities in the underlying decision-making processes, but the fact that the effects are of opposite magnitude suggests that there are important differences as well. The present study is the first to compare temporal and probability discounting of an abused substance (alcohol) with discounting of nonabused but immediately consumable goods (soda and candy). It is also the first to examine probability discounting of consumable rewards, as well as the first to compare the effects of amount on the discounting of monetary rewards and immediately consumable rewards. At issue is whether steeper discounting of consumable than monetary rewards is observed with probabilistic outcomes, as it is with delayed outcomes, and whether immediately consumable rewards show magnitude effects similar to those observed with monetary rewards. METHOD Forty-seven Washington University undergraduates (23 males) were recruited through the Department of Psychology s Human Subjects Pool and received course credit for their participation. Participants were tested individually in a small room containing a computer with a touch-screen monitor (MicroTouch 3M). They were informed that the purpose of the study was to examine their preferences for hypothetical amounts of money, candy, soda, and beer rewards and then were administered temporal- and probability-discounting tasks. For each participant, the order in which the types of reward were presented was determined randomly. Within each reward type, which task (temporal or probability discounting) came first was randomly determined, and within each task, which reward amount (large or small) came first also was randomly determined. Finally, within each amount condition of a task, the order in which the delays or probabilities were presented was randomly determined. Participants were told that on each trial, two amounts of a hypothetical reward (money, beer, candy, or soda) would appear on the screen. For the temporal-discounting task, they were instructed that one amount could be received right now, whereas the other amount could be received after some specified period of time. For the probability-discounting task, they were instructed that one amount could be received for sure, whereas the other amount could be received with some specified probability. Participants also were instructed that the amount of the immediate or certain reward would change after each of their choices, but that the amount of the delayed or probabilistic reward would remain the same for a group of choices. Participants were informed that there were no correct or incorrect choices, and that they were to indicate their choice by touching the option they preferred. Following six practice trials, participants had the opportunity to ask questions, and then the experiment began. For each type of reward, two amounts were studied: 40 units and 100 units. For money, the units were dollars; for candy, the units were bars; for beer and soda, the units were cans. 1 For each delayed amount in the temporal-discounting task, participants made six choices at each of five delays: 1 week, 1 month, 6 months, 1 year, and 3 years. Within each group of six choices, the amount of the immediate reward was adjusted using a staircase procedure that converged rapidly on the amount of immediate reward equal in subjective value to the delayed reward (for a detailed description of this procedure, see Du et al., 2002). For each probabilistic amount in the probability-discounting task, participants made six choices at each of five probabilities: 95%, 90%, 50%, 25%, and 10% chance. Within each group of six choices, the amount of the certain reward was adjusted using a staircase procedure that converged rapidly on the amount of certain reward equal in subjective value to the probabilistic reward. For both temporal and probability discounting, we assessed the degree of discounting by plotting the subjective value of each delayed or probabilistic reward at each amount as a function of the delay until receiving the reward or the probability of receiving the reward. Equation 1 was then fit to the data for each reward type and amount to determine the parameters of the best-fitting temporal- and probability-discounting functions. For purposes of statistical analysis, the area under the subjective values provided a measure of how steeply the reward was discounted. Area measures, normalized based on the maximum possible subjective values and the maximum delay or odds studied, can range between 0.0 and 1.0, with smaller areas indicating steeper discounting (Myerson, Green, & Warusawitharana, 2001). Significance levels for statistical analyses were set at a 5.05. RESULTS The temporal- and probability-discounting data are presented in Figure 1. In all four panels, the curves represent Equation 1, fit to the group median data using a nonlinear, least squares algorithm. For the temporal-discounting data (left panels of Fig. 1), this equation accounted for more than 85% of the variance for both 1 It might be argued that the delay to receipt of some amount of a directly consumable good underestimates the delay to reinforcement. Consider, for example, the reward of 40 candy bars, used in the present experiment. One would be unlikely to consume them all immediately. A similar argument might be made with respect to money, however, in that many goods purchased with money (e.g., two compact discs that might be purchased for $40) also would be consumed over an extended period of time. Thus, the delay to receipt of some amount of money also may underestimate the delay to reinforcement. Although the issue of extended consumption is an important one, and one that has not received sufficient attention from researchers (but see Raineri & Rachlin, 1993), monetary and directly consumable rewards are not qualitatively different in this regard. Volume 18 Number 1 59

Discounting of Rewards Fig. 1. Subjective value of the delayed (left panels) and probabilistic (right panels) rewards. Group median subjective value, calculated as a proportion of the amount of reward (40 or 100 units), is plotted as a function of the time until receipt of the delayed reward and as a function of the odds against receipt of the probabilistic reward. Results for smaller amounts of reward are shown in the upper panels; results for larger amounts are shown in the lower panels. The curved lines represent the hyperboloid discounting function (Equation 1) fit to the data for each type of reward (money, beer, candy, and soda). the small and larger amounts of money, beer, candy, and soda. When Equation 1 was fit to the data from each participant, the median R 2 was greater than 79% for each amount and type of reward. Area measures of temporal discounting were calculated for each participant and entered into a 4 (type of reward: money, beer, candy, or soda) 2 (amount of reward: 40 or 100 units) repeated measures analysis of variance (ANOVA). The ANOVA revealed a significant effect of type of reward, F(3, 138) 5 16.61, Zp 2 5.265, p <.001, reflecting the fact that money was discounted less steeply than beer, candy, and soda, and a significant effect of amount of reward, F(1, 46) 5 11.58, Z 2 p 5.201, p rep >.97, reflecting the fact that smaller reward amounts were discounted more steeply than corresponding larger amounts (see Table 1). There was no significant type-byamount interaction, F(3, 138) 5 1.961, p >.10. A follow-up ANOVA testing for differences among the three directly consumable rewards revealed no effect of type, F(2, 92) 5 2.03, p >.10; a marginally significant effect of amount, F(1, 46) 5 3.588, p rep 5.86; and no interaction, F(2, 92) < 1. For the probability-discounting data, Equation 1 accounted for more than 98% of the variance for both the small and the larger amounts of money, beer, candy, and soda. The results at the individual level also were well described by Equation 1; for each of the amounts and types of reward, the median R 2 was greater than 93%. Again, area measures were calculated for each participant for both amounts at each of the four reward types (money, beer, candy, and soda) and entered into a 4 (type) 2 (amount) repeated measures ANOVA. In contrast to the results for delayed rewards, there was no effect of the type of reward, F(3, 138) 5 2.168, p >.09. As with the delayed rewards, there was a significant magnitude effect, F(1, 46) 5 16.02, p rep >.99, Zp 2 5.258. As Table 1 shows, however, this effect was opposite in direction to the magnitude effect observed 60 Volume 18 Number 1

S.J. Estle et al. TABLE 1 Mean Area Under the Subjective Values of Delayed and Probabilistic Rewards Delayed amount Reward Small Large Small Large Money.547.640.332.286 (.038) (.036) (.021) (.019) Beer.361.387.328.290 (.033) (.036) (.028) (.031) Candy.412.450.376.357 (.039) (.042) (.032) (.035) Soda.409.433.357.309 (.038) (.039) (.030) (.031) Note. Standard errors are given in parentheses. Probabilistic amount these data reveals very small differences between the different commodities and little interpretable pattern. Consistent with this observation, the probability of replicating the pattern of observed differences among the consumable commodities was relatively low, p rep 5.170, suggesting that these differences are unreliable. The patterns of differences observed with probabilistic rewards also were relatively unreliable. The six directional t tests in which the discounting of small and large amounts of probabilistic monetary reward was compared with the discounting of the corresponding amounts of each of the directly consumable rewards yielded a p rep of.131 for the observed pattern, and the six tests comparing the discounting of directly consumable probabilistic rewards yielded a p rep of.249. with delayed rewards: The smaller probabilistic amounts were discounted less steeply than the larger probabilistic amounts. In addition to providing a measure of the probability with which ANOVA outcomes will be replicated, the p rep statistic (Killeen, 2005a, 2005b) provides the basis for an alternative approach to assessing the robustness of specific patterns of results involving multiple comparisons. That is, rather than assessing the probability with which one can replicate the finding that the differences among multiple conditions are not all equal, one can use p rep to assess the probability that they will not be equal in a specific way. As Killeen (2005a) indicated, if one assumes independence, then the probability that a specific pattern of differences will be replicated (i.e., all effects will have the same signs as in the original experiment) is equal to the product of the probabilities of replicating each of the differences that make up that pattern. In the present case, we first examined whether the pattern observed when delayed monetary rewards were compared with other rewards would be highly replicable (i.e., whether candy, soda, and beer would each show steeper temporal discounting than monetary rewards, both when the amounts involved were small and when they were large). The probability of replicating this pattern is given by the product of the p rep values for the six directional t tests in which the discounting of small and large amounts of delayed monetary reward was compared with the discounting of the corresponding amounts of each of the directly consumable rewards. The present data indicated that the probability of replicating this pattern is very high (p rep 5.977), and thus this study provides strong evidence that delayed monetary rewards are discounted less steeply than delayed directly consumable rewards. We also examined whether or not the pattern of differences observed in the temporal discounting of the three directly consumable rewards (see Table 1) would be likely to be replicated, as inferred from the product of the p rep values for the six directional t tests (i.e., beer vs. candy, beer vs. soda, and soda vs. candy, compared at small and large amounts). Inspection of DISCUSSION Delayed monetary rewards were discounted less steeply than delayed directly consumable rewards (beer, candy, and soda), all of which were discounted at equivalent rates. This pattern was observed for both smaller and larger amounts of delayed reward. When rewards were probabilistic, however, there was no difference between the discounting of monetary and directly consumable rewards. In addition, steeper discounting of smaller rewards than larger rewards was observed when rewards were delayed, whereas steeper discounting of larger rewards than smaller rewards was observed when rewards were probabilistic. The observed pattern of opposite magnitude effects for temporal and probability discounting is well established for monetary rewards (for a review, see Green & Myerson, 2004). Nonmonetary rewards have received much less attention in the literature. Although two studies have reported a tendency toward steeper discounting of smaller than larger delayed nonmonetary rewards (Petry, 2001; Raineri & Rachlin, 1993), the present study is the first to show significant magnitude effects with directly consumable delayed rewards. The present study is also the first to demonstrate that, like probabilistic monetary rewards, probabilistic nonmonetary rewards show magnitude effects in which larger amounts are discounted significantly more steeply than smaller amounts. Is there something special about abused substances or substance abusers that leads to steeper temporal discounting of abused substances than monetary rewards, or, as Odum and Rainaud (2003) suggested, are all directly consumable rewards discounted more steeply than money? The present results provide no evidence that abused substances are special, at least for individuals who are not substance abusers. Delayed candy, soda, and beer rewards were all discounted at equivalent rates, as were probabilistic candy, soda, and beer rewards. Interestingly, it is delayed monetary rewards, rather than abused substances, that may turn out to be special. Delayed monetary rewards were discounted less steeply than the directly consumable rewards, Volume 18 Number 1 61

Discounting of Rewards but when rewards were probabilistic, all were discounted at equivalent rates regardless of type. This finding raises the question of why, if monetary rewards are special with respect to the degree of temporal discounting, they are not special with respect to probability discounting. There are, in fact, other differences between temporal and probability discounting of monetary rewards. Most notable are the opposite effects that amount of reward has on the degree of temporal and probability discounting, but it also has been shown that temporal discounting is sensitive to rates of inflation, whereas probability discounting is not (Ostaszewski, Green, & Myerson, 1998), and that the degrees of temporal and probability discounting vary independently across cultures (Du et al., 2002). No overarching account of these differential effects has been proposed, but they all suggest that temporal and probability discounting do not involve the same decision-making processes despite the similar mathematical forms of the discounting functions (Green & Myerson, 2004). Nevertheless, the apparently special status of delayed monetary rewards seems to call for its own explanation. The explanation cannot simply be that money is a conditioned reinforcer whereas directly consumable rewards are primary reinforcers, as Odum and Rainaud (2003) suggested. If money s status as a conditioned reinforcer were what makes it special, then one would expect money to be discounted differently than directly consumable rewards when monetary rewards are probabilistic, as well as when they are delayed. A related possibility, and one that provides a better account, is that when it comes to temporal discounting, money is special because it is fungible (in economic terms). That is, money is a generalized (conditioned) reinforcer (in psychological terms), exchangeable for other primary and secondary reinforcers (Catania, 1998; Skinner, 1953). Therefore, people may discount a delayed monetary reward less steeply than a specific, nongeneralized reward because, unlike specific rewards, money retains its utility, despite the inconstancy of desire. When a delayed monetary reward is received, it may be exchanged for whatever is currently needed or desired. In contrast, the utility of a specific, nongeneralized reward may fluctuate. Of course, its utility may fluctuate either up or down. People are risk averse, however, and thus are likely to consider such a commodity to be worth less than one (i.e., money) whose utility does not fluctuate, even when both have the same mean utility. The present results are consistent with those of Odum and Rainaud (2003) in demonstrating that alcohol rewards are not special with respect to the rate at which they are discounted, at least for individuals who are not substance abusers. It remains possible, of course, that other abused substances could be discounted differently than other directly consumable rewards, or that substance abusers might discount their substances of abuse differently than other directly consumable rewards. Substance abusers are known both to discount delayed monetary rewards more steeply than do nonabusers and to discount abused substances more steeply than they do money (e.g., Bickel, Odum, & Madden, 1999; Coffey, Gudleski, Saladin, & Brady, 2003; Madden, Petry, Badger, & Bickel, 1997; Petry, 2001). To date, however, no studies have compared the rates at which substance abusers discount probabilistic or delayed abused substances with the rates at which they discount other directly consumable rewards. The present findings demonstrate the need for such studies, the results from which will help explicate the relative contributions of the special characteristics of substance abusers and the special characteristics of abused substances as reinforcers, as well as the interactions between these characteristics, to addictive behaviors. Understanding these contributions may shed new light on the causes of substance abuse, thereby leading to more effective prevention and treatment. Acknowledgments This research was supported by National Institutes of Health Grant MH 55308. We thank Tessa Mazzocco for her assistance in conducting this study, Karen Norberg for reminding us of the inconstancy of desire, and Peter Killeen, who (as could have been predicted) was (once again) most helpful in providing us with stimulating suggestions and helpful feedback on our analyses. REFERENCES Bickel, W.K., & Marsh, L.A. (2001). Toward a behavioral economic understanding of drug dependence: Delay discounting processes. Addiction, 96, 73 86. Bickel, W.K., Odum, A.L., & Madden, G.J. (1999). Impulsivity and cigarette smoking: Delay discounting in current, never, and exsmokers. Psychopharmacology, 146, 447 454. Catania, A.C. (1998). Learning (4th ed.). Upper Saddle River, NJ: Prentice Hall. Coffey, S.F., Gudleski, G.D., Saladin, M.E., & Brady, K.T. (2003). Impulsivity and rapid discounting of delayed hypothetical rewards in cocaine-dependent individuals. Experimental and Clinical Psychopharmacology, 11, 18 25. Du, W., Green, L., & Myerson, J. (2002). Cross-cultural comparisons of discounting delayed and probabilistic rewards. Psychological Record, 52, 479 492. Green, L., & Myerson, J. (2004). A discounting framework for choice with delayed and probabilistic rewards. Psychological Bulletin, 130, 769 792. Green, L., Myerson, J., & Ostaszewski, P. (1999). Amount of reward has opposite effects on the discounting of delayed and probabilistic outcomes. Journal of Experimental Psychology: Learning, Memory, and Cognition, 25, 418 427. Killeen, P.R. (2005a). An alternative to null-hypothesis significance tests. Psychological Science, 16, 345 353. Killeen, P.R. (2005b). Replicability, confidence, and priors. Psychological Science, 16, 1009 1012. Madden, G.J., Petry, N.M., Badger, G.J., & Bickel, W.K. (1997). Impulsive and self-control choices in opioid-dependent patients and non-drug using control participants: Drug and monetary rewards. Experimental and Clinical Psychopharmacology, 5, 256 262. Mazur, J.E. (1987). An adjusting procedure for studying delayed reinforcement. In M.L. Commons, J.E. Mazur, J.A. Nevin, & H. 62 Volume 18 Number 1

S.J. Estle et al. Rachlin (Eds.), Quantitative analyses of behavior: Vol. 5. The effect of delay and of intervening events on reinforcement value (pp. 55 73). Hillsdale, NJ: Erlbaum. Myerson, J., & Green, L. (1995). Discounting of delayed rewards: Models of individual choice. Journal of the Experimental Analysis of Behavior, 64, 263 276. Myerson, J., Green, L., Hanson, J.S., Holt, D.D., & Estle, S.J. (2003). Discounting of delayed and probabilistic rewards: Processes and traits. Journal of Economic Psychology, 24, 619 635. Myerson, J., Green, L., & Warusawitharana, M. (2001). Area under the curve as a measure of discounting. Journal of the Experimental Analysis of Behavior, 76, 235 243. Odum, A.L., & Rainaud, C.P. (2003). Discounting of delayed hypothetical money, alcohol, and food. Behavioural Processes, 64, 305 313. Ostaszewski, P., Green, L., & Myerson, J. (1998). Effects of inflation on the subjective value of delayed and probabilistic rewards. Psychonomic Bulletin & Review, 5, 324 333. Petry, N.M. (2001). Delay discounting of money and alcohol in actively using alcoholics, currently abstinent alcoholics, and controls. Psychopharmacology, 154, 243 250. Rachlin, H., Raineri, A., & Cross, D. (1991). Subjective probability and delay. Journal of the Experimental Analysis of Behavior, 55, 233 244. Raineri, A., & Rachlin, H. (1993). The effect of temporal constraints on the value of money and other commodities. Journal of Behavioral Decision Making, 6, 77 94. Skinner, B.F. (1953). Science and human behavior. New York: Mac- Millan. (RECEIVED 10/18/05; REVISION ACCEPTED 6/22/06; FINAL MATERIALS RECEIVED 6/30/06) Volume 18 Number 1 63